Qwen 3.8 Flash-Next Just Got Faster on DGX Spark: The a5b b12x Stack Upgrade

Last week, I wrote about how Qwen 3.8 Flash-Next saved my NVIDIA DGX Spark from becoming a ridiculously expensive desk ornament, running through a 4-backend shootout between veloGB10, TensorFold, blazux, and Azempati’s a5b.

Out of the bunch, Azempati’s a5b stack quickly became one of my absolute favorites. While veloGB10 held the crown for raw single-stream speed, a5b was the rock-solid workhorse: stable, dependable, and capable of chewing through concurrent streams without falling apart.

Well, the open-source community moves fast.

A few days ago, azampatti dropped a substantial upgrade to the a5b stack. I updated my container, spun up the test harness, and ran it through the exact same benchmark suite.

The result? Single-stream decode jumped by 30% — soaring from ~59 tok/s straight to 77 tok/s. And under a full 8-client concurrent load, the DGX Spark pumped out 217 tokens per second of aggregate throughput.

If you’re running a DGX Spark as a local AI server or multi-agent backend, here is the complete breakdown of what changed, how it performs, and why it matters.

---

What’s Under the Hood: The b12x Recipe

To understand why this update is so fast, we have to look at what changed. It’s not a model retrain — it’s pure systems engineering.

Think of it like keeping the same racing driver (the exact same model weights), but handing them a car whose engine and gearbox were custom-machined for this specific racetrack.

┌────────────────────────────────────────────────────────────────────────┐
│                   A5B SERVING STACK EVOLUTION                          │
├──────────────────────┬────────────────────────┬────────────────────────┤
│ Component            │ Classic Build          │ b12x Build             │
├──────────────────────┼────────────────────────┼────────────────────────┤
│ vLLM Container       │ Saren-Arterius image   │ Eugr's spark-vllm-docker│
│ GPU Kernel           │ Generic vLLM CUDA      │ lukealonso's b12x GB10 │
│ Speculative Decoding │ MTP 3 (3 tokens/step)  │ MTP 5 (5 tokens/step)  │
│ Concurrency Bug      │ Slows at 3,5,6,7 users │ Fixed (tested 9 users) │
└──────────────────────┴────────────────────────┴────────────────────────┘

The four core architectural changes:

1. Eugr’s spark-vllm-docker: The previous build used a general vLLM container from Saren-Arterius via Qwen3.8-Flash-Next-Int4-FAST. The new recipe migrates to Eugr’s Docker image pinned specifically for Grace Blackwell / DGX Spark environments.
2. lukealonso’s b12x GPU Kernel: Rather than relying on generic GPU execution paths that cater to broad hardware targets, the b12x kernel was handcrafted specifically for the GB10 unified architecture, slashing execution overhead.
3. MTP 5 (Multi-Token Prediction): The speculative decoding head previously predicted 3 candidate tokens ahead per step. The b12x configuration pushes that to 5 tokens ahead. When the draft head guesses correctly, the main model accepts multiple tokens in a single forward pass.
4. The Concurrent MTP Bug Fix: In the classic build, there was an annoying glitch: when handling 3, 5, 6, or 7 concurrent sessions, certain threads would suddenly drop to a sluggish 1 token/step for the remainder of their response. The new stack completely fixes this — I verified 9 parallel streams running concurrently with zero degradation.

And remember the model itself: azampatti/Qwen3.8-Flash-Next-125B-A5B-INT4-AutoRound. Azempati didn’t just quantize Qwen down to INT4; he pruned the routed experts from 10 down to 5 and post-healed the weights. You get a 125B total parameter model that activates only 4.8B parameters per token, fitting comfortably into the Spark’s 128GB unified memory with room for a 16GB KV cache.

---

Benchmark Results: Classic vs. b12x

To keep the comparison scientifically honest, both runs used identical parameters on the same machine:

• Hardware: NVIDIA DGX Spark (128GB unified memory), GPU clock locked at 2,200 MHz.
• Inference Config: Context window 262k, max 8 concurrent requests, KV cache 16 GB, temperature = 0, thinking = OFF.
• Methodology: Decode tested at 512 tokens × 3 runs (median value reported). Prefill tested with randomized cache-busting prompts across 0.7k to 84k token lengths.

Here are the side-by-side performance numbers:

1. Speed & Concurrency Shootout

Metric	Classic (Old)	b12x (New)	Delta
Single-Stream Decode (Short context)	59.4 tok/s	77.1 tok/s	+30%
Single-Stream Decode (@ ~10k context)	64.3 tok/s	72.7 tok/s	+13%
Single-Stream Decode (@ ~42k context)	59.3 tok/s	73.9 tok/s	+25%
Inter-token Latency (p50)	0.055s	0.060s	~flat
Inter-token Latency (p99)	0.097s	0.073s	-25% (Smoother)
Aggregate Throughput (1 request)	60.4 tok/s	63.5 tok/s	+5%
Aggregate Throughput (2 concurrent)	84.9 tok/s	103.8 tok/s	+22%
Aggregate Throughput (4 concurrent)	138.5 tok/s	157.2 tok/s	+13%
Aggregate Throughput (8 concurrent)	211.0 tok/s	217.3 tok/s	+3%
Per-Stream Speed @ 8 concurrent	29.5 tok/s	30.4 tok/s	+3%

2. Prefill & Time-To-First-Token (TTFT)

Metric	Classic (Old)	b12x (New)	Delta
Prefill Speed @ 0.7k prompt	1,626 tok/s	1,729 tok/s	+6%
Prefill Speed @ 10.5k prompt	2,225 tok/s	2,263 tok/s	+2%
Prefill Speed @ 31.5k prompt	2,363 tok/s	2,083 tok/s	-12%
Prefill Speed @ 84k prompt	2,193 tok/s	1,981 tok/s	-10%
TTFT @ 84k prompt	38.3s	42.3s	+11% (Slower)

---

3. Did We Sacrifice Intelligence for Speed?

Whenever a new optimization recipe claims double-digit speed gains, the first question experienced practitioners ask is: “What broke?”

Because weights weren’t changed, quality should stay virtually identical. I ran both builds against standard coding, tool calling, instruction-following, and bilingual Thai benchmarks:

Benchmark	Classic	b12x	Difference	Verdict
IFEval (541 prompts, strict)	83.4%	81.9%	-1.5%	Tie (Within margin)
BFCL v4 Tool Calling (207 tasks)	86.0%	88.4%	+2.4%	Tie
LiveCodeBench v6 (100 problems, pass@1)	96.0%	95.0%	-1.0%	Tie
MMLU-ProX (Thai) (392 questions)	65.3%	66.3%	+1.0%	Tie
ThaiExam (300 questions)	57.3%	57.7%	+0.4%	Tie
Writing Evaluation (Thai + EN, n=40)	6.92 / 10	7.20 / 10	+0.28	Slight Win
— Thai Subscore (n=20)	6.40 / 10	6.95 / 10	+0.55	Slight Win
— English Subscore (n=20)	7.45 / 10	7.45 / 10	0.00	Tie

Note: Following the 5-point threshold rule, any difference under 5% is a statistical tie. The slight improvement in Thai writing scores is largely thanks to the b12x recipe setting presence_penalty: 0.5 by default, which noticeably curbs Thai loop repetitions.

Zero regressions. You get the 30% speed boost with 100% of the cognitive horsepower intact.

---

The Big Takeaways

1. 75+ tok/s Single Decode Closes the Gap with veloGB10

In my previous shootout, veloGB10 felt like an untouchable speed demon at 104 tok/s, while a5b lingered at ~50–59 tok/s. With b12x pushing a5b into the 70–77 tok/s range, the gap has narrowed significantly. In daily coding, 75 tok/s feels instantaneous. Diff generation is so rapid that the UI streams without any perceived stutter.

2. High Concurrency is Where a5b Truly Shines

If you’re only ever running a solo chatbot for personal use, veloGB10 is still the single-stream sprint champion. But if you’re using your DGX Spark the way it was designed to be used — as an AI server for your workspace, local microservices, or multiple background coding agents running parallel tasks — a5b is unmatched.

Hitting 217 tok/s aggregate throughput across 8 parallel streams (with each stream still clocking over 30 tok/s) means your machine doesn’t choke when three agents ping it at the exact same moment.

3. The Only Trade-off: Massive Prompt TTFT

The only blemish on the score sheet is prefill at very long context windows (>30k tokens). At 84k context depth, waiting for the first token took 42 seconds on b12x versus 38 seconds on classic (an ~11% slowdown). For typical day-to-day prompts under 10k tokens, TTFT is virtually identical (~4.6 seconds), but if your primary workload is massive document ingestion, keep that nuance in mind.

---

Final Thoughts: The Joy of Local Compute

This upgrade reminded me why tinkering with local AI hardware is so exhilarating right now.

I didn’t buy a new GPU. I didn’t upgrade my memory. I didn’t even download a new model weight checkpoint.

The exact same silicon and the exact same model became 30% faster purely because brilliant open-source engineers optimized the runtime stack and tailored the kernels for the GB10 chip.

Huge shoutouts to azampatti for the expert pruning, healing, and recipe curation; Eugr for spark-vllm-docker; lukealonso for the b12x kernel; Saren-Arterius for the foundational vLLM build; and the teams at Qwen and Intel (AutoRound).

If you want to try this out or update your own rig, you can find the scripts, docker setup, and launch configs in azampatti’s repository:
👉 github.com/azampatti/Qwen3.8-Flash-Next-Int4-FAST

If you’re running Qwen 3.8 Flash-Next on a DGX Spark, definitely give the b12x stack a spin. Your terminal will thank you. 🚀

---

Disclaimer: All tests were performed on my personal DGX Spark with GPU clock fixed at 2,200 MHz. Datasets and evaluation scripts mirror the methodology from my October 5th benchmark post.