HARDWARE & AI ENGINEERING

M5 Ultra for Local LLMs: 256GB vs. 512GB and DGX Spark

Sep 08, 2026 M5 Ultra Apple Silicon Mac Studio NVIDIA DGX Spark Local LLM Mixture of Experts +3 M5 Ultra Apple Silicon Mac Studio NVIDIA DGX Spark Local LLM Mixture of Experts Speculative Decoding Memory Bandwidth Hardware Benchmarks

256GB vs. 512GB, with single and dual DGX Spark as reference points

Analysis date: September 8, 2026. This evaluation is an analytical buying framework based on published hardware specifications, community benchmarks, and scaling models prior to retail delivery.

Is the M5 Ultra 256GB the most sensible configuration to buy?

The 256GB M5 Ultra is the best-balanced candidate on paper for DeepSeek-V4-Flash-class inference. Its capacity accommodates the referenced checkpoint, subject to runtime and context allocations. Approaching 60 tokens/s remains conditional on speculative acceptance, runtime support, and retail measurements. The 80-core GPU premium should be evaluated through prefill and verification workloads.

What does the 512GB configuration actually deliver for its premium?

It primarily expands checkpoint capacity while retaining the same 1.2TB/s nominal memory bandwidth. The 512GB tier can accommodate the ~365GB GLM-5.3 UD-IQ4_XS checkpoint, subject to runtime and context requirements. It allows larger checkpoints and some less aggressively compressed 4-bit variants, but does not generally make this model’s Q5 or Q6 versions practical. At 60 committed tokens/s under one-forward-per-token decoding, reading 40B uniformly quantized 4-bit active parameters would alone require 1.2TB/s. Speculative verification can commit several tokens per forward, so its costs need a separate model.

Does the NVIDIA DGX Spark remain competitive for local single-user inference?

Dual Sparks remain a candidate for CUDA-based workflows, subject to model quality and workload requirements. A community DSpark recipe reports 60–67 tokens/s on coding tasks and about 40 tokens/s on mixed content, using two networked 128GB systems. These are developer measurements for a specific deployment, not a matched comparison with the Mac checkpoint or a universal 60+ tokens/s result. A single Spark has lower capacity and bandwidth, and large Flash deployments require more aggressive compression. Dual-machine operation also adds networking, sharding, and configuration work.

What Is Measured and What Is Projected

To keep the technical arguments transparent, this analysis separates data into three distinct tiers:

Official Specifications: Apple-reported hardware metrics (1.2 TB/s bandwidth, UltraFusion 4-die MCM packaging) and NVIDIA published architecture data.
Reported Developer Benchmarks: Public developer logs from Hugging Face model cards, GitHub project repositories, and technical forums. These reflect specific checkpoints, quantizations, and runtimes; they serve as empirical reference points rather than universal baselines.
Mathematical Projections: Bandwidth-scaling models and Roofline estimations. Projections are sensitive to assumptions regarding memory bus efficiency and speculative draft acceptance rates.

Evaluation targets: 30 / 60 / 100 committed tokens/s correspond to approximately 33.3 / 16.7 / 10 milliseconds per output token, or 33.3 / 16.7 / 10 seconds for 1,000 output tokens, excluding prefill. The central comparison uses 60 tokens/s as a chosen target, not a universal usability threshold. Time to first token, output quality, and sustained agent behavior matter separately.

1. The Local Inference Rationale: Autonomy and Control

Investing in dedicated local inference hardware is rarely justified solely by comparing token costs against cloud API pricing. For developers and researchers, the primary returns on investment center on operational control:

Data Sovereignty: Processing proprietary codebases, confidential research notes, and private document archives entirely on-device without exposing data to external gateways.
Model Version Control and Customization: Pinning exact checkpoint versions, avoiding upstream behavioral drifts or sudden policy changes, and running specialized community fine-tunes.
Dedicated Infrastructure: Treating local inference as persistent background compute for personal automation, local RAG indexing, and agentic workflows.

When these requirements warrant a dedicated workstation, hardware physics determine which configurations can realistically deliver an interactive experience.

2. Memory Architecture & The MoE Decoupling

Mixture-of-Experts (MoE) architectures decouple two variables that are strictly tied together in dense models: total parameter footprint (how much memory is required to store the model) and active parameter footprint (how much weight data must be streamed to generate a single token).

Target Category	Representative Model Tier	Total Parameters ( $P_{\text{total}}$ )	Active Parameters / Token ( $P_{\text{active}}$ )
Efficient / High-Throughput	DeepSeek-V4-Flash	~284B	~13B
Flagship Capability	GLM-5.3 Tier	~744B	~40B

Parameter totals may differ across official architecture labels, serialized tensor counts, and runtime scanners. For example, the Flash conversion card reports ~305B tensors and the GLM-5.3 GGUF listing reports ~754B parameters. Active-parameter figures are approximate architecture-level values, not exact per-token traffic measurements; auxiliary/MTP tensors and counting conventions must be checked separately.

Physical Memory vs. Runtime Allocations

A model’s raw weight size is only the baseline. A complete deployment must satisfy the full memory constraint:

M_{\text{Weights}} + M_{\text{KV-Cache}} + M_{\text{Runtime}} + M_{\text{Draft-Model}} + M_{\text{OS}} \le M_{\text{Physical}}

Hardware Platform	Physical Installed Memory	Peak Nominal Bandwidth	Memory & Interconnect Topology
M5 Ultra (256GB)	256 GB	1,200 GB/s	Single-system UMA via UltraFusion (4-die MCM)
M5 Ultra (512GB)	512 GB	1,200 GB/s	Single-system UMA via UltraFusion (4-die MCM, 80-core GPU)
Single DGX Spark	128 GB	273 GB/s	Single-system unified LPDDR5x
Dual DGX Spark	256 GB (2 × 128GB)	2 × 273 GB/s	Distributed two-node cluster via network interconnect

DeepSeek-V4-Flash (MXFP4/MXFP8 hybrid, ~167 GB): The referenced checkpoint enters the 256GB capacity range without additional sub-4-bit compression. Usable context depth still depends on runtime cache formats and OS overhead.
GLM-5.3 (UD-IQ4_XS, ~365 GB): This checkpoint requires the 512GB capacity tier in this comparison; long context also requires room for runtime buffers and KV-cache expansion.

The GLM-5.3 quantization files make the capacity boundary concrete:

GGUF Variant	Approximate File Size	Implication for 512GB
UD-IQ4_XS	365GB	Enters the capacity range; runtime and context must also fit
UD-Q4_K_XL	467GB	Nominally below capacity, with tight runtime and system headroom
UD-Q5_K_XL	562GB	Exceeds installed memory before runtime allocations
UD-Q6_K_XL	684GB	Exceeds installed memory
Q8_0	801GB	Exceeds installed memory

These are file sizes, not measured resident-memory requirements or quality rankings. Unsloth GLM-5.3 GGUF files.

Both M5 Ultra GPU tiers retain 1.2TB/s bandwidth. Spark’s 2 × 273GB/s describes two local buses, not a continuous, lossless 546GB/s bus. Distributed performance depends on sharding, network transfers, collectives, and synchronization. Apple specifications; NVIDIA DGX Spark specifications.

3. Decoding Physics: The Bandwidth Budget

For low-batch autoregressive decoding, a Roofline-style lower bound separates memory transfer from compute:

t_{\text{step}} \gtrsim \max\left( \frac{D_{\text{step}}}{BW_{\text{effective}}}, \frac{F_{\text{step}}}{C_{\text{effective}}} \right)

Here $D_{\text{step}}$ is data moved, $F_{\text{step}}$ is compute volume, and the denominators are realized bandwidth and compute throughput. Dependencies, synchronization, and other overhead can increase execution time. For vanilla decoding, one forward commits one token; this equivalence does not apply to speculative verification.

Weight Traffic and Full Implementation Traffic

With uniform 4-bit active weights, the simplified weight-reading estimates are:

D_{\text{Flash weights}} \approx 13\text{B}\times 0.5\text{ B}=6.5\text{ GB/forward}

D_{\text{flagship weights}} \approx 40\text{B}\times 0.5\text{ B}=20\text{ GB/forward}

At 60 forwards/s, these require 390GB/s and 1,200GB/s respectively. For 40B active parameters at an average 4.5 bits, the estimate rises to 22.5GB/forward, or 1,350GB/s at 60 tokens/s. These estimates assume each active weight is read once and exclude other traffic. The gap to nominal bandwidth is not confirmed performance headroom.

Full implementation traffic belongs in the main budget. The ds4-on-Spark developer estimates about 11GB per vanilla token, including routed experts, dense layers, attention-related work, and KV reads, with roughly ±20% uncertainty. This is a specific aggressively quantized Spark implementation. It is neither a measurement of the Mac MXFP4/MXFP8 checkpoint nor simply 6.5GB plus a transferable overhead allowance. Developer roofline accounting.

Traffic Assumption	GB / Committed Token	30 tokens/s	60 tokens/s	100 tokens/s
13B active, uniform 4-bit weights only	6.5	195GB/s	390GB/s	650GB/s
Specific ds4-on-Spark full-traffic estimate	~11	~330GB/s	~660GB/s	~1,100GB/s
40B active, uniform 4-bit weights only	20	600GB/s	1,200GB/s	2,000GB/s
40B active, average 4.5-bit weights only	22.5	675GB/s	1,350GB/s	2,250GB/s

All rows assume one forward per committed token. The 11GB row is a separate implementation scenario; the others are simplified weight-only scenarios. Multiplying its traffic estimate by a target rate does not predict Spark or Mac inference speed.

The 13B-active weight-only scenario leaves theoretical room at 60 tokens/s, but a full runtime budget still needs measurement. At 100 tokens/s, the illustrative 11GB traffic scenario already requires about 92% of a 1.2TB/s bus. The 40B scenarios exceed nominal bandwidth at 100 tokens/s even before other traffic. Increasing committed tokens per verification is therefore a central route to higher throughput under these assumptions.

A Separate Model for Speculative Decoding

A draft proposes tokens, and the target verifies them. Let $A$ be the number of output tokens actually committed per cycle, including any correction or bonus token. A long-run approximation is:

TPS_{\text{committed}} \approx \frac{E[A]}{E[t_{\text{draft}}+t_{\text{verify}}+t_{\text{misc}}]}

D_{\text{per committed token}} \approx \frac{E[D_{\text{verify}}+D_{\text{draft}}]}{E[A]}

The traffic expression omits miscellaneous memory operations, which must be added for a complete budget. Acceptance length, draft cost, verification cost, and content predictability all matter. Verification processes several candidate positions and can reuse weights, but its compute and attention costs are not necessarily those of a single-token forward. More draft depth does not guarantee more speed.

Model-native speculation is available in selected checkpoint/runtime combinations. For the referenced Flash build, the model card specifies embedded MTP tensors, oMLX 0.5.4rc2 or later, and explicit mtp_enabled configuration. This does not establish support for arbitrary MLX/GGUF conversions or other MoE architectures.

Figure 2 budgets vanilla traffic. Figure 3 scales an already measured DSpark committed-token baseline under fixed workload assumptions. Its points do not follow directly from the weight-only multiplication in Figure 2.

4. Empirical Benchmarks and Bandwidth Projections

Reported Reference Benchmarks

These are reference points across different checkpoints, quantizations, engines, and prompts. They do not establish a platform ranking at equal model quality.

Hardware / Source	Checkpoint and Conditions	Reported Single-Request Decode	Quality / Fidelity Evidence
M3 Ultra 256GB, 80-core GPU · model card	Flash-0731 MXFP4/MXFP8; oMLX 0.5.4rc2; 23K cached context	26.1 t/s vanilla; 41.7 t/s native DSpark	Conversion reports bit-identical weights; this does not establish cross-quantization task equivalence
Single DGX Spark · ds4-on-Spark README	Flash with ~2-bit expert mix; versioned developer tests	Plain decode: 20 t/s at 2K, 17.7 at 48K; an earlier speculative suite averaged 27.7 t/s	Aggressive compression; matched quality against the Mac checkpoint is not established here
Dual DGX Spark · original forum report	Flash DSpark; TP=2; warmed, thinking off; recipe-specific weights and KV format	60–67 t/s coding; ~40 t/s mixed content	Different recipe and task mix; no matched quality comparison supplied here

The single-Spark entries come from different versioned tests, so their ratio is not a controlled speculation speedup. Likewise, later dual-Spark recipes should be treated as separate records rather than silently replacing this report’s conditions. Historical GLM-5 Q4 results cannot validate GLM-5.3; a comparable GLM-5.3 measurement remains an evidence gap.

The M3 benchmark uses output-length slopes to cancel prefill. It measures decode at cached context, not request-to-first-token latency. Full model perplexity is not published in the cited conversion card. Bit-exact format conversion preserves source weight values; it does not demonstrate that native MXFP4, a ~2-bit expert mix, and IQ4_XS deliver equivalent capabilities.

Bandwidth Scaling Sensitivity Scenarios

The M3 Ultra test platform operated on an 819 GB/s bus, whereas the M5 Ultra provides 1,200 GB/s, giving a theoretical bandwidth multiplier of:

r = \frac{1,200}{819} \approx 1.465

To evaluate how generation speed scales if only memory bandwidth increases, let $f$ represent the hypothetical proportion of per-step execution time that scales directly with memory bandwidth:

R_{\text{new}} = \frac{R_{\text{old}}}{(1 - f) + f / r}

These scenarios represent mathematical sensitivity models, not measured hardware limits. They isolate bandwidth scaling and do not account for architectural IPC improvements, larger caches, or kernel-level enhancements.

Interpretation and Limits

For $f=0.5$ and $f=0.8$ , the DSpark baseline projects to 49.6 and 55.9 tokens/s; fully bandwidth-scaling execution gives 61.1 tokens/s. Vanilla projects to 31.0 / 35.0 / 38.2 tokens/s. These are scenarios, not confidence intervals, measured M5 results, or a claim that every workload will fall inside the range.

Applying the same scaling formula to DSpark assumes an unchanged acceptance distribution, draft configuration, and workload. It does not separately predict how M5 changes draft and verification costs. Additional GPU compute, Neural Accelerators, cache behavior, and kernels may change results, but no uplift is assigned here without measurements. The dual-Spark coding/mixed-content difference reinforces why acceptance and task mix must be recorded.

A Common Benchmark and Quality Protocol

A purchasing comparison should publish reproducible configuration records and evaluate speed alongside retained capability:

Dimension	Required Controls or Measurements
Model identity	Exact model revision, conversion revision, quantization per tensor group, retained experts, and auxiliary/MTP tensors
Execution	Engine commit, kernels, OS/drivers, power mode, sharding, cache formats, and networking configuration
Workload	Same prompts, existing context, output length, temperature, sampling settings, seed where applicable, and thinking mode
Speculation	On/off runs, draft depth, accepted/committed tokens per cycle, draft time, verification time, and rollback overhead
Counting	Count committed output tokens, not speculative steps or proposed tokens; separate single-stream from aggregate throughput
Quality	Perplexity delta plus coding, agent, tool-calling, and long-context retrieval evaluations against the source checkpoint
Stability	Repeated sustained runs with thermal state, failures, output correctness, and run-to-run variation reported

A fast compressed checkpoint is useful only if it retains the capabilities the buyer needs. Missing quality measurements should remain explicitly unknown rather than being replaced by assumptions based on nominal bit width.

End-to-End Experience: TTFT, Prefill, and Agents

Decode speed is only one part of interaction latency. A retail evaluation should report cold model-loading time, cold and warm time to first token (TTFT), and prompt processing at 4K, 32K, and 128K context where supported. Unsupported context lengths should be recorded as such. Test prefix-cache hits, cache misses, and small context changes after tool responses separately.

For coding agents, run 20–50 consecutive turns on representative repositories and measure 2K–8K sustained output alongside total task duration and success. This captures cache behavior, memory growth, and stalls that short decode bursts miss. The 80-core GPU could matter more for prefill, multi-token verification, batching, training, or diffusion than for fully memory-bound vanilla decode; that is a hypothesis to test, not a guaranteed gain.

5. Pricing and Configuration Choices

Pricing for the 256GB configurations is now confirmed, while pricing for the 512GB option remains unavailable. The US Apple Store lists 36-core CPU / 80-core GPU / 256GB / 1TB at USD 10,799. The 256GB memory upgrade adds USD 4,000 to the corresponding 96GB configuration. The 512GB option is scheduled for late October. Prices below are US hardware prices before tax; storage, networking, and regional pricing affect the final cost. Apple Store configuration; availability announcement.

Configuration	US Hardware Price	Bandwidth	Buying Question
M5 Ultra, 30C CPU / 64C GPU / 256GB / 1TB	USD 9,499	1.2TB/s	Does the lower-cost GPU tier satisfy your decode and latency requirements?
M5 Ultra, 36C CPU / 80C GPU / 256GB / 1TB	USD 10,799	1.2TB/s	Does prefill, verification, or other compute-heavy work justify the USD 1,300 premium?
M5 Ultra, 36C CPU / 80C GPU / 512GB	Unavailable	1.2TB/s	Is a larger checkpoint essential, with adequate runtime headroom?
Single DGX Spark, 128GB	USD 4,699	273GB/s	Can smaller or more compressed checkpoints meet your quality needs?
Dual DGX Spark, 2 × 128GB	USD 9,398 subtotal	2 × 273GB/s, distributed	Does CUDA compatibility justify the network and operational overhead?

At USD 9,499, the 64-core M5 Ultra 256GB costs just USD 101 more than two DGX Spark units at their combined MSRP of USD 9,398, before tax and Spark interconnect costs. The dual subtotal is not a bundle quote, and similar hardware prices do not imply equivalent model quality or performance.

The 80-core GPU has 25% more GPU cores than the 64-core configuration, but both have the same nominal memory bandwidth. A fully memory-bound single-token workload should not be assumed to gain 25% throughput. Compute-heavy gains depend on kernel support and utilization. The Mac prices also cannot be evaluated against dual Spark on speed alone when checkpoint fidelity and test conditions differ.

Runtime Maturity Is a Purchase Criterion

Capacity does not establish architecture support, efficient Metal/CUDA kernels, correct MTP loading, optimized sparse/linear attention, stable KV caches, or output parity. The documented Flash/oMLX path is specific evidence for one combination. Its maturity does not transfer automatically to GLM-5.3 or future models. Before buying for a particular checkpoint, confirm that its intended runtime can load it, execute it correctly, and sustain your context and workload on the intended hardware.

Distributed Alternatives on Both Platforms

Apple also supports multiple Macs connected through Thunderbolt 5 and RDMA. Its launch announcement claims up to 3× inference performance with four Mac Studio systems in its selected test. This is vendor evidence for a distributed workload, not a GLM-5.3 single-stream benchmark or proof of cost efficiency.

Both multi-Mac and dual-Spark deployments require compatible runtimes and effective model partitioning. Spark’s ConnectX-7 networking and Mac’s Thunderbolt links connect separate physical memory domains; neither makes remote memory equivalent to local bandwidth. Account for collectives, layer synchronization, topology, and orchestration when measuring scale-out gains.

Purchasing Decision Matrix

Configuration	Conditional Primary Fit	Runtime Maturity / Software Risk	Evidence Needed Before Purchase
M5 Ultra 30C / 64C / 256GB	Flash-class capacity and price priority; vanilla decode candidate	Flash has a documented oMLX path; retail M5 kernels and behavior need validation	Same-checkpoint decode, TTFT, context fit, quality, and sustained agent runs
M5 Ultra 36C / 80C / 256GB	Prefill, MTP verification, batching, and other compute-heavy work	Extra compute helps only where the runtime uses it	Controlled 64-core/80-core comparison; no assumed 25% decode gain
M5 Ultra 36C / 80C / 512GB	Larger checkpoints and selected 4-bit GLM-5.3 variants	Capacity fit alone does not establish efficient architecture/kernel support	Runtime memory at target context, retained quality, speed, and final price
Dual DGX Spark	CUDA workflows and predictable coding workloads	Recipe-specific DSpark support plus multi-node setup and maintenance	Matched checkpoint quality, acceptance, TTFT, and mixed-content throughput
Single DGX Spark	Lower entry cost, smaller models, or acceptable aggressive compression	Large Flash deployments depend on specialized quantization and engines	Task quality after compression and sustained latency at usable context

256GB remains the performance-balanced candidate on paper; 512GB is the capacity-oriented choice. Neither is a definitive recommendation before retail testing. Separate the two 256GB GPU tiers, compare retained model quality as well as throughput, and validate vanilla decode, speculative decode, TTFT, long-context behavior, and sustained agent workloads. The right purchase is the configuration that meets those requirements at an acceptable total cost.