The best reason to run a 125B model at home isn't price

Strata puts Alibaba's Qwen3.8-Flash-Next on a gaming PC, and on Alibaba's own numbers it beats February's Opus. The case for running it is control, and the narrow jobs where control matters. Cost does not make it.

Strata is an open-source installer and inference engine, MIT-licensed, created on 24 September 2026. It has more than 25,000 stars on GitHub. Its pitch is one sentence: “Run a 125-billion-parameter AI model on your own gaming PC.” You need a graphics card with 12 GB or more, 32 GB or more of RAM, and about 80 GB of disk.

The model is Qwen3.8-Flash-Next, released by Alibaba on 26 August 2026. People are calling it “at par with Opus 4.6”. That is true on Alibaba’s benchmarks, and it deserves a careful look. But the more useful question for a team is what a model like this is actually good for, because it is not a free replacement for a hosted frontier model.

What Strata does

Qwen3.8-Flash-Next is a mixture-of-experts model: 125 billion parameters in total, about 6 billion active for each token. Strata keeps attention, routing and a cache of the most-used experts on the graphics card. System RAM holds every expert, and the processor works on whichever ones the card does not have. It exposes the result through OpenAI-compatible and Anthropic-compatible local APIs, so coding agents can talk to it. The project explains this in its own docs. The earlier post on this blog explains the general idea of splitting an MoE model between GPU and CPU.

The speeds below come from the project itself, on an RTX 5070 with 12 GB of VRAM, a Ryzen 5 7600 and 64 GB of RAM. They are self-measured, not independent.

How fast Strata writes answers, by model size tokens per second, RTX 5070 12 GB
How fast Strata writes answers, by model size (tokens per second, RTX 5070 12 GB)
MeasureValue
Q2_0 (2.4 bits per weight)94
IQ2_XS79
IQ3_XXS62
IQ3_S (3.5 bits per weight)53

Source: Strata README, speed table (project's own measurement). Smaller packs are faster and lose more quality.

Strata README speed table for an RTX 5070 with 12 GB: the Q2_0 pack writes answers at 94 tokens per second and reads prompts at 2,650 tokens per second.
Strata's own speed table, as published. github.com, captured 16 October 2026.

Two details matter later. First, “Your RAM decides which model fits. 64 GB runs every size.” The card is not the only constraint. Second, nobody has published a result on an RTX 5080 that I could find, and I have not run Strata on mine. That is an open item, not a claim.

At par with Opus 4.6?

Opus 4.6 came out on 5 February 2026. That makes it February’s frontier, not last year’s, and Anthropic has shipped several models since. With that said, here are Alibaba’s own numbers for Flash-Next against Opus 4.6 (Max).

Alibaba's benchmarks: Flash-Next against Opus 4.6 (Max) score, higher is better
Alibaba's benchmarks: Flash-Next against Opus 4.6 (Max) (score, higher is better)
MeasureOpus 4.6 (Max)Qwen3.8-Flash-Next
SWE-bench Pro53.462.5
SWE-bench Multilingual77.581.0
LiveCodeBench88.891.9
GPQA91.391.7
IFBench62.581.3
JobBench36.655.7
Humanity's Last Exam40.035.9

Source: Qwen3.8-Flash-Next model card (vendor-reported). The card says Opus 4.6's SWE-bench Pro score is Anthropic's published figure. HLE is judged by GPT-4o.

Of the nine benchmarks in the card’s table that both models report, Flash-Next leads on eight and trails only on Humanity’s Last Exam. The chart shows seven of them. Three caveats take some of the shine off.

  • The harness is not matched. The card’s footnote says that, except for Opus 4.6 (Max), “for which we report the officially published score, all models are evaluated with the Claude Code harness.” Scores depend on the harness, as the leaderboard post shows.
  • The numbers are the vendor’s. The one independent source I could open, Artificial Analysis, scores Flash-Next 39.8 on its Intelligence Index, “well above average among comparable models (median: 19)”. It also calls the model “very verbose” and slower than average. I found no independent comparison of long agent chains, which is the work where a gap with a frontier model would matter most.
  • Strata does not run the benchmarked model. It runs a 2.4 to 3.5 bits-per-weight quantisation. On LiveCodeBench v6, the quantiser’s own tests give 87.43 for the original weights, 81.14 for the 2.4-bit pack and 86.86 for the 3.5-bit pack. Averaged over their three tests the packs sit near parity, but they were not tested on agentic or long-context work.

So “at par” holds on Alibaba’s coding and office benchmarks, roughly on GPQA, and fails on Humanity’s Last Exam. For anything else, run your own task.

At par on the vendor’s benchmarks is a reason to test, not a reason to switch.

Where local models earn their place

The price argument is weak. Artificial Analysis lists hosted Flash-Next at $0.15 in and $0.47 out per million tokens, while Opus 4.6 costs $5 and $25. Output tokens are 53 times cheaper, a calculated figure. A hosted Flash-class model is the cheap option. Running your own hardware is usually not: a German university hospital measured its on-premise system at €150 per million tokens, “about 10 times the cost of a public cloud solution.”

The strong cases are different.

Narrow, checkable tasks on data that cannot leave. University Hospital Bonn served DeepSeek-R1 through vLLM inside a stack with network segmentation and host-enforced egress filtering. Its governance pathway won approval for “processing unanonymized PHI”, and in a one-week pilot 22 residents and radiologists used it. The finding I find most useful is about task shape: tasks anchored in source text, such as report corrections, were rated most useful, while open-ended conclusion generation caused the most critical errors. A local model does well where you can check the answer against the source.

Work that hosted guardrails refuse. In July, Hugging Face described an intrusion into its systems. Its forensic work involved “real attack commands, exploit payloads, and C2 artifacts,” and “these requests were blocked by the providers’ safety guardrails, which cannot distinguish an incident responder from an attacker.” It ran the analysis on GLM-5.2, an open-weight model, “on our own infrastructure,” and it kept the attacker’s data and credentials in-house as a side benefit. It added: “This is not an argument against safety measures on hosted models.”

A research paper found this is not a one-off. Campbell and colleagues measured models refusing defensive security requests that contain security keywords at 2.72 times the rate of equivalent neutral requests. The refusals peak in system hardening (43.8%) and malware analysis (34.3%).

This is the part I find most persuasive for an individual. A security engineer, a researcher or a lawyer handling sensitive documents can get stuck on a hosted model for reasons that have nothing to do with their intent. A local model does not ask. That is also its risk: a model with no guardrails will help anyone. Organisations need an allow-list of approved models and a check on where weights came from, because Strata’s own docs list an “Uncensored” community build of this model that you set up by hand. The Hugging Face post makes the same point from the other side by saying to have a vetted model “ready before an incident.”

Hugging Face's July 2026 incident post: forensic requests were blocked by providers' safety guardrails, so the team ran the analysis on an open-weight model on its own infrastructure.
The passage in Hugging Face's incident write-up. huggingface.co, captured 16 October 2026.

Access that someone else can switch off. On 30 June CNBC reported that Anthropic had been told to suspend all access to two models “by any foreign national, whether inside or outside the United States,” under a government export directive. Fable 5 was due back for global users the next day. For anyone outside the US, that is a reminder that hosted access is a permission, not a possession.

Owning the hardware

There is a freedom argument here, and it has limits worth stating. A graphics card you own works offline, keeps your prompts on your desk, and does not change when a provider updates a model. That is real. But owning hardware does not make the weights yours. Qwen3.8-Flash-Next ships under the Qwen Community License, not Apache 2.0. For a “Model as a Service or AI Work Assistant business”, the licence says the licensee “shall obtain a separate license from Qwen”. Read the terms before building a product on it.

And Strata itself is a personal workstation tool. By default it answers “one request at a time” and the others wait. A shared deployment needs a gateway, per-user identity, logging and a pinned version, none of which Strata provides, and the project supports only its latest release.

Our take

A gaming PC can now run a heavily quantised copy of a 125B open model at reading speed. On Alibaba’s benchmarks it matches or beats February’s frontier on coding. That is a real shift, but it does not make local the cheap option.

The reasons to run local are control: work that hosted guardrails refuse, data that must stay in the building, and access that a government or a provider can switch off. Use it for narrow tasks whose answers you can check. Keep a hosted frontier model for long agent chains and hard reasoning, and test any “at par” claim on your own task, in your own harness, on the quantisation you will actually run.

I have not yet run Strata on my own machine, an RTX 5080 with 16 GB and 64 GB of RAM. When I do, I will publish the speed, the power draw and a small private task set against a hosted model.

References

  1. Strata GitHub, 16 October 2026.
  2. Qwen3.8-Flash-Next model card Alibaba Qwen, Hugging Face, 26 August 2026.
  3. Qwen3.8-Flash-Next license Alibaba Qwen, Hugging Face.
  4. Qwen3.8-Flash-Next-GSQ-RCO-GGUF ISTA-DASLab, Hugging Face.
  5. Qwen3.8-Flash-Next: intelligence, performance and price analysis Artificial Analysis.
  6. Introducing Claude Opus 4.6 Anthropic, 5 February 2026.
  7. Security incident, July 2026 Hugging Face, 16 July 2026.
  8. Defensive refusal bias (Campbell et al.) arXiv, 2026.
  9. Local LLM deployment at University Hospital Bonn arXiv, 2026.
  10. On-premise LLM at University Medicine Magdeburg PubMed Central.
  11. Anthropic says Trump administration has lifted export controls on Claude Fable 5 and Mythos 5 CNBC, 30 June 2026.