LOCAL AI / REAL-WORLD BENCHMARKS
Qwen3.8-Flash-Next Q5 on a 280W RTX 3090: 512K Context Results
One RTX 3090 capped at 280W, with GPU Unlock reported enabled, 192GB RAM and a 512K YaRN context. The key Strata results, plus what was actually measured.
The results at a glance
All 80/80 structured reasoning tests passed (486,609 tokens maximum). A captured 12-request runtime sample recorded 59.5–83.5 tok/s decoding (median 65.95). A separate longer-workload observation was approximately 77–88 tok/s. The GPU had a 280W cap confirmed by telemetry; GPU Unlock is reported enabled, with boot verification still to be archived.
80/80 Correct tests |
486.6K Max prompt tokens |
280W GPU power cap |
Test system
| GPU | NVIDIA RTX 3090, 24GB (power-limited to 280W) |
| GPU Unlock | Enabled — Zagitta/gpu-unlock (as reported for this system) |
| CPU | Intel Xeon Gold 6248, 20 cores / 40 threads |
| Memory | 192GB DDR4 ECC, 2666 MT/s |
| GPU connection | PCIe 3.0 x16 |
| Inference engine | Strata |
| Model | Qwen3.8-Flash-Next (Unsloth UD-Q5_K_XL) |
| Context / KV | 524,288 tokens (512K with YaRN) / INT8 KV cache |
| Engine / cache | Strata v0.1.58; 3,997 GPU expert slots (~15 GiB); 8 GiB / 4-slot conversation cache |
Strata uses GPU VRAM alongside system RAM for MoE expert storage and inference. My configuration is designed to avoid SSD-backed expert offloading.
Observed inference performance
| Metric | Observed |
|---|---|
| Decode speed in 12 captured runtime requests | 59.5–83.5 tok/s (median 65.95) |
| Live long-context prefill indication (separate snapshot) | ~2,000 tok/s (not controlled) |
| Maximum observed prompt | 486,609 tokens |
| GPU VRAM used | ~23.6GB / 24GB |
| Sustained GPU temperature | ~77°C |
Telemetry is a snapshot, not a controlled identical-request speed test. An earlier 77–88 tok/s long-workload observation is kept separate because conditions differ. The 80/80 suite measures end-to-end time, not pure decode throughput.
The same archived Strata snapshot showed 8 GiB conversation cache across four slots, 79.4% token reuse across 14 requests, and 3,997 GPU-cached experts (15,251 MiB). The MTP drafter was active; 79.0% of offered drafts were accepted in the cumulative snapshot. Warm prompt caches can heavily affect response times.
Reasoning quality: 4K to 480K
I wanted to test whether long context remained useful, not just whether the server accepted it. The benchmark covered multi-hop chains, conflicting records, temporal reasoning, incomplete evidence and multi-record integration. Critical information appeared near the start, middle and end of each prompt.
| Target context | Correct | Median end-to-end time |
|---|---|---|
| 4K | 10/10 | 14.1s |
| 32K | 10/10 | 26.8s |
| 128K | 15/15 | 77.7s |
| 256K | 15/15 | 150.6s |
| 384K | 15/15 | 231.6s |
| 480K | 15/15 | 310.3s |
| TOTAL | 80/80 | Zero invalid runs |
Why the 280W power limit?
The reference RTX 3090 is rated at 350W. I'm running mine at 280W (80% of stock) to keep power consumption and heat under control. Other LLM inference tests suggest that the efficiency sweet spot can be below the maximum power limit, but I haven't run a controlled power sweep on this exact model and engine.
So these results should be read as 280W results—not proof that 280W is the optimal setting for every RTX 3090.
GPU Unlock: an important benchmark detail
One setting I initially omitted is that this server runs GPU Unlock on the RTX 3090. This project is intended to remove a GeForce tensor-compute limitation affecting certain FP16, BF16 and TF32 operations. It is a separate modification from the 280W power cap, and it does not add GPU memory or increase memory bandwidth.
The impact depends on the workload. Compute-heavy prompt processing may benefit, while quantised MoE decoding can remain constrained by memory movement and expert caching. I did not benchmark GPU Unlock on versus off on this setup, so these results should be understood as a 280W RTX 3090 with GPU Unlock reported enabled, not as a controlled stock-card comparison.
Implementation: Zagitta/gpu-unlock (GitHub). The exact installed revision and a successful boot-time unlock message were not captured. To prove it is actually active, inspect the boot journal, not only the kernel command line. The project primarily targets FP16/BF16/TF32 tensor issue rates; any benefit to this quantised MoE setup remains unmeasured.
Project documentation: Zagitta/gpu-unlock.
Reproducing my Strata configuration
For anyone wanting to reproduce these results, here are the paths, startup command and key engine options from my recorded production setup. These were captured around the November 2026 tests. My Strata engine reported v0.1.58 and served its API on 127.0.0.1:18020.
Production server launch command
/opt/llm/Strata/.venv/bin/python \
/opt/llm/Strata/serve/server.py \
--engine strata \
--config /opt/llm/Strata/strata-unsloth-ud-q5_k_xl-512k.json \
--port 18020The JSON config referenced by --config starts Strata with the model and engine options. The service runs under systemd, so I don't manually enter this command for normal use.
Service note: An earlier systemd capture showed a legacy Q4 base unit overridden by /etc/systemd/system/strata.service.d/90-use-q5.conf. A later effective process/API check confirmed Q5 at 512K. The full current service and JSON file have not been published, so this is a launcher plus recorded effective settings, not a complete export.
Recorded engine options
--max-context 524288
--rope-scaling yarn
--rope-scale 2
--yarn-orig-ctx 262144
--kv int8
--kv-resident 32768
--spec 4
--mtp /opt/llm/Strata-data/mtp/rt
--prefill auto:32768
--expert-cache auto
--ple-io ram
--pcie-frac 0.00
--vram-reserve-mib 700
--spec-min-p 0.70
--adapt-every 1
--adapt-swaps 80
--adapt-decay 0.97
--visionThe key settings here are 2× YaRN scaling from 262,144 to 524,288 tokens, INT8 KV caching, 32,768 resident KV tokens, MTP speculative decoding and RAM-based expert storage with GPU expert caching. --pcie-frac 0.00 was also confirmed in the Strata runtime metrics.
Relevant files and model assets
| File / path | Purpose |
|---|---|
/opt/llm/Strata/strata-unsloth-ud-q5_k_xl-512k.json | Production Strata server configuration |
/mnt/optane/strata-q5-models/unsloth-UD-Q5_K_XL/ | Unsloth UD-Q5_K_XL GGUF model shards |
/opt/llm/Strata-data/packs/unsloth-ud-q5_k_xl | Prepared Strata model pack (recorded installation path) |
/opt/llm/Strata/data/expert-profile.bin | Expert-cache profile |
/opt/llm/Strata-data/mtp/rt | MTP drafter files |
/opt/llm/Strata-data/models/mmproj-Qwen3.8-Flash-Next-BF16.gguf | Vision projection model |
GPU Unlock verification
For anyone reproducing the setup, GPU Unlock should be listed alongside the Strata arguments and power limit. On an Ubuntu installation using the linked project, these commands can help check the boot configuration and kernel log. They were not included in the original benchmark capture.
# Check whether the boot opted in to GPU Unlock
cat /proc/cmdline | tr ' ' '\n' | grep 'gpu_unlock.targets='
# Look for GPU Unlock messages from the current boot
sudo dmesg | grep -i 'gpu_unlock'A boot argument alone does not prove the unlock completed successfully; examine the module's reported status in the kernel log. Keep the GPU Unlock version consistent when running comparisons.
280W GPU cap
Strata telemetry confirmed a 280W GPU power limit. The following command applies the same cap on a supported NVIDIA setup; it is a reproducibility command, not a recovered record of exactly how I originally configured persistence:
sudo nvidia-smi -pl 280
nvidia-smi --query-gpu=name,power.limit,power.draw,temperature.gpu --format=csvHow the reasoning tests were run
I used custom Python scripts to generate structured reasoning tasks at different context lengths, submit them to the local model, and automatically check the answers. The scripts varied the placement of relevant information and tested reasoning chains, conflicting evidence, timelines, missing information and combining multiple records. The tables above summarise the results; these are controlled synthetic tests, not a standard public benchmark or a measure of general coding ability.
The complete production JSON and systemd unit were not preserved with the benchmark results, so the configuration settings above reflect the recorded effective setup rather than a verbatim copy of every live file.
Completed: 80/80 structured reasoning cases (20 shorter-context + 60 extended-context); an additional, simpler three-hop suite was reported as 21/21. These are distinct synthetic tests, not general-world accuracy rates.
Observed: Strata engine and cache telemetry, near-487K prompts, 280W GPU power cap. GPU Unlock enabled is the operator-reported state, not a validated on/off comparison.
Still to test: Qwen Code 64K versus 128K, actual coding/tool success, Hermes PA fresh-session persistence, GPU Unlock on/off, and a controlled power-limit sweep. Historical prefill indications and mixed workload decode snapshots are not directly interchangeable.
What I would actually use day to day
I'll keep the 512K backend available for exceptionally large documents and repository investigations. But for routine tasks, smaller working contexts are more practical, especially when a coding agent can retrieve relevant files as needed.
| Workload | Planned context |
|---|---|
| Everyday Hermes assistant | 128K |
| Qwen Code feature development | 64K |
| Complex debugging/refactoring | 128K–256K |
| Exceptional long-context analysis | Up to 512K |
These are planned working profiles, not separately benchmarked coding-agent configurations.
Next: can local Q5 replace cloud coding?
The next test is Qwen Code at 64K versus 128K using the same model and sampling settings. I'll compare real multi-file implementation, debugging, test execution, tool-calling reliability and how often human corrections are needed. My goal is to use frontier models for difficult planning and review while keeping more implementation work local.
A single RTX 3090 with GPU Unlock enabled and capped at 280W, backed by 192GB RAM, handled a nearly 487K-token prompt and passed 80/80 structured reasoning-quality tests. That's an encouraging result for local long-context AI—but coding-agent reliability is the next challenge.
Tested November 2026. Results are specific to this hardware, reported GPU Unlock-enabled setup, confirmed 280W cap, Q5 quantisation, Strata build and synthetic benchmark. End-to-end latency includes prompt processing and generation; cache effects were not isolated. No benchmark here demonstrates a causal GPU Unlock benefit.
