Is Fine-Tuning Really Dead? A Survival Strategy Read from June 2026’s Verified Signals
As frontier LLMs and agent skills keep improving, the industry has started to feel that fine-tuning is no longer necessary. OpenAI is, in fact, winding down ...
LLMOps
Why an 8x Larger Model Is 5x Cheaper: The Real Structure of LLM Inference Costs
We dissect, with a roofline model, the paradox that the 284B DeepSeek V4 Flash prices its output tokens 5x cheaper than the 35B Qwen3.6. From KV cache reads ...
GLM-5.2 on AMD MI355X at 2,626 tok/s: the serving economics MXFP4 and SGLang built
We examine a case of serving the GLM-5.2 743B MoE model on a single AMD MI355X node at 2,626 tok/s per node, at more than twice the cost efficiency of Blackw...
Intelligence That Runs at Home: The Personal AI Computer and the Economics of On-Prem Serving
Starting from the Personal AI Computer build guides shared by tom_doerr, which reach up to 384GB of VRAM, we work out through calculation how VRAM decides wh...
Build Expensive, Run Cheap: Trimming Skill Costs Every Night
Most agent work doesn’t need a frontier model. Build a skill once with an expensive model, push its format down into code, and you can run the same quality o...
Qwen3.6-27B in NVFP4: The Economics of Single-GPU Blackwell Serving
NVIDIA re-quantized Qwen3.6-27B to NVFP4 so it serves on a single Blackwell GPU with vLLM out of the box. We break down how the mixed precision (MLP in NVFP4...
How to Prompt Claude Fable 5: Five Principles from Anthropic’s Official Guide
We break down Anthropic’s official prompting guide for Claude Fable 5. It lays out five principles: strip out instructions written for older models, audit pr...
How to Systematically Learn LLM Internals: From Tokenization to Inference Optimization
If you operate LLMs in production but can’t explain why the KV cache eats memory or what GQA actually saves, your optimization work is running on intuition. ...
Dual-Pool Token-Budget Routing — Cutting vLLM Inference GPU Time by 31–42% with Two-Pool Scheduling
Short requests waiting behind long ones in a single queue silently waste GPU time — this is Head-of-Line (HoL) blocking. Dual-Pool Token-Budget Routing, prop...
The Year Open-Weight Models Caught the Frontier: Why Self-Hosting Economics Now Decide the Outcome
By mid-2026, open-weight models had closed to within a 3-to-6-month capability gap of frontier labs, and that gap is no longer widening. The real decision ha...
Fitting GLM-5.2 469B on a Single Node with NVFP4: A Deep Dive into the vLLM Serving Recipe
The GLM-5.2-NVFP4 checkpoint published by NVIDIA lets you serve a 469B MoE model on a single Blackwell node (8x RTX PRO 6000) with vLLM. We break down the mi...
Inside slime, the Open-Source RL Framework That Built GLM-5.2
slime, the asynchronous reinforcement-learning infrastructure behind the post-training of Z.ai’s 1M-context open-weight model GLM-5.2, has been fully open-so...
Token Usage Explodes, AI Spend Halves: Coinbase’s Better-Defaults Strategy
Coinbase CEO Brian Armstrong’s recipe for controlling AI cost was not usage caps or spend alerts, but better defaults, routing, and caching. Backed by the fi...
Ornith-1.0: An Open Coding Model That Writes Its Own RL Scaffolds
Released by DeepReinforce on June 25, 2026, Ornith-1.0 is an open-source coding model family built on a self-scaffolding mechanism: during reinforcement lear...
NVIDIA GLM-5.2-NVFP4: Serving a 753B Frontier MoE at 4-Bit Precision
NVIDIA has released a NVFP4 (4-bit) quantization of ZAI’s 753B MoE reasoning model GLM-5.2 on Hugging Face. Rather than blanket-quantizing every weight, the ...
744B in 1 Bit: The On-Premises Question Unsloth’s GLM-5.2 Dynamic GGUF Raises
Unsloth shrank GLM-5.2’s (~744B MoE) 1.51TB of BF16 weights down to 176GB with a 1-bit Dynamic GGUF. A frontier-class open model now fits on a single 256GB M...
NVIDIA Qwen3.6-35B-A3B-NVFP4: Running 35B at 3B Speed with FP4 Quantization
NVIDIA released an NVFP4 (4-bit) quantized version of Alibaba’s Qwen3.6-35B-A3B. Cutting weights from 16-bit to 4-bit reduces disk and GPU memory by about 3....
The Model That Thinks 220K Tokens: How GLM-5.2 Flips the Self-Hosting Math
When Matt Pocock used the /teach skill to have GLM-5.2 solve a Rubik’s cube, even the lowest effort setting produced roughly 220,000 tokens of thinking trace...
Connecting Claude Code to Self-Hosted Models with free-claude-code: A Real-World Test of a 17-Provider Routing Proxy
free-claude-code (36.7k stars) intercepts Claude Code traffic through an Anthropic-compatible FastAPI proxy and routes it to 17 providers. This report docume...
DFlash Speculative Decoding: Up to 15x Faster vLLM Inference with Block Diffusion Drafting
DFlash replaces the autoregressive token-by-token drafter used in EAGLE-3 with a block diffusion drafter that proposes a block of future tokens in a single f...
Routing Claude Code to Your Own Models - Cost-Efficient Coding Infra with claude-code-router
Routing Claude Code traffic across glm-5.2, MiniMax-M2.7, and Kimi K2 with claude-code-router. A hands-on record of verifying all three models live, fixing M...
Don’t Pick an Inference Engine. Pick a Hardware Strategy.
Asking whether to use vLLM, SGLang, or TensorRT-LLM first is the wrong order. One engineer’s hardware-first selection method says: let the hardware you have ...
The Local LLM Inference ‘Bible’: Decide the Hardware First, and the Engine Follows
A breakdown of the free comprehensive local LLM inference guide published by Ahmad Osman, r/LocalLLaMA GPU moderator. From llama.cpp to vLLM, TensorRT-LLM, a...
How We Controlled LLM Operating Costs with Routing and Guardrails - Lessons from a $705/Day Incident
We honestly disclose a $705 single-day billing incident and prove with numbers the root cause found in a one-month audit (Opus main session at 90.3%) along w...
Fine-Tuning LLMs On-Premises Without Exporting Patient Data - Healthcare and Biomedical AI Infrastructure
A complete workflow for healthcare organizations to fine-tune and serve domain-specific LLMs on in-house GPU clusters without sending patient data to externa...
GLM-5.2: An MIT Open-Weight Model That Surpasses GPT-5.5, and the Sovereign Serving Opportunity
A deep look at GLM-5.2, the 744B MoE coding model released under the MIT license by China’s Z.ai (Zhipu). It claims SWE-bench Pro 62.1, beating GPT-5.5 at on...
OPD Post-Training: How GLM-5.2 Merged 10+ Expert Models in Two Days
Building GLM-5.2, Z.ai open-sourced its entire RL post-training infrastructure. We analyze OPD post-training, which merged more than ten expert models in abo...
RL Post-Training as Infrastructure: The slime Open-Source Framework and RL Scaling
slime, open-sourced by Z.ai, is an LLM post-training framework built for RL scaling. This post analyzes the infrastructure behind GLM-5.2’s post-training pro...
753B Open Weights on a Consumer GPU: GLM-5.2 and the Economics of On-Premise LLM Serving
An analysis of running the 753B-scale SOTA open-weight model GLM-5.2 on an RTX 4090 consumer GPU. We cover the DSA sparse-attention kernel port and the econo...
Gemma 4 12B on an 8GB GPU: What QAT and TurboQuant Mean for Consumer Inference Economics
We look at a community benchmark running Gemma 4 12B on an RTX 4060 8GB using QAT and TurboQuant, and unpack what quantization-aware training and consumer-GP...
Cutting LLM Serving Costs in Half on Blackwell GPUs with NVFP4
What advantages NVIDIA Blackwell’s native 4-bit floating-point format NVFP4 offers over H100 FP8, and how to apply it in the vLLM/TensorRT-LLM stack.
Preparing for What Comes After NVFP4: A Complete Guide to Quantization Methods for vLLM Serving (Including Unsloth)
Beyond Blackwell-only NVFP4, this guide covers every quantization method you can serve with vLLM today on Hopper and Ampere – AWQ, GPTQ, FP8, W4A16, compress...
vLLM + Prometheus + Kueue: Building an LLM Serving Observability Stack on Kubernetes
How to build an LLM serving observability stack that monitors GPU utilization, KV cache pressure, and token throughput in a Kubernetes multi-tenant environme...
Buying More GPUs Won’t Fix It: llm-d Distributed Inference and a Heterogeneous Architecture with GPUs and Any NPU/XPU
llm-d is an inference scheduler that gets more requests through the same GPUs rather than buying more. We cover the principles of KV-cache aware routing and ...
The Real Cost of a Single GPU: How ThakiCloud Optimizes AI Inference Spend
GPU depreciation formulas, Kueue gang scheduling, vLLM scale-to-zero, and model-tier routing – all in one place. A code-and-math walkthrough of how ThakiClou...
NVIDIA Nemotron 6 Million Multilingual Reasoning Dataset Released – Strengthening the Open-Source AI Ecosystem
NVIDIA releases a 6-million-example multilingual reasoning dataset, providing high-quality training data expanded across five languages: French, Spanish, Ger...
dots.ocr: SOTA Multilingual Document Parsing with 1.7B Parameters - Complete Analysis
How to implement multilingual document layout analysis and OCR in a single vision-language model using dots.ocr, released by RedNote.