Pin llama.cpp v1.0.0: Major Version Bumps Now Mean ABI Breaks
llama.cpp and ggml both reached v1.0.0 on 2026-11-13. Major bumps now signal C API/ABI breaks. Here are the semver rules, packaging channels and what to pin.
14 posts
llama.cpp and ggml both reached v1.0.0 on 2026-11-13. Major bumps now signal C API/ABI breaks. Here are the semver rules, packaging channels and what to pin.
DeepSeek open-sourced DeepInfer, its production inference framework, under Apache-2.0. It is offline-only, SM100/Ascend 950 only. Env vars and gaps inside.
IBM's Granite Switch 4.1 packs chat, a decision router, and the fastest OpenASR speech model into one 8B/30B endpoint with no extra VRAM, dropping in for /v1/systemone via vLLM and llama.cpp.
Liquid AI's LFM2.5-24B-A2B packs a 24B MoE with only 2B active parameters, 131K context, and 184 tok/s on an M5 Max — enough to keep agent loops fully local.
Mistral Large 4 public preview: 1T-parameter multimodal MoE, 49B active, $1.36/$4.18 per million tokens, weights promised by end of October.
Reflection's Beam is a 501B MoE with 23B active parameters, 80.1 on Terminal Bench 2.1 and 80.9 on SWE-bench Verified. Apache 2.0 weights this month.
Strata runs Qwen3.8-Flash-Next, a 125B MoE, on 12GB consumer cards: 94 tok/s write speed at Q2_0 on an RTX 5070, with an OpenAI/Anthropic-compatible server on localhost.
Fireworks Research's Ember-1 is a Kimi K3 derivative that drops 35-50% of reasoning tokens at matching accuracy, priced against K3's $3/$15 per M token rates.
Google's eighth-gen TPUs split into TPU 8t for training (121 ExaFlops, 9,600 chips) and TPU 8i for inference (288GB HBM, 80% better perf-per-dollar).
NVIDIA's NVHBM moves memory control to the HBM stack itself, not the GPU. This architectural shift reveals memory bandwidth, not compute cores, is the real scaling bottleneck for AI inference.
vLLM 0.27 adds CondensePyramid V1 attention for multi-modal long context, new token-tier limits on templates, faster ASR CPU preprocessing via multi-threading, and CPU W4A16 INT4 MoE support.
TileRT 1.5 runs a 1T-param Xiaomi model at 1000+ tokens/s on a single 8x B200 node. Here's the exact stack, commands, and where it fits for low-latency serving.
vLLM 0.27.0 (Aug 10) ships JIT warmup and runner-owned Triton warmup that eliminate first-request compile stalls, plus Kimi K3 support — but the torch 2.13.0 upgrade is a breaking change that needs migration testing before you bump.
vLLM v0.26.0 adds runtime draft-weight updates, so you can retune speculative decoding without restarting the engine and flushing the KV cache.