local-inference
Posts tagged “local-inference”.
-
AI Brief, 14 November 2026: the cache that gets faster as its hit rate falls
llama.cpp shipped v1.0.0 on 13 November, its first stable release three and a half years in, and its new expert cache roughly doubles throughput on a small cache while the hit rate drops twelve points. Perplexity's Decider v1.3 account puts a decision model's weights under Apache 2.0, so the fail-open rate no vendor will publish is measurable without one.