Google targets AI inference bottlenecks with TurboQuant

The technique aims to ease GPU memory constraints that limit how enterprises scale AI inference and long-context applications ...

AI Inference Takes Center Stage At KubeCon Europe 2026

KubeCon Europe 2026 made AI inference its central focus with major CNCF donations including llm-d, Nvidia's GPU DRA driver ...
DIGITIMES

In-depth: Google TurboQuant cuts LLM memory 6x, resets AI inference cost curve

Google has introduced TurboQuant, a compression algorithm that reduces large language model (LLM) memory usage by at least 6x ...

Red Hat sees inference as AI’s next battleground — with Kubernetes at the core

Red Hat is pushing Kubernetes inference into the mainstream by contributing llm-d to the CNCF, as enterprises race to run AI ...
Semiconductor Engineering

LLM Inference on GPUs (Intel)

“Transformer based Large Language Models (LLMs) have been widely used in many fields, and the efficiency of LLM inference becomes hot topic in real applications. However, LLMs are usually ...
XDA Developers on MSN

Stop obsessing over your GPU's core clock — memory clock matters more for local LLM inference

Your self-hosted LLMs care more about your memory performance ...
NextBigFuture

OpenAI Strawberry LLM Reasoning Needs More Compute and Energy for Inference

Jim Fan is one of Nvidia’s senior AI researchers. The shift could be about many orders of magnitude more compute and energy needed for inference that can handle the improved reasoning in the OpenAI ...
Business Wire

Red Hat Launches the llm-d Community, Powering Distributed Gen AI Inference at Scale

Forged in collaboration with founding contributors CoreWeave, Google Cloud, IBM Research and NVIDIA and joined by industry leaders AMD, Cisco, Hugging Face, Intel, Lambda and Mistral AI and university ...