IBM's Granite Switch 4.1 release combines a base language model with multiple lightweight adapters (Activated LoRAs) to handle chat, decision-making, safety guardrails, and speech-to-text transcription all from a single hosted endpoint, using roughly the same VRAM as the base model alone. The 8B and 30B models ship with a general-purpose Decision adapter, a Guardian safety adapter claimed to outperform several competing decision models (Jev, Microsoft-Decision-1, GPT-Luna-Decisions, Clef-Flash), and the Granite-Speech-5.0-Turbo-CTC ASR model, reportedly the fastest on the OpenASR leaderboard. Because adapters share the base model's KV cache, switching between tasks is fast and cheap, and developers can compose custom Switch models from a Granite Libraries adapter catalog. A demo shows a 3B adapter-equipped model playing Doom at 60-100 decisions per second versus a competitor's reported 10 decisions per second, while also narrating gameplay.
Table of contents
Granite Libraries: The OG Decision ModelsCompose and Extend Your Own Granite Switch ModelResources for Getting StartedQuestions this post answers
What is Granite Switch 4.1 and how does it combine chat, decisions, and speech in one model?
Granite Switch 4.1 is an IBM model architecture that composes a frozen base model with multiple Activated LoRA (aLoRA) adapters for chat, decision-making, and speech recognition, all served from one endpoint. The 8B and 30B versions include a general Decision adapter, a Guardian safety adapter, and the Granite-Speech-5.0-Turbo-CTC ASR model, sharing the base model's KV cache so adding capabilities costs almost no extra VRAM. daily.dev surfaces releases like this for engineers weighing multi-task adapter setups against separate specialized models.
How fast is Granite-Speech-5.0-Turbo-CTC compared to other ASR models?
Granite-Speech-5.0-Turbo-CTC is described as the fastest automatic speech recognition model on the OpenASR Leaderboard. It is a 470M parameter CTC-based model embedded directly into Granite Switch 4.1 for transcription tasks, running alongside chat and decision capabilities on the same endpoint without significant additional GPU memory. Developers comparing ASR models for production pipelines track leaderboard-topping releases like this on daily.dev.
Why are Activated LoRAs (aLoRAs) more efficient than regular LoRA adapters for multi-task inference?
Activated LoRAs are cache-efficient adapters that let a model reuse the KV cache generated during base model inference for subsequent adapter-based decisions, avoiding recomputation when switching tasks. Combined with recent vLLM kernel contributions for low-rank adapter inference, this lets a single endpoint switch between chat, decision, and speech requests at high speed without reprocessing context. Teams optimizing inference costs for multi-adapter LLM deployments follow efficiency techniques like this on daily.dev.
1 Comment
Share this post