IBM's Granite Switch 4.1 packs chat, decision adapters, and speech into one model

IBM released Granite Switch 4.1 this week in 8B and 30B sizes, and the pitch is density: one hosted endpoint serves natural-language chat, general and specialized decision tasks, and speech-to-text transcription, and the composed model takes up approximately the same VRAM on a GPU as the base model alone. It is an update to the Granite Switch project IBM introduced last spring, which built a modular approach to model development around swappable adapters.

One checkpoint, three jobs

Each 4.1 model ships preloaded with three components. A new Decision aLoRA handles general-purpose decision making of the kind the Typesafe ecosystem calls /v1/systemone. A specialized Guardian aLoRA targets safety-related decision tasks, where IBM reports it outperforms on average a set of closed decision models, including Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash. And Granite-Speech-5.0-Turbo-CTC, which IBM says is the fastest ASR model on the OpenASR leaderboard, is embedded for transcription. All three, along with the underlying base model, are composed into a single model.

Why switching between jobs is cheap

The engineering is in the adapters. The Switch adapters are built as cache-efficient Activated LoRAs, so the KV cache created by a base-model generation can be reused for any subsequent adapter-based decision. Add the IBM team's recent vLLM contributions to speed up low-rank adapter inferencing, plus a co-located ASR encoder, and the model can move between a chat request, a decision request, and a spoken request without re-running the base for each job. IBM's public demo runs the 8B model transcribing audio, generating summaries, screening for harmful content, and testing 15 other user-defined flags, all on a single endpoint in real time.

Specialists against generalists

Granite Libraries, IBM's earlier release of specialized task adapters, is where the accuracy argument comes from. IBM's own evaluation compares granitelib adapters with generalist decision models on specialized classification tasks and finds the specialists win on the tasks they were built for. Because the adapters share the base model's KV cache, adding specialists alongside a generalist decision adapter costs almost nothing. IBM published a "race" between Granite-Switch-4.1-30B (frozen base plus four specialized aLoRAs) and Jev-27B (frozen Qwen base plus one generic decision LoRA), and reports the 30B model completed a real-world five-step RAG workflow considerably faster. In a toy example, a custom 3B Granite Switch model plays Doom at 60 to 100 decisions per second, versus the 10 per second Typesafe reported for Jev, and narrates its own gameplay while it does it. These comparisons come from IBM's post; independent runs are the next step before treating them as a settled ranking.

How it runs

The 8B and 30B sizes are meant to fit wherever your workload already runs. The models work in vLLM through a plugin, have native inference support in llama.cpp and downstream platforms, and can slot into existing /v1/systemone API calls while also serving regular chat on a shared prefix cache. You are not locked into IBM's configuration either: anyone can compose their own Granite Switch model by mixing and matching adapters from Granite Libraries, and IBM plans an Adapter SDK for training new adapters in Mellea, its training platform. For a team whose pipeline already mixes chat, classification-style decisions, and transcription on the same traffic, the interesting claim is the cost one: more jobs at the same VRAM. The verification work is yours. The decision-accuracy numbers are IBM's, so it makes sense to run your own classification data before retiring the separate classifiers.