From Scratch to SOTA: Training a 3B State-Space Vision Model — Krishna Prasad Srinivasan, Sarvam

A document model cannot learn from knowledge that never became machine-readable. Krishna Prasad Srinivasan of Sarvam opens with that problem: Indian-language material exists, but much of it has never been digitized. He cites a language-distribution study placing Indian-language representation well below 1% of Common Crawl, despite India’s importance as a market for frontier models. Digitization is therefore both an immediate application and a way to create material for future training.

The proposed tool is Sarvam Vision, a three-billion-parameter vision-language model for English and 22 official Indian languages, small enough to run on a single GPU. Srinivasan reports that it beats models 100 times larger on document AI. That comparison is a reported task-specific result; it does not establish superiority across general vision or language tasks.

The difficulty has three parts:

• Coherent extraction: A pile of recognized text is insufficient if its logical relationships disappear. The goal is knowledge extraction, which requires preserving how document components belong together.
• Script representation: The visible shapes in Indian scripts can correspond to complex combinations of Unicode elements. Recognizing a shape and producing the correct text representation require language competence as well as vision.
• Scarce training data: Most of the supported Indian languages are low resource, making the labeled examples needed to teach that mapping hard to obtain.

When development began in late 2025, the team saw most OCR vision-language models handling entire pages as monolithic inputs. Sarvam chose block-level OCR instead. A document harness supplies two distinct functions—layout and reading order—around the small recognition model. This divides the work: the surrounding system handles page organization, while the VLM recognizes individual blocks. Srinivasan describes subsequent 2026 releases as converging on this harness-plus-small-model approach.

The language backbone makes a second departure: it uses a state-space model, or SSM. In the comparison presented here, transformer attention lets every token look at every other token, creating (L \times L) interactions for a sequence of length (L). An SSM instead carries a state through the sequence and updates it token by token. Its sequence-processing compute grows linearly, and the carried state stays constant in size as the sequence grows. Constant state memory should not be read as a claim that every part of training or inference consumes constant memory.

Why does the difference matter for OCR? A page can produce 5,000–10,000 visual tokens. All-pairs interactions become expensive at that length. The following comparison shows where earlier information lives: attention exposes interactions among tokens; the SSM carries information forward through an updated state. That compression can produce what Srinivasan calls “lossy recall.” Block-level recognition makes the tradeoff more acceptable for this application, in exchange for avoiding the high inference cost of long transformer sequences.

The four-stage curriculum begins with text, rather than document images. Sarvam reports pretraining the three-billion-parameter language backbone on 13 trillion tokens spanning English, Indian languages, math, and code. The purpose is a strong language prior: when the image leaves a word ambiguous, the model already knows which words fit the surrounding text.

Consider the talk’s half-blurred word example. Some of the word’s visible shape survives, but the image alone does not make the reading clear. Knowledge of the language constrains what could belong at that position, allowing the surviving visual evidence and surrounding text to resolve the ambiguity. The intended observable change is an ambiguous image becoming a readable word. It is an explanation of the training rationale, rather than a demonstrated before-and-after result.

The remaining stages add capabilities in order:

• Continual pretraining: 300 million image-text pairs teach the language model to interpret pixels and develop general vision capabilities.
• Supervised fine-tuning: 100 million OCR samples specialize that general VLM for English and all 22 Indian languages. The examples include tables, equations, and handwriting, so specialization extends beyond ordinary printed prose.
• Reinforcement learning: The final stage seeks improvements beyond what supervised fine-tuning achieves, using machine-verifiable OCR rewards.

The curriculum itself is a familiar recipe. Sarvam’s claimed advantage lies underneath it: the data engine and evaluations. For most of the 22 languages, labeled data is not readily available. The team builds synthetic-data pipelines and derives training material from real documents, then uses evaluation performance to improve what goes into training. Evaluations have to measure usefulness to end users as well as leaderboard performance. The talk also points toward agentic vision capabilities in future releases, extending the work beyond the current OCR focus.

OCR is attractive for reinforcement learning because many correctness checks can run automatically. The described loop samples a group of outputs, scores them with unit tests, and reinforces performance relative to the group’s average before repeating. This is reinforcement learning with verifiable rewards, or RLVR. It makes repeated scoring scalable because the checks can run as software.

How does a test become a training signal? The diagram follows the group through scoring and reinforcement. The important relationship is the feedback: generating outputs is only the first step; their measured quality changes what the model is encouraged to produce in the next round.

Srinivasan reports state-of-the-art performance at launch on two global English document benchmarks, while acknowledging that later releases have improved the field’s English results. He separately reports a continuing lead across 22 Indian languages against frontier models. These are Sarvam’s reported comparisons; the promised public Indic benchmark, discussed later, was still forthcoming in the recording.

The model powers an agentic document workbench with human-in-the-loop digitization and extraction. Confidence scores expose uncertainty; block-level grounding connects extraction to document blocks; agentic proofreading supplies another review capability. Those features carry the block-based design into the user workflow, where a recognized result can be inspected before downstream use.

Insurers, banks, governments, and historical-preservation organizations are using Sarvam Vision to digitize more than 35 million pages across English and the 22 Indian languages, Srinivasan reports. That is the scale of the digitization work being undertaken, rather than a stated count of completed pages. Deployment options include an API, on-premises use, and the agentic platform. The presentation closes by describing pricing as highly competitive and general-purpose VLMs with broader vision capabilities as the next planned step.

The Q&A first broadens the multilingual claim: Srinivasan says competence across the 22 languages improves English as well, and suggests the effect applies beyond Indian low-resource languages. He keeps a proposed connection to coding outside this project’s scope. The model’s present job is extracting information and knowledge from documents.

Synthetic documents support both supervised fine-tuning and reinforcement learning. For RL, however, Srinivasan recommends complex real-world documents that support useful tests. The reward examples target different failure modes:

• Character error rate: Score the fidelity of recognized characters.
• Table structure: Check whether the output preserves the table’s organization.
• Equations: Test extraction of mathematical document content.
• Language and grammar: Apply checks pertinent to the language being processed.

The practical choice is to construct rewards around the document behavior that needs improvement, then use those rules for iterative training.

The planned public Indic benchmark covers 22 languages and documents from the 1800s through modern times. Its scope includes different layouts and document types: prose, poetry, literature, tables, and finance. Historical range and structural variety matter because useful document intelligence must handle more than contemporary, cleanly typeset text.

A later answer puts a crucial limit on the half-blurred-word example: language knowledge must help recover the document, without rewriting it. If the original contains a mistake, faithful OCR should preserve that mistake. Translation and transliteration therefore are not the extraction objective. Grammar rewards need to be understood alongside this fidelity requirement, rather than as permission to correct source text. Srinivasan also notes that transliteration-derived data is entering OCR training, while leaving its quality and usefulness unresolved.

The final discussion connects three needs: document capability, training data, and sovereignty. Complex Indian-language paperwork needs a model that can process it. Digitizing that paperwork creates material for downstream training, which Srinivasan connects to a longer-term goal of personalized agents in users’ preferred languages. For governments and other sensitive users, adoption also depends on controlling where document data is sent and when and how it is used. On-premises availability gives that requirement a concrete deployment option.

The closing answers add two practical details. Srinivasan describes the training mix as 40% Indic, with the remainder English, math, code, and other material. Deployments across different states are processing regional-language, English, and mixed-language documents. A feedback pipeline has started, with training on deployment feedback described as an upcoming step. That returns the project to its original problem: making documents readable creates both immediate utility and a route to better training data.