DataChain

DataChain builds Python tools for researchers and AI engineers to process, version, and reuse datasets containing video, images, sensor data, and documents. Its Data Context Layer makes datasets searchable by schema, statistics, and LLM summaries. Coding agents including Claude Code, Cursor, and Codex can inspect schemas, previews, and lineage before writing pipelines, helping them build on existing results instead of repeating earlier work.

Co-founders Dmitry Petrov, CEO, and Ivan Shcheklein, CTO, previously built DVC at Iterative AI. In 2025, lakeFS acquired the DVC open-source project from Iterative.ai and assumed its stewardship; the transaction covered DVC, not DataChain. DataChain’s engineering approach combines Pydantic schemas, typed file references, and immutable versioned datasets. Saving a dataset automatically records its parents, source code, author, and time, preserving the provenance needed to understand how a result was produced.

The product separates stored files from their context: original bytes remain in object storage, while a database manages metadata and dataset history. Python operations translate into SQL against SQLite or ClickHouse. Free open-source software provides a local starting point; DataChain Studio and enterprise offerings add centralized dataset management, access controls, and distributed CPU/GPU compute in the customer’s cloud. On-premises deployment is also available.