Owen Kephart

Owen Kephart is a software engineer whose work on Dagster addresses two connected problems: understanding how data was produced and deciding when to recompute it. At Elementl, the company behind Dagster, he developed data orchestration tools and authored a proposal to make automatic execution easier to compose, predict, and test. His career also includes Yelp and a Member of Technical Staff role at Vercel.

Making pipeline failures easier to investigate

At Yelp, recovering from a single pipeline failure could consume days, while tracing a data-quality problem could require hours of searching through code. Owen described those difficulties in Dagster’s account of asset-oriented debugging. They give concrete context for his subsequent work: an engineer investigating an incorrect table needs to trace its dependencies and understand which computations actually ran.

Dagster organizes orchestration around data assets—tables, models, and other outputs—with execution history and metadata attached to them. That gives an investigation a starting point in the affected data and a path through the computations that produced it. Owen contributed to this shared platform at Elementl.

His Airbyte and Dagster tutorial explains why orchestration becomes more demanding as a workflow crosses tools. The example ingests GitHub and Slack data through Airbyte, transforms it with dbt, and fits a predictive model in Python. A transformation that depends on several independent ingestion jobs must wait for the right inputs before running; coordinating the workflow also means tracking dependencies across SQL and Python. The tutorial brings those operations together with logs and the history of the tables they update.

Making automatic execution predictable

In July 2024, Owen authored Dagster’s Declarative Automation proposal, addressing limitations the team had encountered in its earlier automatic materialization system. Materialization computes and stores a data asset; automating it requires deciding which changes should trigger that computation and which inputs must be ready first.

The proposal made several consequential choices explicit:

• Composable conditions. The existing rules covered familiar cases but were difficult to combine for unusual requirements. Owen proposed building automation policies from smaller conditions describing an asset’s state, giving developers a way to express requirements beyond the predefined rules.
• A narrower default for historical computation. Adding an asset could previously trigger computation for every historical partition. The proposed eager policy would respond to new events and, by default, target only the latest time partition. This reduced surprise backfills while changing what developers could expect automation to compute automatically.
• Schedules that wait for fresh dependencies. Scheduling a downstream computation at the same time as its inputs could repeatedly produce results from old data. The proposed policy would wait until upstream assets had updated after the scheduled tick, making the schedule depend on input readiness as well as the clock.

These were proposed changes to a system developed with the Dagster team. Owen’s contribution was to articulate the replacement design and its tradeoffs, including the need to test automation policies before allowing them to run unattended. The proposal connects the debugging problem to the execution problem: engineers need both an account of what happened and a policy whose next action they can anticipate.