▶ Watch ↗AI Engineer World's Fair 202621:32
Modality Misalignment and Originality Attribution in Short-Form Video — Aditya Gautam, Meta
Read the full talk →Key ideas
Scroll to read ↓Aditya Gautam of Meta explains how specialized vision-language models divide perception, retrieval, and review to catch brief content changes and investigate copied videos—and how training, evaluation, and selective processing make that architecture practical at scale.
- Time-linked clip evidence makes a brief topic change inspectable: the perceiver extracts it, and the reviewer compares it with surrounding content.5:26 ↗
- Offline indexing and clustering prepare the corpus for online retrieval; topics, embeddings, and entities supply different routes to similar clips and authors.6:58 ↗
- Production failures need trace diagnosis before correction: reasoning, tool calls, and retrieval can each cause a poor final result.14:53 ↗
- Choose specialized model sizes and reasoning budgets against domain quality, latency, and cost, while checking the LLM judge against fresh human labels.16:46 ↗
- Frame compression, cached decisions, and metadata pruning save work at different stages; ongoing evaluation and monitoring keep those optimizations useful as the data changes.19:32 ↗