▶ Watch ↗AI Engineer World's Fair 202618:16
The Best Models Still Reason Like Toddlers — Andrew Dai, Elorian
Read the full talk →Key ideas
Scroll to read ↓Andrew Dai traces counting and video-tracking failures to the gap between recognizing a scene and reasoning through its details, then explains Elorian’s approach to visual thinking and its proposed uses in robotics, construction and mechanical design.
- Recognizing an object can supply the wrong answer when a task requires inspecting its visible parts. The partial chessboard and Catan examples show familiar patterns replacing direct counts.1:29 ↗
- Visual reasoning includes tracking changes over time. A robot video requires retaining earlier actions and noticing later ones, beyond identifying the objects in view.3:41 ↗
- Evaluate whether the image’s detailed relationships are necessary to answer the question. Tiny visual tasks and questions answerable from text or rough recognition do not establish readiness for complex visual work.6:19 ↗
- Visual intermediate steps can keep reasoning attached to the scene: locate hotel candidates with boxes, then narrow the selection to red hotels.10:36 ↗
- The proposed applications connect visual reasoning to an existing checking or action system: robot planners and controllers, written construction policies, or programmatic and simulation validation for mechanical designs.13:02 ↗