Paper-to-video pipeline
Personal project · 2026 to present
Takes an arXiv paper and produces a narrated video with shot planning and animation.
- LLM agents
- Speech synthesis
- Procedural animation
The reason for building this is specific. When I read a paper, the hard part is almost never the math. It is building intuition for the method as a whole: how data moves through it, which step actually matters, why it was designed this way rather than another. Video conveys that far better than text, but nobody has time to make a video for every paper they read.
The pipeline runs in four stages: parse the paper and extract the method, organise it into a script with a narrative order, plan visuals and animation per section, then synthesise narration and render.
Two things are still unsolved. Method diagrams generate at inconsistent quality. And the narration drifts toward restating the abstract instead of actually explaining. The second problem is the harder one, because it is really a question of whether the system understands the paper well enough to teach it.