The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Recurrence and Attention for Long-Context Transformers with Jacob Buckman - #750
Jacob Buckman joins the TWIML AI Podcast to discuss achieving long context in transformers. The episode covers bottlenecks, techniques like windowed attention and power retention, compute architecture reasoning, and Manifest AI's open source projects, Vidrial and PowerCoder.