May Papers: xLSTM, Schedule-Free Optimizers, and Multi-token prediction
May is always an eventful time of year for ML researchers, with final ICML paper decisions and ICLR taking place in early May, and NeurIPS submission deadlin...
Recent Posts
April Papers: TriForce, QuaRot & Mixture-of-Depths
For our April selection of AI research papers, there is a clear common thread: efficient LLM inference. But as it happens, ML researchers are showing there a...
A transformer walk-through, with Gemma
Transformer-based LLMs seem mysterious, but they don’t need to. In this post, we’ll walk through a modern transformer LLM, Google’s Gemma, providing bare-bon...
March Papers: Low-Rank Galore & 1.58-Bit Weights
March was a fruitful month for AI research, with plenty of papers for us to choose from. A trend in the work we’ve selected is the pushing of previously publ...
February Papers: Longer RoPEs & Better Quantisation
Improving LLM inference is a key research topic at the moment, and something we’re particularly interested in at Graphcore because of its hardware implicatio...