Speeding up LLM inference using SparQ Attention & llama.cpp
With the rapid advances in the capabilities of large language models (LLMs), there is an increasing need for efficient inference platforms that would enable ...
Posts by Tag
- LLMs 25
- efficient-inference 12
- training-dynamics 8
- efficient-training 7
- quantisation 6
- transformers 5
- fine-tuning 3
- mixture-of-experts 3
- sparsity 3
- mup 3
- unit-scaling 2
- GNNs 2
- computer-vision 2
- scaling-laws 2
- inference 2
- long-context 2
- position-embeddings 2
- speculative-decoding 2
- retrieval-augmented-generation 2
- sparse-attention 2
- not-transformers 2
- chip-design 1
- fp8 1
- number-formats 1
- materials 1
- DFT 1
- active-learning 1
- multi-modality 1
- diffusion 1
- automated-theorem-proving 1
- synthetic-data 1
- distributed-training 1
- local-updates 1
- optimization 1
- learning-rate-schedules 1
- RNNs 1
- state-space-models 1
- hallucinations 1
LLMs
Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability
The key idea
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
The key idea
Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
The key idea
Scalable MatMul-free Language Modeling
The key idea
xLSTM: Extended Long Short-Term Memory
The key idea
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
The key idea
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
The key idea
A transformer walk-through, with Gemma
Transformer-based LLMs seem mysterious, but they don’t need to. In this post, we’ll walk through a modern transformer LLM, Google’s Gemma, providing bare-bon...
G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering
The key idea
FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design
The key idea
Massive Activations in Large Language Models
The key idea
Simplifying Transformer Blocks
The key idea
ChipNeMo: Domain-Adapted LLMs for Chip Design
The key idea
efficient-inference
Speeding up LLM inference using SparQ Attention & llama.cpp
With the rapid advances in the capabilities of large language models (LLMs), there is an increasing need for efficient inference platforms that would enable ...
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
The key idea
Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
The key idea
Scalable MatMul-free Language Modeling
The key idea
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
The key idea
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
The key idea
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
The key idea
training-dynamics
Scale-preserving nonlinearities for u-μP
My colleagues and I always get excited when, every once in a while, deep learning research throws up a fun little maths problem. Our recent work on u-μP does...
Sparse maximal update parameterization: A holistic approach to sparse training dynamics
The key idea
The Road Less Scheduled
The key idea
Massive Activations in Large Language Models
The key idea
Simplifying Transformer Blocks
The key idea
Almost-scaled dot-product attention
TL;DR: Scaled dot product attention isn’t properly scaled, and that’s a good thing!
efficient-training
Mixture of a Million Experts
The key idea
quantisation
Scalable MatMul-free Language Modeling
The key idea
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
The key idea
transformers
Speeding up LLM inference using SparQ Attention & llama.cpp
With the rapid advances in the capabilities of large language models (LLMs), there is an increasing need for efficient inference platforms that would enable ...
A transformer walk-through, with Gemma
Transformer-based LLMs seem mysterious, but they don’t need to. In this post, we’ll walk through a modern transformer LLM, Google’s Gemma, providing bare-bon...
Massive Activations in Large Language Models
The key idea
Simplifying Transformer Blocks
The key idea
fine-tuning
mixture-of-experts
Mixture of a Million Experts
The key idea
DiPaCo: Distributed Path Composition
The key idea
sparsity
Speeding up LLM inference using SparQ Attention & llama.cpp
With the rapid advances in the capabilities of large language models (LLMs), there is an increasing need for efficient inference platforms that would enable ...
Mixture of a Million Experts
The key idea
mup
Scale-preserving nonlinearities for u-μP
My colleagues and I always get excited when, every once in a while, deep learning research throws up a fun little maths problem. Our recent work on u-μP does...
unit-scaling
Scale-preserving nonlinearities for u-μP
My colleagues and I always get excited when, every once in a while, deep learning research throws up a fun little maths problem. Our recent work on u-μP does...
Almost-scaled dot-product attention
TL;DR: Scaled dot product attention isn’t properly scaled, and that’s a good thing!
GNNs
G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering
The key idea
Scaling Deep Learning for Materials Discovery
The key idea
computer-vision
scaling-laws
inference
long-context
position-embeddings
speculative-decoding
retrieval-augmented-generation
sparse-attention
Speeding up LLM inference using SparQ Attention & llama.cpp
With the rapid advances in the capabilities of large language models (LLMs), there is an increasing need for efficient inference platforms that would enable ...
not-transformers
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
The key idea
xLSTM: Extended Long Short-Term Memory
The key idea
chip-design
ChipNeMo: Domain-Adapted LLMs for Chip Design
The key idea
fp8
FP8-LM: Training FP8 Large Language Models
The key idea
number-formats
FP8-LM: Training FP8 Large Language Models
The key idea
materials
Scaling Deep Learning for Materials Discovery
The key idea
DFT
Scaling Deep Learning for Materials Discovery
The key idea
active-learning
multi-modality
diffusion
Back to Top ↑automated-theorem-proving
Back to Top ↑synthetic-data
Back to Top ↑distributed-training
DiPaCo: Distributed Path Composition
The key idea
local-updates
DiPaCo: Distributed Path Composition
The key idea
optimization
The Road Less Scheduled
The key idea
learning-rate-schedules
The Road Less Scheduled
The key idea
RNNs
xLSTM: Extended Long Short-Term Memory
The key idea