Our Research

Llamba: Scaling Distilled Recurrent Models for Efficient Language Processing

Aviv Bick, Tobias Katsch, Nimit Sohoni, Arjun Desai, Albert Gu

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Albert Gu*, Tri Dao*

Based: Simple Linear Attention Language Models Balance the Recall-Throughput Tradeoff

Simran Arora, Sabri Eyuboglu, Michael Zhang, et al.

Zoology: Measuring and Improving Recall in Efficient Language Models

Simran Arora, Sabri Eyuboglu, Aman Timalsina, et al.

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

Albert Gu*, Tri Dao*

Efficiently Modeling Long Sequences with Structured State Spaces

Albert Gu, Karan Goel, Christopher Ré

It's Raw! Audio Generation with State-Space Models

Karan Goel, Albert Gu, Chris Donahue, Christopher Ré

S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces

Eric Nguyen, Karan Goel, Albert Gu, et al.

How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections

Albert Gu, Isys Johnson, Aman Timalsina, et al.

Model Patching: Closing the Subgroup Performance Gap with Data Augmentation

Karan Goel, Albert Gu, Yixuan Li, Christopher Ré

HiPPO: Recurrent Memory with Optimal Polynomial Projections

Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, Christopher Ré