Our Research
Architectures and algorithms for the next generation of AI.
2024
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
Albert Gu*, Tri Dao*
Based: Simple Linear Attention Language Models Balance the Recall-Throughput Tradeoff
Simran Arora, Sabri Eyuboglu, Michael Zhang, et al.
Zoology: Measuring and Improving Recall in Efficient Language Models
Simran Arora, Sabri Eyuboglu, Aman Timalsina, et al.
2022
Efficiently Modeling Long Sequences with Structured State Spaces
Albert Gu, Karan Goel, Christopher Ré
It's Raw! Audio Generation with State-Space Models
Karan Goel, Albert Gu, Chris Donahue, Christopher Ré
S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces
Eric Nguyen, Karan Goel, Albert Gu, et al.
How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections
Albert Gu, Isys Johnson, Aman Timalsina, et al.