Index of papers

131 reviewed papers · 31 tags

Chameleon: Mixed-Modal Early-Fusion Foundation Models

2024 arXiv multimodal · pretraining · generative
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

2024 arXiv rl · rlhf · reasoning
Mixtral of Experts

2024 arXiv scaling · transformer · efficiency
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action

2024 CVPR multimodal · scaling · generative
3D Gaussian Splatting for Real-Time Radiance Field Rendering

2023 SIGGRAPH vision · generative · efficiency
Adding Conditional Control to Text-to-Image Diffusion Models

2023 ICCV diffusion · generative · vision
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

2023 ICML multimodal · vlm
Consistency Models

2023 ICML diffusion · generative · efficiency
Direct Preference Optimization: Your Language Model is Secretly a Reward Model

2023 NeurIPS rlhf · rl
Efficient Memory Management for Large Language Model Serving with PagedAttention

2023 SOSP efficiency · scaling · attention
Fast Inference from Transformers via Speculative Decoding

2023 ICML efficiency · transformer
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

2023 arXiv attention · efficiency · transformer
Generative Inverse Design of Metamaterials with Functional Responses by Interpretable Learning

2023 generative · diffusion
Let's Verify Step by Step

2023 ICLR rlhf · reasoning
LLaMA: Open and Efficient Foundation Language Models

2023 arXiv nlp · scaling · pretraining
Mamba: Linear-Time Sequence Modeling with Selective State Spaces

2023 arXiv sequence · efficiency
Mastering Diverse Domains through World Models

2023 arXiv rl · generative · scaling
NExT-GPT: Any-to-Any Multimodal LLM

2023 ICML multimodal · generative · vlm
QLoRA: Efficient Finetuning of Quantized LLMs

2023 NeurIPS nlp · efficiency · finetuning
RWKV: Reinventing RNNs for the Transformer Era

2023 EMNLP Findings sequence · efficiency
Segment Anything

2023 ICCV vision · segmentation · foundations
Stochastic Interpolants: A Unifying Framework for Flows and Diffusions

2023 arXiv flow · generative · diffusion
Toolformer: Language Models Can Teach Themselves to Use Tools

2023 NeurIPS reasoning · search
Tree of Thoughts: Deliberate Problem Solving with Large Language Models

2023 NeurIPS reasoning · search
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

2022 NeurIPS nlp · reasoning
Classifier-Free Diffusion Guidance

2022 arXiv diffusion · generative
Constitutional AI: Harmlessness from AI Feedback

2022 arXiv rlhf · rl
DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps

2022 NeurIPS diffusion · efficiency
Elucidating the Design Space of Diffusion-Based Generative Models

2022 NeurIPS diffusion · generative
Flamingo: a Visual Language Model for Few-Shot Learning

2022 NeurIPS multimodal · vlm
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

2022 NeurIPS efficiency · attention
Flow Matching for Generative Modeling

2022 ICLR flow · generative · diffusion
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow

2022 ICLR flow · generative
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers

2022 ICLR efficiency · scaling · transformer
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

2022 ICLR Workshop foundations · reasoning
High-Resolution Image Synthesis with Latent Diffusion Models

2022 CVPR generative · diffusion · vision
Large Language Models are Zero-Shot Reasoners

2022 NeurIPS nlp · reasoning
ReAct: Synergizing Reasoning and Acting in Language Models

2022 ICLR reasoning · policy
Robust Speech Recognition via Large-Scale Weak Supervision

2022 ICML sequence · pretraining · scaling
Scalable Diffusion Models with Transformers

2022 ICCV diffusion · transformer · generative
Training Compute-Optimal Large Language Models

2022 DeepMind nlp · scaling
Training Language Models to Follow Instructions with Human Feedback

2022 NeurIPS nlp · rlhf · finetuning
Video Diffusion Models

2022 NeurIPS diffusion · generative · vision
Decision Transformer: Reinforcement Learning via Sequence Modeling

2021 NeurIPS rl · transformer · sequence
Diffusion Models Beat GANs on Image Synthesis

2021 NeurIPS generative · diffusion · vision
Efficiently Modeling Long Sequences with Structured State Spaces

2021 arXiv sequence · efficiency · foundations
Emerging Properties in Self-Supervised Vision Transformers

2021 ICCV vision · pretraining · transformer
Evaluating Large Language Models Trained on Code

2021 arXiv pretraining · scaling · reasoning
Improved Denoising Diffusion Probabilistic Models

2021 ICML generative · diffusion
Learning Transferable Visual Models from Natural Language Supervision

2021 ICML multimodal · vlm · pretraining
LoRA: Low-Rank Adaptation of Large Language Models

2021 ICLR nlp · efficiency · finetuning
Masked Autoencoders Are Scalable Vision Learners

2021 CVPR vision · pretraining · transformer
Mastering Atari with Discrete World Models

2021 ICLR rl · generative · sequence
RoFormer: Enhanced Transformer with Rotary Position Embedding

2021 arXiv transformer · attention · nlp
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

2021 ICCV vision · transformer
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

2021 JMLR scaling · transformer · efficiency
Zero-Shot Text-to-Image Generation

2021 ICML multimodal · generative · vision
A Simple Framework for Contrastive Learning of Visual Representations

2020 ICML vision · pretraining · embeddings
An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale

2020 ICLR vision · transformer
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

2020 NeurIPS vision · pretraining · embeddings
Denoising Diffusion Implicit Models

2020 arXiv diffusion · generative
Denoising Diffusion Probabilistic Models

2020 arXiv diffusion · generative
End-to-End Object Detection with Transformers

2020 ECCV vision · detection · transformer
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

2020 JMLR nlp · transformer · pretraining
Language Models are Few-Shot Learners

2020 NeurIPS nlp · transformer · scaling
Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

2020 Nature rl · search · generative
Momentum Contrast for Unsupervised Visual Representation Learning

2020 CVPR vision · pretraining · embeddings
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

2020 ECCV vision · generative · foundations
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

2020 NeurIPS search · pretraining · reasoning
Scaling Laws for Neural Language Models

2020 OpenAI nlp · scaling
Score-Based Generative Modeling through Stochastic Differential Equations

2020 ICLR generative · diffusion
The Curious Case of Neural Text Degeneration

2020 ICLR sequence · generative
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

2020 NeurIPS sequence · pretraining · embeddings
A Style-Based Generator Architecture for Generative Adversarial Networks

2019 CVPR generative · gan · vision
Decoupled Weight Decay Regularization

2019 ICLR optimization · regularization
Dream to Control: Learning Behaviors by Latent Imagination

2019 ICLR rl · generative · sequence
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

2019 ICML vision · cnn · efficiency
Language Models are Unsupervised Multitask Learners

2019 OpenAI nlp · transformer · pretraining
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

2019 ICLR foundations · regularization · efficiency
A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play

2018 Science rl · search
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

2018 NAACL nlp · transformer · pretraining
Deep Contextualized Word Representations

2018 NAACL nlp · embeddings · pretraining
Graph Attention Networks

2018 ICLR graph · attention
Group Normalization

2018 normalization · vision
Improving Language Understanding by Generative Pre-Training

2018 OpenAI nlp · transformer · pretraining
Recurrent World Models Facilitate Policy Evolution

2018 NeurIPS rl · generative · sequence
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

2018 ICML rl · policy
Attention Is All You Need

2017 NeurIPS nlp · transformer · attention
Deep Reinforcement Learning from Human Preferences

2017 NeurIPS rl · rlhf
Densely Connected Convolutional Networks

2017 CVPR vision · cnn
Density Estimation using Real NVP

2017 ICLR generative · flow
Inductive Representation Learning on Large Graphs

2017 NeurIPS graph · embeddings
Instance Normalization: The Missing Ingredient for Fast Stylization

2017 normalization · vision
Mask R-CNN

2017 ICCV vision · segmentation · detection
Proximal Policy Optimization Algorithms

2017 arXiv rl · policy
Semi-Supervised Classification with Graph Convolutional Networks

2017 ICLR graph · foundations
Asynchronous Methods for Deep Reinforcement Learning

2016 ICML rl · policy
Deep Residual Learning for Image Recognition

2016 IEEE vision · cnn · foundations
Layer Normalization

2016 normalization · foundations
Mastering the Game of Go with Deep Neural Networks and Tree Search

2016 Nature rl · search
Neural Machine Translation of Rare Words with Subword Units

2016 ACL sequence · foundations
WaveNet: A Generative Model for Raw Audio

2016 arXiv generative · sequence · cnn
You Only Look Once: Unified, Real-Time Object Detection

2016 CVPR vision · detection
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

2015 normalization · foundations
Continuous control with deep reinforcement learning

2015 ICLR rl · policy
Deep Unsupervised Learning using Nonequilibrium Thermodynamics

2015 arXiv diffusion · generative
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

2015 ICCV optimization · initialization
Distilling the Knowledge in a Neural Network

2015 NeurIPS Workshop efficiency · foundations
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

2015 NeurIPS vision · detection
Going Deeper with Convolutions

2015 CVPR vision · cnn
Human-Level Control through Deep Reinforcement Learning

2015 Nature rl
Trust Region Policy Optimization

2015 ICML rl · policy
U-Net: Convolutional Networks for Biomedical Image Segmentation

2015 MICCAI vision · segmentation · cnn
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

2015 ICLR generative · gan · vision
Adam: A Method for Stochastic Optimization

2014 ICLR optimization
Dropout: A Simple Way to Prevent Neural Networks from Overfitting

2014 JMLR optimization · regularization
Generative Adversarial Networks

2014 NeurIPS generative · gan
GloVe: Global Vectors for Word Representation

2014 EMNLP nlp · embeddings
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

2014 EMNLP nlp · sequence
Neural Machine Translation by Jointly Learning to Align and Translate

2014 ICLR nlp · attention
Sequence to Sequence Learning with Neural Networks

2014 NeurIPS nlp · sequence
Very Deep Convolutional Networks for Large-Scale Image Recognition

2014 ICLR vision · cnn
Auto-Encoding Variational Bayes

2013 ICLR generative · vae
Efficient Estimation of Word Representations in Vector Space

2013 ICLR Workshop nlp · embeddings
Representation Learning: A Review and New Perspectives

2013 IEEE TPAMI foundations
ImageNet Classification with Deep Convolutional Neural Networks

2012 NeurIPS vision · cnn
A Fast Learning Algorithm for Deep Belief Nets

2006 Neural Computation foundations · generative
Gradient-based learning applied to document recognition

1998 Proceedings of the IEEE vision · cnn · foundations
Long Short-Term Memory

1997 Neural Computation nlp · sequence
Learning Representations by Back-propagating Errors

1986 Nature foundations · optimization
The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain

1958 Psychological Review foundations

Chameleon: Mixed-Modal Early-Fusion Foundation Models

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Mixtral of Experts

Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action

3D Gaussian Splatting for Real-Time Radiance Field Rendering

Adding Conditional Control to Text-to-Image Diffusion Models

BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

Consistency Models

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

Efficient Memory Management for Large Language Model Serving with PagedAttention

Fast Inference from Transformers via Speculative Decoding

FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

Generative Inverse Design of Metamaterials with Functional Responses by Interpretable Learning

Let's Verify Step by Step

LLaMA: Open and Efficient Foundation Language Models

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

Mastering Diverse Domains through World Models

NExT-GPT: Any-to-Any Multimodal LLM

QLoRA: Efficient Finetuning of Quantized LLMs

RWKV: Reinventing RNNs for the Transformer Era

Segment Anything

Stochastic Interpolants: A Unifying Framework for Flows and Diffusions

Toolformer: Language Models Can Teach Themselves to Use Tools

Tree of Thoughts: Deliberate Problem Solving with Large Language Models

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Classifier-Free Diffusion Guidance

Constitutional AI: Harmlessness from AI Feedback

DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps

Elucidating the Design Space of Diffusion-Based Generative Models

Flamingo: a Visual Language Model for Few-Shot Learning

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

Flow Matching for Generative Modeling

Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow

GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers

Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

High-Resolution Image Synthesis with Latent Diffusion Models

Large Language Models are Zero-Shot Reasoners

ReAct: Synergizing Reasoning and Acting in Language Models

Robust Speech Recognition via Large-Scale Weak Supervision

Scalable Diffusion Models with Transformers

Training Compute-Optimal Large Language Models

Training Language Models to Follow Instructions with Human Feedback

Video Diffusion Models

Decision Transformer: Reinforcement Learning via Sequence Modeling

Diffusion Models Beat GANs on Image Synthesis

Efficiently Modeling Long Sequences with Structured State Spaces

Emerging Properties in Self-Supervised Vision Transformers

Evaluating Large Language Models Trained on Code

Improved Denoising Diffusion Probabilistic Models

Learning Transferable Visual Models from Natural Language Supervision

LoRA: Low-Rank Adaptation of Large Language Models

Masked Autoencoders Are Scalable Vision Learners

Mastering Atari with Discrete World Models

RoFormer: Enhanced Transformer with Rotary Position Embedding

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

Zero-Shot Text-to-Image Generation

A Simple Framework for Contrastive Learning of Visual Representations

An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale

Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

Denoising Diffusion Implicit Models

Denoising Diffusion Probabilistic Models

End-to-End Object Detection with Transformers

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Language Models are Few-Shot Learners

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

Momentum Contrast for Unsupervised Visual Representation Learning

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Scaling Laws for Neural Language Models

Score-Based Generative Modeling through Stochastic Differential Equations

The Curious Case of Neural Text Degeneration

wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

A Style-Based Generator Architecture for Generative Adversarial Networks

Decoupled Weight Decay Regularization

Dream to Control: Learning Behaviors by Latent Imagination

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Language Models are Unsupervised Multitask Learners

The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play