EleutherAI works on a wide range of research projects.
AlphaFold2 Replication
A replication of AlphaFold2 architecture (very accurate protein structure prediction model) with an open license.
Projects
A listing of EleutherAI projects, past and present.
CARP
Contrastive learning for story critiques.
CLASP
A CLIP-like model for amino acid sequence prediction.
Eval Harness
GPT-Neo
An implementation of model & data-parallel autoregressive language models with Mesh TensorFlow for distributed TPUs.
GPT-NeoX
An implementation of 3D-parallel autoregressive language models for distributed GPUs.
Mesh Transformer JAX
An implementation of model & data-parallel autoregressive language models with JAX and Haiku for distributed TPUs.
OpenWebText2
An enhanced version of OpenWebTextCorpus.
The Pile
A large, diverse, open-source language modeling dataset.