Build training corpora from a 100B-document index
The index behind Exa, delivered as a clean, comprehensive corpus you can train on. Draw from 1.4T tracked URLs and the billions of topics about the world, including news, case law, financial markets, research papers, points of interest, code documentation, and more.
We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.