To submit to the leaderboard, submit a pull request that adds your results to the Markdown table below. The table should be sorted by increasing loss.
Note that your submission can run for at most 24 hours on 2 H100s, and that you must evaluate on the C4 100 domains dataset that we provide. In addition, your data should be exclusively filtered from the common crawl WARC files that we provide.
In your pull request description, you should include:
- The final validation loss that was recorded
- A link to an associated learning curve. You may either upload an image directly to the repo (use the ./images) folder or link to a publicly-viewable plot from a service like Weights and Biases.
- A description of what you did / how you built your dataset.
| Name | Validation Loss | Link |
|---|---|---|
| chengshu | 3.397 | Link |
| Marcel Rød | 3.415 | wandb |
| rishabh | 3.417 | wandb |
| Abhinav Garg (1/5th data, full run pending) | 3.432 | Link |
| Tony Sun | 3.440 | wandb |
| Beicheng Lou | 3.448 | wandb |
| Shijia Yang | 3.450 | wandb |
| Mason Wang | 3.485 | wandb |
| Jason Wang | 3.497 | wandb |
| Sarah Chen | 3.504 | wandb |
| sundararajan | 3.511 | wandb |
| xiongb | 3.511 | image |
| Matty Reed | 3.545 | wandb |
| Sudharsan Sundar | 3.582 | wandb |
| Santiago Hernández (78k steps) | 3.700 | wandb |
| Michael Ryan | 3.823 | wandb |
| mattding | 10.365 | wandb |
