A paper per day for 100 days

I read a paper every day for 100 days, documenting them here. A ⭐ means that I need to come back to the paper after I know more. A ❌ means that it isn't necessary for you to read, as it was kind of a meh read.

1. ImageNet Classification with Deep Convolutional Neural Networks
2. ⭐Attention is all you need
3. Visualizing and Understanding Convolutional Networks
4. A ConvNet for the 2020s
5. ⭐The Matrix Calculus you need for Deep Learning
6. Deepface Closing the Gap to Human-level Performance
7. ⭐Improving Language Understanding by Generative Pre-Training (GPT 1)
8. ⭐Language Models are Unsupervised Multitask Learners (GPT 2)
9. Language Models are Few-Shot Learners (GPT 3)
10. ❌Character-Level Language Modeling with Deeper Self-Attention
11. ⭐Recurrent Neural Networks (RNNs): A gentle Introduction and Overview
12. ⭐RWKV: Reinventing RNNs for the Transformer Era
13. ❌Pytorch
14. DIFFEDIT: DIFFUSION-BASED SEMANTIC IMAGE EDITING WITH MASK GUIDANCE
15. ⭐RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
16. ⭐RT-1: ROBOTICS TRANSFORMER FOR REAL-WORLD CONTROL AT SCALE
17. Platypus: Quick, Cheap, and Powerful Refinement of LLMs
18. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
19. Pointer networks
20. Layer normalization
21. Going Deeper with Convolutions
22. Rethinking the Inception Architecture for Computer Vision
23. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
24. Long Short Term Memory
25. Deep Residual Learning for Image Recognition - Resnet
26. U-Net: Convolutional Networks for Biomedical Image Segmentation
27. Very Deep Convolutional Networks for Large-Scale Image Recognition
28. A Decision-Theoretic Generalization of on-Line Learning and an Application to Boosting
29. Bagging Predictors
30. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
31. Improved Techniques for Training GANs
32. Conditional Generative Adversarial Nets
33. Generative Adversarial Networks (the original paper)
34. Diffusion Models Beat GANs on Image Synthesis
35. Denoising Diffusion Probabilistic Models
36. Understanding Diffusion Models: A Unified Perspective
37. Deep Unsupervised Learning using Nonequilibrium Thermodynamics (the original paper)
38. High-Resolution Image Synthesis with Latent Diffusion Models
39. Hierarchical Text-Conditional Image Generation with CLIP Latents
40. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
41. Adding Conditional Control to Text-to-Image Diffusion Models
42. Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning
43. Greedy function approximation: A gradient boosting machine.
44. XGBoost: A Scalable Tree Boosting System
45. Random Forests
46. Mastering the game of Go with Deep Neural Networks & Tree Search
47. Generally capable agents emerge from open-ended play
48. Highly accurate protein structure prediction with AlphaFold
49. Adam: A Method for Stochastic Optimization
50. Autograd: Effortless Gradients in Numpy
51. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
52. Dropout: A Simple Way to Prevent Neural Networks from Overfitting
53. Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
54. Torch: A modular machine learning software library
55. Automatic differentiation in PyTorch
56. TensorFlow: A system for large-scale machine learning
57. Experiments on Learning by Back Propagation
58. Support Vector Networks maybe idk this one's a bit old
59. Latent Dirichlet Allocation maybe idk this one's a bit old
60. Statistical Modeling: The Two Cultures
61. Textbooks are all you need
62. A Fast Learning Algorithm for Deep Belief Nets
63. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
64. Subject-Diffusion
65. Generating long sequences with sparse transformers
66. Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding
67. Improved Techniques for Training GANs
68. Dynamic Evaluation of Neural Sequence Models
69. Grandmaster level in StarCraft II using multi-agent reinforcement learning
70. Efficient Transformers: A survey
71. FlashAttention
72. FlashAttention 2
73. SpikeGPT
74. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
75. Learning to Model the World with Language
76. Learning Transferable Visual Models From Natural Language Supervision
77. FLatten Transformer: Vision Transformer using Focused Linear Attention
78. DeepSpeed Chat
79. MusicGen
80. PaLM-E: An Embodied Multimodal Language Model
81. PaLI-X: On Scaling up a Multilingual Vision and Language Model
82. Visual Instruction Tuning
83. Image Transformer

Other things to read

• Reinforcement learning papers
• NLP papers
• Ethics papers
• ViTs (Vision Transformers)
• Statistics (PCA, fisher vectors and fisher kernels, boltzmann machines)
• SIFT+FV networks
• GELUs and other activation functions
• facial recognition
• 3d model representation
• https://ai.googleblog.com/2018/03/using-evolutionary-automl-to-discover.html
• https://www.fast.ai/posts/2018-08-10-fastai-diu-imagenet.html
• Mixup, Cutmix, RandAugment, Random Erasing, and regularization scehemes like Stochastic Depth and Label Smoothing
• Inverted bottleneck
• BatchNorm vs LayerNorm
• https://www.reddit.com/r/MachineLearning/comments/hj4cx/comment/c1vt6ny/
• Bishop's Pattern Recognition and Machine Learning
• http://alumni.media.mit.edu/~tpminka/statlearn/glossary/
• Statistical Pattern Recognition by Andrew Webb
• Programming Collective Intelligence
• https://www.d2l.ai/