Planning.md

Google QUEST Q&A Labeling - Planning

Working cycle:

1. Read competition information and relevant content to feel comfortable with the problem. Create a hypothesis based on the problem.
2. Initial data exploration to feel comfortable with the problem and the data.
3. Build the first implementation (baseline).
4. Loop through [Analyze -> Approach(model) -> Implement -> Evaluate].

1. Literature review (read some kernels and relevant content related to the competition).

• Relevant content:

  • Encoder-decoders in Transformers: a hybrid pre-trained architecture for seq2seq
  • Universal Sentence Encoder - TF Hub
  • Semantic Similarity with TF-Hub Universal Encoder - Colab
  • Multilingual Universal Sentence Encoder for Semantic Retrieval
  • Universal Sentence Encoder - Paper
  • BERT - Input
  • HuggingFace Transformers
  • Smaller, faster, cheaper, lighter: Introducing DistilBERT, a distilled version of BERT
  • The Illustrated Transformer
  • The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
  • Hugging Face: State-of-the-Art Natural Language Processing in ten lines of TensorFlow 2.0
  • Simple BERT using TensorFlow 2.0
  • TF Hub - bert_en_uncased_L-12_H-768_A-12
  • GloVe: Global Vectors for Word Representation
  • Multilingual Universal Sentence Encoder for Semantic Retrieval
  • Semantic Similarity with TF-Hub Universal Encoder

• GitHub:

  • Git - Transformers: State-of-the-art Natural Language Processing
  • GloVe model for distributed word representation

• Papers:

  • How to Fine-Tune BERT for Text Classification?
  • Universal Language Model Fine-tuning for Text Classification
  • Universal Sentence Encoder

• Videos:

  • BERT for Kaggle Competitions | Yuanhae Wu | Kaggle Days
  • Deep Learning Formulas for NLP Applications | Chenglong Chen | Kaggle
  • Solving NLP Problems with BERT | Yuanhao Wu | Kaggle

• Kernels:

• Discussions:

  • Everything you always wanted to know about BERT (but were afraid to ask)
  • BERT & Friends Reference
  • 1ST PLACE SOLUTION - Jigsaw competition
  • Let's Complicate Things [Bert]

• Insights:

• Positive Insights

• Negative Insights