Topic | Study Links | Deliverable | Estimated Time |
Transfer Learning | Yandex Course; NLP ImageNet Moment; Illustrated BERT; fastai NLP; | Notebook: Try out BERT on an email classification problem. Write a 200-500 word blog on Transfer Learning | 4-6 hours |
Sub-word Models | cs224n slides; BERT Pytorch implementation - check the tokenizer; Mikolov paper; Words vs Morpheme: Paper | Notebook: Sub-word sentence classification model on a standard dataset; comparing to full word for code-mixed data | 10 hours |
Multi-Task Learning (MTL); NER | Socher Guest Lecture on MTL; decaNLP; Flair; Haptik Chatbot NER; HMTL by HuggingFace; Cyclic Learning Rate Scheduler | Notebook: Yandex Homework; | 5-7 hours |
~No New Topics~ | Yes, you should write a blog - Rachel Thomas; | Blog or new experiment notebook: Based on any 2 or more themes | 3-7 hours |
Model Interpretations and Visualisation | Kaggle Learn module on the topic; example: BERT Multi-lingual Vocabulary; GPT-2 demo | Notebook or blog: Explain BERT and USE predictions with one or more methods | 6-10 hours |