Large-scale Generative Models for NLP and Speech Processing

Natural language processing has given rise to innumerable industrial applications. While many new tasks have emerged in NLP and speech processing over the last decades, methods to solve them have increasingly converged towards a unified modeling paradigm. In this course, we will use large-scale generative models and sequence-to-sequence modeling to delve into state-of-the-art statistical machine learning methods and apply them to major NLP and speech processing tasks — language modeling, machine translation, speech recognition, information extraction. Students should expect to get an in-depth understanding of these methods, through theoretical analysis and hands-on lab sessions. Grading will involve a project, to be carried out over the course of the class. Topics to be covered:
1. Language Modeling
2. Speech to Text
3. Extracting Structured Information from Text
4. Machine Translation
5. Code Generation