CSE 8803 DLT: Deep Learning for Text Data (2020 Fall)

• Lecture time: Monday & Wednesday 5PM–6:15PM
• Location: https://bluejeans.com/5341114422 & Instructional Center 215
• Instructor: Chao Zhang
• Teaching Assistant: Lingkai Kong (lkkong@gatech.edu); Rui Feng (rfeng@gatech.edu)
• Piazza: https://piazza.com/gatech/fall2020/cse8803dlt
• Office Hours:
  • Instructor Office Hour: Monday 4-5 PM, https://bluejeans.com/5341114422
  • TA Office Hour: Wednesday 4-5 PM, https://bluejeans.com/1146576855

This course will introduce state-of-the-art machine learning techniques for mainstay problems in text data analysis, with particular emphasis on deep learning methods that have recently achieved enormous success. The students will learn about trending problems in this field, key methods for solving these problems, and their advantages and disadvantages. The students are also expected to read, review, and present research papers, as well as conduct a research oriented course project. The course will provide useful techniques for students who want to solve practical problems involving text data, and better prepare those who want to do edge-cutting research in text mining, natural language processing, artificial intelligence, and text-rich interdisciplinary research. The learning objective is that by the end of this course, the students are able to formulate their text analysis problems at hand, choose appropriate statistical models for the problems, and even come up with innovative solutions for solving open research problems in this field.

Prerequisites for this course: (1) the students should be have basic knowledge in machine learning and taken a relevant course before (e.g., CX4240, CSE6740, CS4641); (2) the students should be comfortable with reading research papers and giving presentations; (3) the students should have solid programming skills—the course project can be programming demanding.

In the task space, we will cover a range of text data analysis tasks, including:

• Text representation learning
• Text classification (sentiment analysis, document classification)
• Sequence labeling (POS tagging, NER, event extraction)
• Text generation (text summarization, image captioning)
• Knowledge graph construction and reasoning
• Question answering and dialogue systems

In the methodology space, we will introduce state-of-the-art deep learning techniques for these problems, including:

• Deep representation learning (Word2Vec, Transformer, BERT)
• Sequence labeling models (CRF, LSTM-CRF)
• Deep transfer/multi-task learning
• Neural symbolic methods
• Deep generative models (GPT, VAE, and their variants)

Specifically, we have the following modules:

• Background knowledge
  • Course overview and logistics
  • ML/DL review
• Text representation learning
  • Bag-of-words, n-gram, TF-IDF
  • Dimension reduction, matrix factorization, topic models
  • Word2vec, Glove
  • Transformer and BERT
• Deep text classification
  • Traditional text classifiers: naive Bayes, logistic regression, SVM
  • Deep classifiers: CNN, RNN and their variants
  • Attention models
• Learning with less labeled data
  • Transfer learning and fine-tuning
  • Multitask learning
  • Semi-supervised learning
  • Weakly supervised learning
• Sequence labeling and structured prediction
  • Motivating tasks: POS tagging, NER, event extraction
  • Conditional random fields for sequence labeling
  • Deep latent variable models for sequence labeling
• Text generation
  • Motivating tasks: summarization, image captioning, translation
  • Deep language models
  • GPT and VAE for text generation
• Knowledge representation and reasoning
  • Knowledge graph construction
  • Knowledger graph reasoning
  • Combining logic and neural models
  • Question answering and Dialogue systems with knowledge bases
• Project presentations

This course is hybrid touch point mode. All the lectures and presentations will be delivered online at https://bluejeans.com/5341114422, and we have touch point meetings for in-person discussions. More information about the touch point meetings can be found in the FAQs.

Disclaimer: The instructor reserves the right to modify the planned schedule and grading policy as needed during the course.

• What are those "touch point" meetings for?
  • The purpose of touch point meetings is to offer opportunities for students who may need in-person discussions for projects or course content. In each touch point meeting, the instructor or a TA will be present in the classroom physically, to address questions on projects, papers, or other course content for participating students. No new content will be delivered in touch point meetings.
• Are those touch point meetings required?
  • No, they are completely optional. They are only for students who really need in-person discussions. There will be no penalty even if you attend none of them.
• If I choose to go to touch point meetings, what protections are available?
  • First, we will follow the social distancing rule and limit the maximum number of students for each touched point meeting. Please register here if you want to attend a touch point meeting: touchpoint meeting student signup form.
  • Second, any student who chooses to attend touch point meeting MUST wear a face mask, according to Georgia Tech's policy. See more information from Georgia Tech at this link and this link.
• For remote lectures, will there be recordings?
  • Yes, for every remote lecture and paper presentation, we will record on Bluejeans and shared a link after class.

• Speech and Language Processing, by Dan Jurafsky and James H. Martin
• Deep Learning for NLP
• Deep Learning, by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
• Dive into Deep Learning, by Aston Zhang, Zack C. Lipton, Mu Li, and Alex Smola

Other resources, such as deep learning toolboxes and datasets, will be provided throughout the course.