CSE 8803 DLT: Deep Learning for Text Data

Table of Contents

Logistics

  • Lecture time: Monday & Wednesday 3:00PM–4:15PM
  • Location: CoC Room 52
  • Instructor: Chao Zhang
  • Teaching Assistant: TBD
  • Piazza: TBD
    • Piazza will be the main place for course discussions and announcements. If you have questions, please ask it on Piazza first because 1) other students may have the same question; 2) you will get help faster compared to sending emails.
  • Office Hours:
    • Instructor Office Hour: TBD
    • TA Office Hour: TBD

Overview

This course will introduce state-of-the-art machine learning techniques for mainstay problems in text data analysis, with particular emphasis on deep learning methods that have achieved enormous success recently. Students will learn about trending problems in this field, key methods for solving these problems, and their advantages and disadvantages. The course will provide useful techniques for students who want to solve practical problems involving text data, and better prepare those who want to do edge-cutting research in text mining, information retrieval, natural language processing, and text-rich interdisciplinary research. The learning objective is that by the end of this course, the students are able to formulate their text analysis problems at hand, choose appropriate statistical models for the problems, and even come up with innovative solutions for solving open research problems in this field.

In the task space, we will cover a range of text data analysis tasks, including:

  • Text classification (sentiment analysis, document classification)
  • Sequence labeling (POS tagging, NER, event extraction)
  • Knowledge graph construction and reasoning
  • Text generation (text summarization, image captioning)
  • A number of other trending topics in this area.

In the methodology space, we will introduce state-of-the-art deep learning techniques for these problems, including:

  • Deep representation learning (Word2Vec, Transformer, BERT)
  • Sequence labeling models (CRF, LSTM-CRF)
  • Deep transfer/multi-task learning
  • Neural symbolic methods
  • Deep generative models (GAN, VAE, and their variants)

Prerequisites for this course: (1) the students should be have basic knowledge in machine learning and taken a relevant course before (e.g., CX4240, CSE6740, CS4641); (2) the students should have solid programming skills—both the assignments and the course project can be programming demanding.

Content

  • Background knowledge
    • Course overview and logistics
    • Text data analysis: tasks and applications
    • ML/DL review
  • Text representation learning
    • Bag-of-words, n-gram, TF-IDF
    • Dimension reduction, matrix factorization, topic models
    • Word2vec, Glove
    • Transformer and BERT
  • Deep text classification
    • Traditional text classifiers: naive Bayes, logistic regression, SVM
    • Deep classifiers: CNN, RNN and their variants
    • Attention models
  • Learning with less labeled data
    • Transfer learning and fine-tuning
    • Multitask learning
    • Semi-supervised learning
    • Weakly supervised learning
  • Sequence labeling and structured prediction
    • Motivating tasks: POS tagging, NER, event extraction
    • Conditional random fields for sequence labeling
    • Deep latent variable models for sequence labeling
  • Text generation
    • Motivating tasks: summarization, image captioning, translation
    • Deep language models
    • VAE and GAN for text generation
  • Knowledge representation and reasoning
    • Knowledge graph construction
    • Knowledger graph reasoning
    • Combining logic and neural models
  • Other tending topics in text data analysis
    • Multimodal analytics of text and other data
    • Adversarial learning and model robustness
    • Question answering and dialogue systems
  • Project presentations

Schedule

Date Topic Assignment Due Readings
8/19/19 Course Overview Sign up for Piazza    
8/21/19 Text Analysis Basics      
8/26/19 Machine Learning Review      
8/28/19 Bag-of-words, N-Grams, TF-IDF      
9/2/19 No Class (Labor Day)      
9/4/19 Dimension Reduction and Topic Models   Proposal Due  
9/9/19 Word Embedding, Word2Vec, Glove HW1 Out    
9/11/19 Transformer and BERT      
9/16/19 Traditional Text Classifiers      
9/18/19 Deep Text Classification      
9/23/19 Attention Models   HW1 Due  
9/25/19 Transfer Learning and Fine-Tuning HW2 Out    
9/30/19 Multitask Learning      
10/2/19 Semi-Supervised Learning      
10/7/19 Weakly Supervised Learning      
10/9/19 Sequence Labeling Tasks   HW2 Due  
10/14/19 No Class (Fall Recess)      
10/16/19 Conditional Random Fields HW3 Out Mid Report Due  
10/21/19 Deep Models for Sequence Labeling      
10/23/19 Text Generation Tasks      
10/28/19 Deep Language Models      
10/30/19 GAN and VAE for Text Generation   HW3 Out  
11/4/19 Knowledge Graph Construction      
11/6/19 Knowledge Graph Reasoning      
11/11/19 Neural Symbolic Methods      
11/13/19 Other Trending Topics   HW3 Due  
11/18/19 Project Presentations      
11/20/19 Project Presentations      
11/25/19 Project Presentations      
11/27/19 No Class      
12/2/19 No Class   Final Report Due  
12/4/19 Reading Day      

Grading

  • Homeworks (45%)
    • There will be three assignments. Each one is designed for testing your understanding of the taught algorithms. It could be either programming or written analysis.
    • All assignments follow the "no-late" policy. Assignments received after the due time will receive zero credit.
    • All students are expected to follow the Georgia Tech Academic Honor Code.
  • Project (50%)
    • You are expected to complete a project on machine learning for text data. Your project needs to be clear about 1) the problem you are attempting to solve; 2) a survey of existing literature for the problem and the technical method you propose to solve the problem; 3) the results and conclusion you attain.
    • Each project needs to be completed in a team of 2-3 people. Team members need to clearly claim their contributions in the project report.
    • You will need to submit the following:
      • Project proposal (10%): formulation of the problem and expected outcome
      • Mid-term report (10%): literature survey and technical roadmap
      • Presentation (15%): group-wise in-class project presentation
      • Final report (15%): a complete and final project report
  • Class participation (5%)
    • Your class participation score will be graded based on attendance and in-class quizzes.
    • Participation in class discussions (including asking relevant questions in class, volunteering to answer questions on Piazza) will be considered when determining your final grade. It will be especially useful when you are right on the edge of two letter grades.

Resources

Other resources, such as deep learning toolboxes and datasets, will be provided throughout the course.