Advanced Natural Language Processing / Fall 2025


Advanced natural language processing is an introductory graduate-level course on natural language processing aimed at students who are interested in doing cutting-edge research in the field. In it, we describe fundamental tasks in natural language processing as well as methods to solve these tasks. The course focuses on modern methods using neural networks, and covers the basic modeling, learning, and inference algorithms required therefore. The class culminates in a project in which students attempt to reimplement and improve upon a research paper in a topic of their choosing.


Course Details

Instructor


Logistics

  • Class times: TR 2:00pm - 3:20pm
  • Room: TEP 1403
  • Course identifier: LTI 11-711
  • Piazza: Piazza
  • Code: GitHub
  • Office hours:
    Location Day Time
    Sean Welleck GHC 6513 Tuesday 4:00-5:00 PM
    Joel Mire WEH 3110 Tuesday 3:30-4:30 PM
    Chen Wu GHC 5417 Tuesday 4:00-5:00 PM
    Dareen Alharthi GHC 5417 Monday 10:00-11:00 AM
    Neel Bhandari GHC 5417 Friday 12:00-1:00 PM
    Akshita Gupta GHC 5417 Friday 4:00-5:00 PM
    Ashish Marisetty GHC 5417 Friday 2:00-3:00 PM
    Manan Sharma Online (Link) Tuesday 11:00 AM-12:00 PM
    Sanidhya Vijayvargiya GHC 5417 Wednesday 12:00-1:00 PM

Grading

  • The assignments will be given a grade of A+ (100), A (96), A- (92), B+ (88), B (85), B- (82), or below.
  • The final grades will be determined based on the weighted average of the quizzes, assignments, and project. Cutoffs for final grades will be approximately 97+ A+, 93+ A, 90+ A-, 87+ B+, 83+ B, 80+ B-, etc., although we reserve some flexibility to change these thresholds slightly.
  • Quizzes: Worth 20% of the grade. Your lowest 3 quiz grades will be dropped.
  • Assignments: There will be 4 assignments (the final one being the project), worth respectively 15%, 15%, 20%, 30% of the grade.

Course description

The course covers key algorithmic foundations and applications of advanced natural language processing.

There are no hard pre-requisites for the course, but programming experience in Python and knowledge of probability and linear algebra are expected. It will be helpful if you have used neural networks previously.

Acknowledgements. This semester's course is based on Advanced NLP Spring 2025, which itself was adapted from Advanced NLP Fall 2024, designed and taught by Graham Neubig.

Class format

Lectures: For each class there will be:
  • Reading: Most classes will have associated reading material that we recommend you read before the class to familiarize yourself with the topic.
  • Lecture and Discussion: There will be a lecture and discussion regarding the class material. This will be recorded and posted online for those who cannot make the in-person class.
  • Code/Data Walkthrough: Some classes will involve looking through code or data.
  • Quiz: There will be a quiz covering the reading material and/or lecture material that you can fill out on Canvas. The quiz will be released by the end of the day of the class (11:59pm) and will be due at the end of the following day (11:59pm).
Questions and Discussion: Ideally in class or through Piazza so we can share information with the class, but coming to office hours is also encouraged.

Schedule

Quizzes

There will be a quiz covering the reading material and/or lecture material that you can fill out on Canvas. The quiz will be released by the end of the day of the class (11:59pm) and will be due at the end of the following day (11:59pm).

Quizzes are worth 20% of the grade. Your lowest 3 quiz grades will be dropped. We provide the drop days in case you have to miss a quiz (e.g., due to travel, unexpected circumstances). Please do not contact the TAs or Instructors about additional quiz drops; we provide three quiz drops to cover unforeseen circumstances.

Assignments

The aim of the assignment and project is to build basic understanding and advanced implementation skills needed to build cutting-edge systems or do cutting-edge research using neural networks for NLP, culminating with a project that demonstrates these abilities through a project.

Read all the instructions on this page carefully
You are responsible for reading these instructions and following them carefully. If you do not, you may be marked down as a result.

Assignment Policies

Working in Teams:

There are 4 assignments in the class. Assignment 1 must be done individually, while Assignments 2, 3, and 4 must be done in teams of 2-3 (individual submissions will not be accepted for these assignments). If you are having trouble finding a group, the instructor and TAs will help you find one after the first initial survey.

Submission Information:

To submit your assignment you must submit via Canvas a zip file containing:

  • your code: This should be in a directory “code” in the top directory unless specified otherwise.
  • system outputs (assignments 1 and 2): The format will be specified separately for each assignment.
  • a report (assignments 2, 3 and 4, optional for assignment 1): This should be named “report.pdf” in the top directory. This is for assignments 2, 3 and 4, and can be up to 7 pages for assignments 2 and 3 and 9 pages for assignment 4. References are not included in the page count, and it is OK to submit appendices that include supplementary information such as hyperparameter settings or additional output examples, although there is no guarantee that the TAs will read them. Submissions that exceed the page count will be penalized one third grade for each page over (e.g., A to A- or A- to B+). You may also submit report.pdf for assignment 1 if you have any interesting information to convey to the TAs, for example, if you did anything interesting above and beyond the minimal requirements.
  • a link to a GitHub repository containing your code (assignments 2, 3 and 4): This should be a single line file “github.txt” in the top directory. Your GitHub repository must be viewable to the TAs in charge of the assignment by the submission deadline. If your repository is private, make it accessible to the TAs by the submission deadline. If your repository is not visible to the TAs, your assignment will not be considered complete, so if you are worried, please submit well in advance of the deadline so we can confirm the submission is visible. We use this repository to check contributions of all team members.

Late Day Policy:

In case there are unforeseen circumstances that don't let you turn in your assignment on time, 5 late days total for assignments 1, 2, 3.1, and 3.2 will be allowed. Note that other than these late days, we will not be making exceptions and extending deadlines except for documented health reasons, so please try to be frugal with your late days and use them only if necessary. Assignments that are late beyond the allowed late days will be graded down one third-grade per day late (e.g., A to A- for one day, and A to B+ for two days).

Plagiarism/Code Reuse Policy:

All assignments are expected to be conducted under the CMU policy for academic integrity. All rules here apply and violations will be subject to penalty including zero credit on the assignment, failing the course, or other disciplinary measures. In particular, in your implementation:

  • Code or pseudo-code provided by the TAs or instructor may be used freely without restriction.
  • For assignment 2, you may not just re-use an existing implementation written by someone else. The implementation should basically be your own.
  • Code written by other students in the class cannot be used (except, obviously, you can share code within your group for assignments 2, 3, and 4).
  • If you are doing a similar project for a graded class at CMU (including independent studies or directed research), you must declare so on your report, and note which parts of the project are for 11-711, and which parts are for the other class. Consult with the Instructor during office hours or on Piazza if you are unsure.

Consulting w/ Instructors/TAs:

For assignments and projects, you are free to consult with the TAs and instructors during office hours, project hours, and through Piazza. If you don't have much experience with NLP, it will be helpful to consult with the instructors and TAs to learn about how to do the assignments and course project.

Because this is a project-based course, we assume that many of the students taking the course will be interested in turning their assignments or project into research papers. In this case, if you have received useful advice from the instructor or TAs that made the project significantly better, consider inviting them to be co-authors on the paper. Of course, you do not need to do so just because the paper is a result of the class, only if you feel that their advice or help made a contribution.

Details of Each Assignment

  1. Assignment 1: Build Your Own LLaMa (Individual assignment)
    • Released: Sep 9
    • Due: Sep 25
  2. Assignment 2: End-to-end NLP System Building (Group assignment)
    • Released: Sep 25
    • Due: Oct 9
  3. Assignment 3: Project Proposal & State-of-the-art Reimplementation (Group assignment)
    • Assignment 3.1: Literature Review & Project Proposal
      • Released: Oct 9
      • Due: Oct 30
    • Assignment 3.2: Baseline Reproduction
      • Released: Oct 9
      • Due: Nov 13
  4. Assignment 4: Final Project (Group assignment)
    • Released: Oct 9
    • Due: Dec 9

Details to be provided later.

Poster Presentation

Time/Location

  • Time: 2:00PM-3:20PM, December 2nd, 2025 and December 4th, 2025
  • Location: Hallway below LTI (GHC4400)
We will announce which teams will be presenting on which days during the course. If you have a major, immovable conflict that will prevent your team from presenting on one day please contact us via piazza and we will try to make accommodations.

Goals and Grading

The intention of the poster is several-fold:
  • That you share your preliminary results with the TAs and instructor so we can give feedback to make any last adjustments to improve your final project report.
  • That you can see the other projects in the class to learn from them and get any ideas that may improve your final project report.
  • That you can practice explaining the work that you did.
Posters are graded pass/fail based on (a) whether you completed a poster; (b) whether you attended your poster session. No exceptions will be made; you must attend your poster session in order to receive a grade.

What information should be included in a poster? It should be mostly:
  • What is the problem you’re solving
  • What is your method for solving that problem
  • What are the results
There is not a set format for creating a poster, but if you would like some guidance, I would suggest creating three columns, where the left one describes “1”, the middle one describes “2”, and the right one describes “3”. The middle one can be a bit wider.

Poster Printing

If you are a member of SCS, we suggest that you use SCS poster printing. If you are not a member of SCS, you can send your PDF to the TAs no less than 5 days before your presentation, and we will print it for you.