LLMs in Lingustic Research 2024/25
🧩 Homework
Hw 03 - Intro to transformers
🏠 Home
📓 Syllabus
✍️ Assignments
📝 Assignment 01
📝 Assignment 02
🗓️ Class material
Week 01 - Intro
👨🏫 Lecture Material
Week 02 - NN basics
👨🏫 Lecture Material
Week 03 - NN basics II
👨🏫 Lecture Material
Week 04 - Transformers (part 1)
👨🏫 Lecture Material
Week 05 - Transformers (part 2)
👨🏫 Lecture Material
Week 06 - Hands-on with Transformers (part 1)
👨🏫 Lecture Material
Week 07 - GPT-2 implementation
👨🏫 Lecture Material
Week 08 - finetuning text classification
👨🏫 Lecture Material
Week 09 - finetuning machine translation
👨🏫 Lecture Material
Week 10 - morphological inflection using T5
👨🏫 Lecture Material
Week 11 - morphological inflection with T5 using CHILDES
👨🏫 Lecture Material
🧩 Homework
Hw 01 - Colab
Hw 02 - NN basics
Hw 03 - Intro to transformers
🧩 Homework
Hw 03 - Intro to transformers
Homework 3: Intro to transformers
Author
Akhilesh Kakolu Ramarao, Anna Sophia Stein
Reading
Read the first 8 pages of
Attention is All You Need
paper.
Read about Self-attention mechanism here:
https://jalammar.github.io/illustrated-transformer/
Watch these videos:
How large language models work, a visual intro to transformers
Attention in transformers, visually explained
Illustrated Guide to Transformers Neural Network: A step by step explanation
Hw 02 - NN basics