From Seq2Seq to Transformer

Learning Machine Learning (LML) Seminar

Organizers：

Artane Jérémie Siad, Ning Su, David Pechersky, Justin Yeh, Zhen Zhang

Speaker：

Justin Yeh, Tsinghua University

Time：

Mon., 18:00-19:00, April 20, 2026

Venue：

Jing Zhai

Title：

From Seq2Seq to Transformer

Abstract:

This talk introduces the foundational sequence-to-sequence (seq2seq) architecture and the attention mechanism that revolutionized it. We start with the encoder-decoder framework, covering training basics and simple models, then explain why attention is needed and how it works. From there, we build up to the Transformer, the modern workhorse of seq2seq. We also discuss practical essentials: subword segmentation (e.g., Byte Pair Encoding), inference methods like beam search, and finally touch on how we can analyze and interpret what these models have learned. The goal is to go through each component in technical detail, from the ground up making the ideas accessible to a beginner audience without glossing over how things actually work.