Unlock the magic of language models — join us at our three day CSH Autumn School 2024 from September 30th to October 2, 2024 and elevate your expertise!
Most likely you have heard of language models such as GPT-4: They are said to revolutionize many aspects of work and life - for example, language models can assist you when setting up a website, finding creative titles for your papers, giving advice on what to gift to your parents on Christmas...
But why and how do they actually work? Their capabilities might seem magical - but actually, you can understand and build them yourself!This course aims to provide you the basic tools to understand, work with, and build your own language models.
The course is organized following the standard language modeling pipeline used for a specific (supervised) language modeling task:
- Transform the text into a machine-readable numerical representation using tokenization (and positional embeddings).
- Learn a useful general-purpose representation of language: a foundation model (e.g. BERT), focusing on non-generative models. The following building blocks are required:
- Feed-forward artificial neural networks: Architecture, training ((stochastic) gradient descent, backpropagation), hyperparameters and overfitting are discussed and compared to standard linear regression models.
- The self-attention mechanism and transformer models to efficiently model time-dependency in language.
- Training without labels using masked-language modeling.
- Use the language representation for specific (supervised) tasks
- Know how to apply the foundation model as feature extractor, or fine-tune the foundation model for the specific task.
- The main application is sentiment classification.
All aspects will be supplemented by practical exercises in Python, focusing on various applications of sentiment analysis.
Participants can apply their acquired skills in a small project on the last day of class (sentiment of product/movie reviews, or project based on interest of the individual participant).
Application is managed via the F.I.T. platform (weiterbildung.uni-hohenheim.de).
Further Information can be found in the accompanying PDF here.
