Build LLMs From Scratch - AiA
Monday 4 November 2024
Starts 18:30 PM
Finishes 20:30 PM
Organized by 351 Portuguese Startup Association
Venue: Le Wagon Lisbon
Address: R. do Centro Cultural 45, 1700-006
Lisboa
About this event
Join our event-series to learn how to build Large Language Models (LLMs) from scratch.
This is our FIRST meeting.
\*\*\*
Have you used LLMs but remain curious/dubious about how they work?
You want to learn about LLMs but find it overwhelming?
You see the potentials of LLMs but don’t know where to start?
*Join us to build your LLM from scratch!* Learn by doing together with a supportive community.
**Full details: [check out this page](https://dan-tran.notion.site/LLMs-from-scratch-AIA-10ad49a1a8f68091a03eebe942202b7b)**
\*\*\*
## **What**
This is a hands-on learning bootcamp focusing on Large Language Models (LLMs), spanning several months. You will learn to design, pretrain, and fine-tune your own GPT-like model.
## **Who**
The program is suitable for people with a quantitative background. Working knowledge of Python, Pytorch, and Machine Learning is a plus but not mandatory.
Two main prerequisites are:
- be comfortable with computers & maths (bachelor level)
- willing to commit *\~40h/month* to focused learning
## **How**
- Study: read/watch materials, run the code, and solve the exercises
- Research: read papers, explore and experiment, try to break things
- Group meeting: monthly meetings, summarize key insights, Q&A
- Discord: discuss about anything, share resources, ask questions
## **Wanna join?**
IMPORTANT:
- Check out [[program repo on Github](https://github.com/AiAnonymousPT/COURSE-LLMs-from-Scratch)]
- Complete the [\[Meeting-0 materials](https://github.com/AiAnonymousPT/COURSE-LLMs-from-Scratch/blob/main/Meeting-0/README.md)]
- Please fill [[Google Form](https://forms.gle/C7QZ3LyVt6iQEtJa9)] to join WhatsApp & Discord group
- Logistics: meetings are in-person, location & time will be updated in Discord & Meetup
See you there! Dan, Mark & Hal
\*\*\*
# **Program & Schedule**
We plan to meet every 3/4 weeks, around the last week of the month.
* Month 0 Getting started — Mon, Nov 4
* Month 1 Tokenization & Embeddings — Mon, Nov 25
* Month 2 Project: build your own tokenizer — Mon, Jan 6
* Month 3 Attention Mechanisms — Mon, Jan 27
* Month 4 Transformer & GPT Architecture — Mon, Feb 24
* Month 5 Pretraining LLMs — Mon, Mar 24
* Month 6 Fine-tuning LLMs — Mon, Apr 21
* Month 7 Final Project: build your own GPT-2 — Mon, May 26
The first 3 months have optional materials allowing for everyone to acquire the fundamental knowledge required to build LLMs.
\*\*\*
## **About Us**
Our goal is to democratize Machine Learning and AI. We experiment with hands-on projects on LLMs like RAGs, quantization, and other real-life applications. We believe that it does not matter who you are, where you come from, you can build and contribute to shaping the future with better and safer AI technologies.
\*\*\*
## **FAQs**
1.Is there any fee?
No fee, no hidden cost, aside from the textbook. If you can’t afford it, reach out to us. Most materials are publicly available.
2.What will I learn after completing the program?
At minimum, you’ll gain a much deeper understanding of LLMs than from YouTube and blog-posts. At best, with the right resources, you’ll spin out new custom LLMs every month.
3.How can I join the Discord and WhatsApp group?
Fill out the form in the event description to receive invitations to join the groups.
4.What kind of support is available?
Support is available in real-time on Discord. We may organize co-working days on top of the monthly meeting.
5.What if I miss a meeting?
Meeting discussions and resources will be available on Discord and GitHub to help you catch up.
6.Can I join after the series has started?
It’s best to join from the start, but you can join later if you have the required time/knowledge to catch up.
This page last updated Tuesday 29 October 2024 at 01:22.
Problems? Report an error or inappropriate listing here.
Information displayed here is provided in good faith but we are not responsible for the content of any listing. Sometimes events can be cancelled or changed at short notice. Please check with the venue or organizer before you travel!