Skip to content
Tech News
← Back to articles

CS336: Language Modeling from Scratch

read original more articles
Why This Matters

This coursework emphasizes practical experience in language modeling, encouraging students to utilize cloud GPU resources for training and benchmarking. It highlights the importance of understanding AI ethics and collaboration policies, while promoting hands-on learning with real-world tools. The approach prepares students for industry challenges in developing and deploying language models effectively.

Key Takeaways

Coursework

Assignments

GPU compute for self-study

All (currently tentative) deadlines are listed in the schedule

If you are following along at home, you can access GPU compute from a cloud provider to complete the assignments.

Here are a few options (public pricing for a single B200 GPU on March 28, 2026):

For convenience and to save money, we recommend debugging correctness of your implementation on CPU first and then using GPU(s) (with the count recommended in the assignments) for completing training runs (A1, A4, A5) or benchmarking GPU operations (A2).

Honor code

Collaboration : Study groups are allowed, but students must understand and complete their own assignments, and hand in one assignment per student. If you worked in a group, please put the names of the members of your study group at the top of your assignment. Please ask if you have any questions about the collaboration policy.

: Study groups are allowed, but students must understand and complete their own assignments, and hand in one assignment per student. If you worked in a group, please put the names of the members of your study group at the top of your assignment. Please ask if you have any questions about the collaboration policy. AI tools : Prompting LLMs such as ChatGPT is permitted for low-level programming questions or high-level conceptual questions about language models, but using it directly to solve the problem is prohibited. We strongly encourage you to disable AI autocomplete (e.g., Cursor Tab, GitHub CoPilot) in your IDE when completing assignments (though non-AI autocomplete, e.g., autocompleting function names is totally fine). We have found that AI autocomplete makes it much harder to engage deeply with the content. See the AI policy.

... continue reading