Reproducing the deep double descent paper

reproducing deep double descent 04 Jun, 2025 This summer, I've been at the recurse center intensively trying to catch up to the current state of the machine learning world. I don't have any prior background in ML, so I've been taking some classes and reading a lot of papers. Two weeks in, I now have some basic working knowledge and wanted to get my hands dirty. After reading the Deep Double Descent paper, I wanted to see if I understood enough to reproduce the results. In a previous post, I went over some notes about doing the training for this on a rental GPU, but I figured I'd go into details about the project itself. Please note the understanding here is still one of a student - if you spot something wrong, please send me a message! double descent background For a long time, the ML community thought that models could only be so big before they started degrading in accuracy. Around the start of the GPT era, folks realized that you could get better test-time results from a model ... Read full article.