Java Virtual Threads Ate My Memory: A Web Crawler's Tale of Speed vs. Memory

Virtual Threads Ate My Memory: A Web Crawler’s Tale of Speed vs. Memory Dario Balinzo Follow 5 min read · 2 days ago 2 days ago -- 1 Listen Share Photo by Julian Hochgesang on Unsplash I built a simple web crawler using good old platform threads. It was just a multithreaded crawler, nothing fancy. But then, curiosity struck: “What happens if I use Virtual Threads instead?” Virtual Threads are one of my favorite recent additions to the Java ecosystem. Switching from platform threads to Virtual Threads dramatically improved the URL processing rate… until the whole thing blew up with an OutOfMemoryError . Yep. Fast became too fast. I love exploring Java’s new features. Virtual Threads, Records, Pattern Matching, you name it. But every tool has tradeoffs. In software engineering, solving one problem can introduce another. Our job isn’t just writing code; it’s balancing performance, safety, and maintainability. This post is about my little hacking session, where I tried to unleash the ... Read full article.