GoKawiilWandering Thoughts blog begins blocking visitors with generic HTTP User-Agent strings
Chris Siebenmann, who runs the Wandering Thoughts blog and CSpace wiki, says he is now attempting to block requests from clients whose HTTP User-Agent header is too generic or suspicious. He attributes the move to a surge in high-volume crawlers, some reportedly gathering data for LLM training, that strain his site's resources.