Show HN: LLM plays Pokémon (open sourced)

Fire Red Agent Overview This is my attempt at getting a large language model to play Pokémon FireRed autonomously. My bot has rudementery capabilities to play the game, explore, battle, and respond to game events. To me, this is the future of TV. While building the bot, I felt like I was producing television more than I was programming a computer. Ultimately, I ran into some technical hurdles around programmatic input control, which led me to pause development. Watch the demo video. Watch the technical deep dive. How It Works Emulator Integration I used RetroArch to run Pokémon FireRed. I struggled with sending inputs programmatically. RetroArch has a UDP-based input system (RetroPad), but I couldn't get it to work reliably. Instead, I resorted to using OSA Script (AppleScript) to send keyboard events to the emulator, which meant the game had to be in focus. This was a huge limitation because it took over my entire computer, making background work impossible. Game Memory Manag ... Read full article.