GoKawiilI don’t know if you’ve read “The Bitter Lesson”?
Ars: Mm-hmm, yeah.
[The Bitter Lesson is a 2019 essay by computer scientist and reinforcement learning pioneer Richard Sutton. In part, it argues that efforts to bake domain-specific structures into AI systems have often “proved ultimately counterproductive” and that the methods that win out over time are general-purpose ones that scale with available compute.]
Wu: Yeah, that is one of our guiding principles for our team. And I think it’s really hard—because the models are changing so quickly—it’s really hard to say that this will definitely be the next form factor. We have a few guesses. We dogfood internally a lot of these ideas, but we’re pretty open-minded to just being wrong, and we just need to stay really close to the model capabilities.
Ars: I know some of these things have arisen from seeing users who are just using it this way and then deciding to productize that and make it more convenient.
Are there things that you’re seeing right now that are like that, that you haven’t productized yet where you’re saying, “OK, now we need to be thinking about this in the near future”?
Wu: We try to go from conviction to a product shift pretty quickly, ideally in a week or so. So usually there isn’t a big delay between us feeling that user pull and shipping something.
I think there is maybe this next level where Claude can anticipate what you want. Like, it can proactively know that, “Oh, you’re working on a voice feature,” so it should monitor GitHub issues and feedback in Slack and Twitter and whatever for people saying that there are bugs in voice or that they have new feature requests, and it just makes the routine for itself to monitor for this.
It’s actually not that far away, but I think this is an imminent next step… Claude should probably decide to actually listen for feedback on your feature and then decide how to notify you on its ideas. So the engineer doesn’t need to set up an automation, but Claude just thinks, “OK, this is what you work on, so let me monitor it and then propose what you could do today.”
Ars: Developers using these tools are frustrated that there’s just not enough compute to go around. The limits are a problem.
... continue reading