GoKawiilAs projects like Pacing the Frontier look to major labs as a way to keep AI research safe, open-source models have become a sore spot for the industry. With free distribution and little control over how they’re used, open-weight models aren’t easily controlled, leading some labs to treat them as downright scary.
But at the Ai4 conference in Las Vegas last week, three of the world’s most respected AI researchers — Nobel Prize winner Geoffrey Hinton, World Labs CEO and co-founder Fei-Fei Li, and Coursera co-founder Andrew Ng — spoke out on the issue. And while they disagreed on particular tactics, all three made a powerful case for keeping AI open.
For the three speakers, the core concern was allowing a handful of major AI companies to control the pace of progress. When a few companies control access to a technology, as Apple and Google do with mobile operating systems, innovation can slow and the companies that control the platforms can influence what gets built on them.
Andrew Ng said that he worried about a similar dynamic emerging in AI. “I don’t want there to be gatekeepers,” Ng said. “That limits how all of us can access AI.”
Companies have an incentive to protect their competitive advantages, including by influencing the rules that govern the industry. That could create a dynamic where only the largest, best-capitalized firms with the resources to build the most advanced AI systems.
Ng’s solution was to maintain multiple providers, with models and companies competing rather than allowing a handful of of players to dominate the field. “If I were to try to give one prescription, it would be to promote openness,” Ng said, “because AI is amazing technology and I want it to be in everyone’s hands.”
But not everyone agreed that open-weight models would help preserve that state of play. Hinton, in particular, drew a distinction between open-source software, which makes the underlying code available for inspection and modification, and open-weight models, which release the parameters of a trained AI model to the public.
“Open source is great. You show people the code, and lots of people look at the lines of code and say, ‘Oh, there’s a bug.’ Open weights means you train a big model and then you give people the weights. That’s very different,” Hinton said. “I was against open [weights] because it makes it so easy for people to take these big foundation models, which are very expensive to train, and for much less money train them to do bad things like cyber attacks.”
But whatever his reservations, Hinton acknowledged that open-weight models are already a permanent fixture of AI. “I think that battle’s been lost. We now have open-weight models, so the barrier to lots of people getting these big models, which was the cost of training foundation models, that barrier has disappeared. It’s too late.”
Yet accepting reality didn’t mean ignoring the risks. Hinton’s position was clear: AI would continue to advance, and he thought that was largely a good thing. He said it would boost productivity and improve education and healthcare. “Worrying about the possible bad effects of AI and the things that intelligent beings might do when they’re smarter than us. I don’t think that’s unfair. I think it is unfair to label anybody who thinks like that as a fear-monger,” Hinton added.
... continue reading