Google in 1999: Search engines escape the portal matrix

Google in 1999: Search Engines Escape the Portal Matrix Like Morpheus in The Matrix, Google gave web users a stark choice in 1999: take the red pill and experience a new world of search quality, or choose the blue pill and stick with the bloated world of portal search. Google founders, 1999; photo by William Mercer McLeod. "Aren’t you rather late to the game?" It's January 1999 and that question was put to Google's young founders, Larry Page and Sergey Brin. The interviewer was Karsten Lemm from a German startup magazine called Stern. Lemm, who later in the article cited AltaVista and Excite as "established search engines," was right to be skeptical. After all, Page and Brin's company, Google, had only been incorporated four months ago. "It’s possible to do a much better job on search, and it’s the main application that people use on the Internet," replied Page. "So there’s a big opportunity, because if you do a better job really matters to people. People make decisions based on information they find on the Web. So companies that are in-between people and their information are in a very powerful position. There’s clearly space there for other players." "Users may not even realize but subconsciously they end up using your search engine because it works better for them," Brin added. "Users end up going where the search is best." Both Page and Brin were convinced they had "the best search" (as Page actually said in this interview). But it took time for the rest of the web surfing population, other than Google's early users, to notice. Google in January 1999. Six months later, on June 7, 1999, Page and Brin held their first press conference. Only a sprinkling of reporters turned up to the Gates Building in Stanford University, where the pair had started their search engine research project a few years before. Today they were announcing Google's first big funding round — $25 million from two of Silicon Valley’s most prestigious venture capital firms, Sequoia Capital and Kleiner Perkins Caufield & Byers. But despite the support of these two VC heavyweights, nobody in the room that day had any clue what kind of juggernaught Google would turn into over the coming years. Except perhaps for Page and Brin. Dressed in a white polo shirt adorned with the Google logo, Page — the 26-year old CEO — declared that Google’s goal was “to organize the world’s information, making it universally accessible and useful.” He said that Google would one day use artificial intelligence and have a million computers. It seemed as fanciful as the plot of The Matrix, which had been released to movie theatres just a few months before. Google's first press conference. It was true that Google was growing fast at the time — the press release stated that traffic had been “growing at a rate of 50% per month since Google’s inception, fueled only by word of mouth.” But it was still largely a user base of geeks in the know. Most people used the portal search engines — AOL, Yahoo, MSN, Netcenter, Excite, Alta Vista, Lycos, and some others. These products weren’t necessarily motivated to send their users to external websites; why do that, when portals made their money from on-site advertising? But Google was different. It had no advertising (proudly so, at this point) and its search box was the only thing users would see on its otherwise blank white home page. "No portal litter" — a tech magazine blurb about Google in 1999; via Reddit. Google hadn’t gotten much mainstream attention before June 1999. But one or two savvy tech journalists had picked up on its appeal. In February 1999, Newsweek reporter Steven Levy name-checked the fledgling search engine: “Google, the Net's hottest new search engine, draws on feedback from the Web itself to deliver more relevant answers to customer queries.” But Levy had only mentioned Google as an aside. His article was actually a profile of American entrepreneur Bill Gross, who (among other things) ran a search engine company called GoTo. But as Levy explained, GoTo was basically a ‘pay-to-play’ search engine. “If a GoTo user looks for ‘New York Yankees,’ the first 10 choices are paid advertisers (‘Buy Yankees gear at Fogdog Sports’). On the 11th try you finally get Yankees.com, the official site of the world champs. (On Google, this comes up first.)” GoTo.com, June 26, 1998. The search results on all the other portals were nearly as bad as GoTo. That’s if you could even find the search box on their busy web pages — bursting with news headlines, directory listings, weather widgets, horoscopes, chat and email logins, and so forth. What Google did differently was provide search results that people actually wanted to see, as a straight list of blue links on a plain white background. PageRank vs. Keyword Search The reason Google’s results were qualitatively better was because of an algorithm called PageRank, which ranked web pages based on how many other pages linked to it and how influential those pages were. The other search engines relied on keyword-based methods of ranking search results — how often a keyword occurred on a page and how strongly associated a keyword was with the search query. During 1999, search engines tended to be judged more on the size of their index. And by that measure, Google was roughly middle of the pack. A study conducted in February 1999 and published in Nature magazine in July found that Google had indexed 7.8% of the known web. AltaVista, perhaps the most well-known search engine of the time, had indexed 15.5% — putting it second on Nature’s list. A search engine called Northern Light was first on 16%. Nature search study, diagrammed by Search Engine Watch. “The publicly indexable World-Wide Web now contains about 800 million pages, encompassing about 6 terabytes of text data on about 3 million servers,” wrote Steve Lawrence and Lee Giles in the Nature article, entitled “Accessibility of information on the web.” Interestingly, they concluded that new “popularity” based forms of indexing — the authors specifically mentioned Google and a site called DirectHit — would decrease the accessibility of information on the web. The article concluded: “For ranking based on popularity, we can see a trend where popular pages become more popular, while new, unlinked pages have an increasingly difficult time becoming visible in search-engine listings. This may delay or even prevent the widespread visibility of new high-quality information.” This finding was quickly debunked by those who closely monitored the emerging search engine industry, such as Danny Sullivan from Search Engine Watch, a search news and tips website he founded in 1997. As Sullivan archly noted in his newsletter dated August 2, 1999, “the number of pages a search engine has indexed has nothing to do with whether it is any good.” Search Market in 1999 It was difficult to ascertain market share of the various search engines in 1999. In August, based on numbers from an analytics firm called Media Metrix, Sullivan estimated that “Yahoo had risen to its highest share ever” as the most popular search engine, with about 46% audience reach. In second place, on 40%, was MSN, “showing an incredible rise, probably helped by the earlier release of Internet Explorer 5.” AltaVista, Go (Infoseek), Excite and Lycos were among a group of search engines that had between 10-25% reach (it was common in those days for people to use multiple search engines). Google didn’t even rate a mention in Media Metrix's statistics. One factor that made it difficult to estimate the popularity of search engines was that some of them were licensed to other websites and portals. Inktomi had deals with Yahoo, MSN, Snap and HotBot. In a March 1999 article in The Wall Street Journal, Inktomi’s CEO, David Peterschmidt, compared his company to an “arms merchant” — implying that Inktomi would sell its backend search technology to the highest bidders. “We are a universal arms merchant," he said. "Inktomi doesn't care who wins. We care that they are using Inktomi's arms.” Inktomi's multifarious offerings; screenshot from March 1, 2000. Google was ostensibly following a similar strategy to Inktomi, at least for revenue generation. Since it wasn’t running ads, Google felt that its best chance to earn money was to license its technology. The first such deal was with Netscape, which had recently been acquired by AOL. On June 24, 1999, Netscape announced a new version of “Netscape Search,” largely powered by Google’s search technology. But unlike Inktomi, Google left open the option of turning its own site, google.com, into a major traffic destination. CEO Larry Page told Danny Sullivan in July 1999 that its goal was "to produce no holds barred, the best search available.” But he also didn’t rule out adding other features. “I won't say we won't add services,” Page cagily said, “but we wouldn't put free email on our site unless we thought we could do a much better job.” Search Engine Watch article about Google, July 1999. Feeling Lucky Perhaps the only non-vanilla feature that Google had at this time was the “I’m feeling lucky” button. It added a touch of serendipity to Google’s otherwise straightforward search engine. Chris Sherman explained its subtle, yet quietly powerful, impact in an article published in About.com on June 29, 1999: “Clicking this button skips search results altogether and takes you directly to the page Google thinks is the best match for your search. And it works so well it’s almost scary.” So those in the know had an inkling that Google was building something formidible. Heading into the new millennium, there was a sense that Google represented a bright new future for the internet. As Sherman concluded, “with hungry new players like Google pushing search technology into the next generation, things can only get better.” Google.com, November 29, 1999. The "beta" label well and truly gone now. Cybercultural is a website about internet history and its impact on our culture. Subscribe for free: Buy the Book My Web 2.0 memoir, Bubble Blog: From Outsider to Insider in Silicon Valley's Web 2.0 Revolution, is now available to purchase: Or search for "Bubble Blog MacManus" on your local online bookstore.