The future of search

Dirt is a daily email about entertainment.

Terry Nguyen, Dirt's senior staff writer, on the fragmented future of search.

Years ago, I met a man who insisted on conducting every single Google search in incognito mode. His compulsion was born by fear—fear of what his browsing history might reveal about himself, even if most of his queries, so he claimed, were innocuous. He was somewhat aware of the limits of browsing incognito: His search activity could still be visible to internet service providers and the organization managing the WiFi network. Yet he chose to hide behind this confessional curtain, as if it could fully shield him from Google’s panoptic eye.

“Google was invented so that people could learn about the world, not so researchers could learn about people,” wrote the data scientist Seth Stephens-Davidowitz, “but it turns out the trails we leave as we seek knowledge on the internet are tremendously revealing.” After years of sifting through anonymous Google search data, Stephens-Davidowitz argues in his book Everybody Lies: Big Data, New Data, and What the Internet Reveals About Who We Really Are that it may be “the most important dataset ever collected on the human psyche.”

On the English-speaking internet, “Google” is a transitive verb for searching things up. But it has also become a familiar entity, a confidant for our curiosities, fears, desires, and beliefs. During the pandemic, people treated the search engine as a pocket health care provider, using it to investigate their symptoms for self-diagnosis.

This behavior is not exactly antithetical to Google’s mission. It underscores the expansive and private psychology behind users’ search intentions. In addition to facts, more people are seeking emotional responses for affirmation or assurance. However, Google is quite lousy at answering pathos-driven queries. Its top results are not always helpful, even for simple factual searches. Users have remarked on its decline for years. Even Marissa Mayer, who designed Google search’s early interface, has admitted that she’s seeing “more ads, more links that might as well be ads, and more links to spammy web pages.” But over the course of Google’s two-decade reign, few noteworthy competitors have surfaced—until now.

We are reaching an inflection point with search technology. Microsoft and Google are publicly experimenting with large language models (LLMs), deep-learning algorithms trained on vast amounts of existing text and user data to answer queries. In February, both companies released AI-automated chatbots that can “assist” users on their search journeys, prompting media buzz of the impending “search engine wars.” TikTok, over the past year, has made strides in its search capabilities. Its emergence as a discovery tool reveals the possibilities for search in a fragmented content landscape. Users have grown accustomed to tailoring their searches by platform for better results and more personalized answers: Amazon for products, Depop for secondhand clothing, Yelp for restaurants, YouTube for tutorials, Reddit for personal anecdotes. Where does that leave Google and the future of a centralized search engine?

In Jorge Luis Borges’ 1941 short story “The Library of Babel,” the universe is a vast library that holds ​​an unfathomable number of books, presumed to contain every possible ordering of letters and symbols. The internet is no omniscient library, but the parallels between Borges’ story are apparent. The web is a boundless compendium of information and data scattered across billions of pages—content, as we’ve loathed to call it. In this virtual information library, Google may be the closest thing we have to a librarian.

Since the 1990s, the company has managed an extensive index of websites, organizing and delivering information through a ranking system which can be influenced by search engine optimization, or SEO. This system cemented Google’s role as a reliable navigator, directing users to sites with the information they were searching for, from recipes to news. But reliability is no guarantee of quality.

Consider my recent Google search for “local restaurants in lisbon portugal.” My query yielded three recommended places. There were two restaurants in Manhattan, close to where I conducted the search: a bar called Local NYC on 33rd and a Local on Sullivan Street (the establishments are unassociated), before the Lisbon-based eatery A Provinciana, which has a 4.5-star rating and 2,000 Google reviews. Below these recommendations were a list of suggested queries, an algorithmic presumption that I wished to know what other people were searching for (I did not). Then came the SEO-optimized articles from Hotels.com, Devour Tours, CN Traveller, and similar travel blogs. Headlines from these sites are often stacked with keywords, like authentic, best, hidden, and essential, to entice the wayward user, even though most people have an implicit understanding that these results are not very good.

“Google results don’t feel authentic, or even written by a real human because the top articles are all in this keyword-loaded language,” said Dmitri Brereton, an engineer who researches search engines and AI. “Social media, especially TikTok, solves that authenticity problem because some experiential things are just better seen. It can’t get more authentic than a video of a person dining at a restaurant.”

To make matters worse, the sheer number of URLs has grown exponentially since Google search’s launch, while low-quality content has steadily proliferated. The prevalence of “keyword-loaded language” is a byproduct of Google’s ranking system and a more crowded internet. As more marketers learned how to game the system, SEO-driven websites and link farms have begun to dominate search results. Their strategy relies on manipulating Google’s search algorithm for page views, which in turn generates ad revenue.

Rather than wade through the first few pages of Google, users are looking elsewhere to fulfill their search needs. With TikTok gaining steady traction among young users, the platform has emerged as a looming threat to Google’s dominion. In 2021, TikTok overtook Google’s 15-year reign as the world’s most popular web domain, according to data from CloudShare. Recent data from Morning Consult also found that Gen Z-aged adults are less likely to use Google search to investigate major news events, compared to older users. Instead, more are turning to TikTok. (A plurality of polled Gen Z-ers still depend on Google search for news; the age group just has “a distinct preference” for using TikTok for news.)

Trend pieces began addressing this search-oriented vibe shift last year. “Move Over Google. TikTok is Gen Z’s Go-To Search Engine,” AdWeek reported in August. “For Gen Z, TikTok is the New Search Engine,” reads a New York Times headline from September. Even tech reporters have experimented with only searching for answers on TikTok as a Google substitute. In these trend stories, Gen Z users are portrayed as search aberrants, keen on finding information as fast as possible at the cost of accuracy. The algorithm “knows what I want to see,” a 22-year-old user told the Times. “It’s less work for me to actually go out of my way to search.”

The data point that appeared to ignite the frenzy towards “TikTok SEO,” as some marketers call it, can be traced to an offhand comment made by a Google executive last July: “In our studies, something like almost 40% of young people [between the ages of 18 and 24], when they’re looking for a place for lunch, they don’t go to Google Maps or Search,” said Prabhakar Raghavan, Google’s senior vice president. “They go to TikTok or Instagram.” The type of queries and expectations that younger users have with search are “completely different,” he added. They’re longer and more conversational.

The concept of “search” has expanded beyond Google’s domain over the past decade, prompting users to modify their search patterns and proclivities. The search giant pales in comparison to Reddit and TikTok when it comes to opinion-based or experiential searches, where users seek out multiple opinions and anecdotes about real world experiences. (That’s why users have begun to append “Reddit” to their search queries—an observation Brereton made in a viral blog last year on Google search’s inefficacy.)

Like Facebook and YouTube before it, TikTok has constructed a mini “walled garden,” or enclosed ecosystems wherein user data is accessible only to the organization. TikTok is its own island of searchable content: Users are prompted by potential search queries while scrolling, and certain video comments are search-enabled via a magnifying glass symbol.

The company has worked hard to outgrow its reputation as a lip-syncing app for teens. TikTok has increasingly positioned itself as a utility for people of all ages, rather than an entertainment platform. The #TikTokTaughtMe campaign, launched last June, encouraged users to share real-world knowledge they’ve gleaned from the platform: the proper way to wear a U-shaped travel pillow, how to fold a fitted sheet, the fact that flamingos are born gray.

A commercial aired in the UK also leaned into this instructional ethos: A woman in her 20s helps her middle-aged father move into a new home. The father whips out his phone and opens TikTok to show her a nifty carpet cleaning hack. Later, the duo uses TikTok to evenly hang up a picture frame, search for book recommendations, and prepare a quick dinner. No matter what your age or your interests, the ad implied, you can always search and learn something new on TikTok.

“TikTok is already eating a chunk of Google’s pie,” Brereton said. “If somebody else uses language models intelligently for search, they can also eat a part of the pie.”

The second half of this piece will run next Friday, March 10. Dirt's paid subscribers can read its entirety here.

 🌱 JOIN THE DIRTYVERSE

• Join our Discord and talk Dirt-y with us. It’s free to join! Paid subscribers have access to all channels.

• Follow @dirtyverse on Twitter for the latest news and Spotify for monthly curated playlists.

• Shop for some in-demand Dirt merch. 🍄