June 27, 2017

Google and the FTC | Peer to Peer Review

With a company mission statement that reads “Google’s mission is to organize the world’s information and make it universally accessible and useful,” charges of antitrust in business dealings might seem unexpected. Yet Google has been investigated over complaints of unfair business practices both in the United States and Europe. This is because of what is absent from the mission statement, which is that Google’s search results exist to deliver potential customers to companies with services or goods to sell. As Siva Vaidhyanathan says, we (the users) are not Google’s customers, we are its product. The advertisers are Google’s customers.

Most recently, on January 3, 2013, the U.S. Federal Trade Commission dropped its investigation into allegations of “search bias” against Google. This particular complaint was brought after Google developed Google Shopping and was accused by competitors of favoring its own products in search results. The complainants stated that Google’s entry into ecommerce pushed competitors to a lower position in the search results, sometimes even demoting them off the key first page of results, which for an online vendor is akin to Siberian exile. With its 70 percent share of U.S. Web searches, a product’s position in Google’s search results can make or break a company.

If you saw the headlines about this FTC decision, you may have the impression that Google was completely exonerated, and that the issue is closed. However, a closer reading of the documents, and in particular the footnotes, reveals more. The FTC did indeed find that Google had modified its search results (as it does, according to Google, about 500 times per year) in a way that negatively affected the page position of certain of its competitors. The Commission, however, came to the conclusion that “… Google would typically test, monitor, and carefully consider the effect of introducing its vertical content on the quality of its general search results, and would demote its own content to a less prominent location when a higher ranking adversely affected the user experience.” They came to this conclusion based on demonstrations and documentation provided by Google. Unlike the common practice in science, where results must be reproduced independently by others before they are declared valid, Google alone is allowed to manage the input and output from its search black box with no corroboration. What Google did here was more like magic than science: “Look, nothing up my sleeve. Oh, where did that rabbit come from?”

The Commission also accepts Google’s definition of “positive user experience,” measured as the number of click-throughs on advertiser links. (For extra credit: what is the meaning of a “click”?) The Commission excused changes that moved rival sites off the first page as “… improving the overall quality of Google’s search results because the first search page now presented the user with a greater diversity of websites.” How they reached this conclusion is not known. As a New York Times article put it: “… many of the yardsticks the commission used to measure its outcomes were remarkably similar to Google’s own. Not surprising, they cast Google in a positive light.”

A lengthy footnote to the Commission’s brief (four-page) report describes two areas where Google was found to be engaged in inappropriate practices, and which it has promised never to do again. (It is not clear why these were only presented in a footnote.) The first of these was that Google had scraped content off of competitors’ web sites and presented it as its own. The other was that Google placed restrictions on its Adwords API that hindered the ability of companies to advertise simultaneously on multiple search engines. Google’s “promise” is in the form of a letter to the Commission, stating what practices it will eschew in the future, and promising that ” If Google receives written notice from the Commission … Google will, within 60 days, address the Commission’s concerns or explain to the Commission why it believes that it has acted in a manner consistent with its commitments.”

This should be ringing alarm bells throughout the land. Google’s letter is non-binding, which is not how the Commission usually deals with antitrust violations. In addition, Google defines its own terms, and allows itself the option to simply explain to the Commission that what it is doing is just fine. There are no penalties if the Commission does not agree. The message here is “Trust us, we’re Google.” However, in a dissenting statement, one of the Commissioners stated:

Our “settlement” with Google is not in the form of a binding consent order and, as a result, the Commission cannot enforce it by initiating contempt proceedings. The inability to enforce Google’s commitments through contempt proceedings is particularly problematic, given that the Commission has charged Google with violating a prior FTC consent agreement.

It would be hard not to conclude that Google is being given special treatment. Some of this treatment may be the result of Google’s ability to snow the Commissioners. Google’s famed PageRank is a shell game of humongous proportions. Google defends its ranking by describing it as an algorithm, implying that the decisions are made on the basis of data-crunching by unbiased machines. What members of the commission may not know is that this is also an algorithm: if pageowner=microsoft.com, PageRank=PageRank-100. (That is, if the page belongs to Microsoft, demote the page’s rank by 100 points.) Algorithms can be as biased as those creating them would like them to be. Google’s PageRank algorithm remains a mystery, although we know that Google can and does target individual sites for demotion or actual disappearance from its results. Using the complexity of the algorithm and the number of changes that it makes to it each year, Google argued against any decisions that would require it to open up the algorithm to the Commission. Instead, the Commission agreed to accept Google’s explanation of the ranking results.

The Commission also seems to have missed something significant in Google’s “promise” letter, something that we should see as a clue to the Commissioners’ lack of technical expertise. Google outlines an “opt out” action for those sites that do not want Google to scrape and re-use their data. However, in a footnote, Google states: “Web site owners will be permitted to exercise the opt-out on a domain name basis.” They go on to explain that the opt-out cannot be applied to sub-domains or sub-directories in a domain. This is the classic “all or nothing” opt-out ploy, where the opt-out is defined so broadly as to force the user to forego desired services along with the undesired ones if they wish to opt out. (Note that the standard “opt out” on the Web, the robots.txt file, does allow users to define specific directories, and can be applied to sub-domains.) It would mean that a site could not bar Google from those areas where they have competing products and at the same time get the advantage of Google’s presentation of their results in other product categories. My guess is that Google took a close look at the sites of competitors who might wish to opt out, and found a significant number where they would have more to lose with a broad opt-out than they would gain from the elimination of Google’s competitive use of their data.

The reason for the Commission’s reluctance to dig into PageRank may be explained by a well-founded fear of its overwhelming complexity. The Commission’s report states that over the two-year period of their investigation, they reviewed over 9 million pages of documents from Google and others, and undoubtedly much of that was technical in nature. That is over 25,000 350-page books. If nothing else, the Commission knows that at the click of a mouse, Google can bury it in PDFs or billions of lines of incomprehensible code. It is also clear that no one today, much less the FTC, has the ability to prove or disprove any statement that Google makes about its PageRank, and most likely no one could do so without Google’s help and consequent influence.

Yet it is still hard to accept the acquiescence of the Commission to Google’s viewpoint. It does make me wonder if, after watching Men in Black, Larry and Sergei haven’t gone into their laboratory and come out with the neuralyzer that makes people believe what Google tells them. Only the Men in Black were the good guys. And they were funny.

About Karen Coyle

Karen Coyle (kcoyle@kcoyle.net) is a librarian with over thirty years of experience with library technology. She now consults in a variety of areas relating to digital libraries. As a consultant she works primarily on metadata development and technology planning. She is currently investigating the possibilities offered by the semantic web and linked data technology.

Create a Maker Program in Your Library
School Library Journal’s newest installment of Maker Workshop will feature up-to-the-minute content to help you develop a rich maker program for your library. During this 4-week online course, you’ll hear directly from expert keynote speakers doing inspiring work that you can emulate, regardless of your library’s size or budget. Course sessions will explore culturally relevant making and how to assess your community’s needs, mobile maker spaces, multi-media, and more!


  1. The FTC weren’t snowed by Google. Give them more credit. What they determined was that they rushed into an investigation at the insistence of competitors who can’t beat Google in the market and found that the entire concept of the investigation was counter to the principles the FTC is there to uphold.

    Search bias? Of course Google practices search bias. It’s built into the algorithm. It’s also called quality. Google created and adjusts Page Rank to provide searchers with the results they want. If Google isn’t meeting searcher needs – if, that is, the adjustments they make create an inferior service, searchers will go elsewhere. As Google gets around 70% of the search business, it seems they are doing a pretty good job.

  2. I think Google or yahoo imperialism is true to some extent and it is necessary to perform some actions in this regard.