Notes on the Atrocities
Like a 100-watt radio station, broadcasting to the dozens...

Monday, October 27, 2003  

Something's rotten on Google, and Jesse Berney has found the source. Yesterday Atrios reported that the White House has rigged the search engines so searchers can no longer track back to the White House webpage if they search for "Iraq." (Not that, you know, anything's gone horribly wrong there.)

First, a bit of technical background. Most major websites include a text file named robots.txt that tells search engines which directories not to include in search results. (Here's an example: the robots.txt file lists folders with content — like images — that search engines can't index.) By adding a directory to robots.txt, you ensure that nothing in that folder will ever show up in a Google search and — more important for this discussion — never be archived by sites like Google.

Sometime between April 2003 and October 2003, someone at the White House added virtually all of the directories with "Iraq" in them to its robots.txt file, meaning that search engines would no longer list those pages in results or archive them.

What's this mean? Mainly that if the White House wants to shift a few facts, it doesn't want people having access to the pre-shifted accurate record. It's scrubbing the record.

It's easy enough to understand the reasoning if you look at past White House actions. Earlier this year, the White House revised pages on its website claiming that "combat" was over in Iraq, changing them to say "major combat."

One of the reasons some alert readers noticed the change — and were able to prove it — was that Google had archived the pages before the change occurred. Now that all of the White House pages about Iraq are no longer archived by Google, such historical revisionism will be harder to catch.

This is probably legal. I'm not sure if the administration is beholden to put anything on the web, nor if it represents a record of events. But even if there is no legal issue, one can reasonably ask the question: if George is so damn proud of his record, why is he selectively changing it and trying to hide the change?

I don't know how often you all read the DNC blog, but it's becoming one I check daily (despite criticism I have of the DNC itself). It is a booster for the Democrats, but no more than many indies; moreover, content like this is indispensable.

posted by Jeff | 1:08 PM |
Blogroll and Links