Twitter, search robots get welcomes from Obama White House

Twitter, search robots get welcomes from Obama White House

Presidential Twitter account gains followers; redesign erases indexing limits


Kottke, in an e-mail response to a question from Computerworld , said the material that was disallowed from being indexed by the Bush administration primarily consisted of "text-only duplicates of hundreds of pages. I would argue those text-only pages should be included in search indexes, but it doesn't look like there [was] a conspiratorial attempt to hide information."

Nevertheless, Kottke thinks that by eliminating the Bush disallow list on its first day in office, the Obama administration was sending out a symbolic message. "One of Obama's big talking points during the campaign and transition was a desire for a more transparent government, and the spare robots.txt file is a symbol of that desire," Kottke wrote.

The nearly blank disallow slate may not last long, though. "As the site grows and accumulates more content," Kottke added, "I'm sure they'll need to add to the directories and scripts that search engines shouldn't index."

Follow Us

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags twittersearchBarack Obama


Show Comments