CDT Critique of Searchability of Federal Websites

The Center for Democracy & Technology reports that many important information sources within the federal government are essentially hidden from the very search engines that the public is most likely to use in the Center's Hiding in Plain Sight: Why Important Government Information Cannot Be Found Through Commercial Search Engines.

CDT Recommendations to Help Agencies Ensure Their Content Is Accessible to Search

• Congress should pass the E-Government reauthorization act, which would require OMB to create best practices to encourage searchability of federal Web sites.
• OMB should officially recognize the importance of commercial search to Internet users and work with the CIO Council to adopt policies to help users find information.
• Agencies should adopt an information policy that makes public accessibility of online content and resources a priority.
• Agencies should create Sitemaps of content on their sites, with special attention given to materials stored in databases and accessible only through drop-down menus. For example, many agencies have FAQ databases that are not accessible to search crawlers but contain very succinct and useful answers to common questions.
• Agencies should review their use of robots.txt files in order to ensure they are used in the least restrictive way possible. Every effort should be made to include, rather than exclude, materials from the website, whether materials were excluded purposefully or accidentally in the past.

Hat tip to Mark Giangrande (DePaul Law Library), Tech Law Prof Blog, who writes, "This is just shocking news, except possibly to reference librarians who regularly use government web sites." Yup. [JH]