# 2008-11-17 Yanga, Yandex # 2007-11-01 favorstarbot # 06/04/07 Krugle, Nutch # 16/03/06 sproose, iVia, DataFountains # 12/12/05 MJ12bot # 11/10/05 DTAAgent, SBIder # 07/09/05 updated, UbiCrawler # 19/07/05 PRiBONbot, OmniExplorer_Bot # 19/05/05 AtlocalBot # 22/04/05 IRLbot # 18/04/04 Uptimebot # 21/03/04 Linkwalker # 27/02/04 Almaden User-agent: Yanga Disallow: / User-agent: Yandex Disallow: / User-agent: favorstarbot Disallow: / User-agent: Krugle Disallow: / User-agent: Nutch Disallow: / User-agent: sproose Disallow: / User-agent: iVia Disallow: / User-agent: DataFountains Disallow: / User-agent: MJ12bot Disallow: / User-agent: DTAAgent Disallow: / User-agent: SBIder Disallow: / User-agent: updated Disallow: / User-agent: UbiCrawler Disallow: / User-agent: PRiBONbot Disallow: / User-agent: OmniExplorer_Bot Disallow: / User-agent: AtlocalBot Disallow: / User-agent: IRLbot Disallow: / User-agent: Uptimebot Disallow: / User-agent: LinkWalker Disallow: / # Picsearch User-agent: psbot Disallow: / User-agent: sitecheck.internetseer.com Disallow: / User-agent: Art-online Disallow: / User-agent: NPBot Disallow: / User-agent: http://www.almaden.ibm.com/cs/crawler Disallow: / # All robots - index all dirs/pages exc. resource User-agent: * Disallow: /resource