User-agent: Mediapartners-Google Disallow: User-Agent: AhrefsBot Disallow: / User-Agent: Bender Disallow:/ User-agent: MJ12bot Disallow: / User-Agent: MSIECrawler Disallow:/ User-agent: NPBot Disallow: / User-agent: msrbot Disallow: / User-Agent: Open* Disallow: / User-Agent: Twiceler Disallow: / User-Agent: Scooter* Disallow: / User-Agent: WebSearch* Disallow: / User-Agent: ZyBorg* Disallow: / User-Agent: Wget* Disallow: / User-Agent: TurnitinBot* Disallow: / User-Agent: SlySearch* Disallow: / User-Agent: Seekbot* Disallow: / User-Agent: NetResearchServer* Disallow: / User-Agent: Seekbot* Disallow: / User-Agent: mozDex* Disallow: / User-Agent: Link* Disallow: / User-Agent: larbin* Disallow: / User-Agent: ia_archiver Disallow: / User-Agent: Charlotte Disallow: / User-agent: 008 Disallow: / User-agent: Yandex Disallow: / User-agent: discobot Disallow: / User-agent: * Disallow: /_/search/ Disallow: /_/print/ Disallow: /_/Users/ Crawl-delay: 1 Sitemap: http://harte.thefreelibrary.com/sitemap.xml.gz