User-agent: LinkpadBot Disallow: / User-agent: Sidetrade indexer bot Disallow: / User-agent: Yandex Disallow: / User-agent: YandexBot Disallow: / User-agent: GrapeshotCrawler Disallow: / User-agent: Seekport Crawler Disallow: / User-agent : Barkrowler User-agent : Barkrowler/0.9 Disallow: / User-agent: Pandalytics/1.0 User-agent: CCBot/2.0 User-agent: SurdotlyBot User-agent: Cincraw/1.0 User-agent: Twingly Recon-Klondike/1.0 User-agent: YaK/1.0 User-agent: DF Bot 1.0 Disallow: / User-agent: arquivo-web-crawler Disallow: / User-agent: Gluten Free Crawler/1.0 Disallow: / User-agent : serpstatbot/1.0 Disallow: / User-agent: DomainCrawler Disallow: / User-agent :arquivo-web-crawler Disallow: / User-agent: PetalBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: dotbot Disallow: / User-agent: A6-Indexer Disallow: / User-agent: AhrefsBot Disallow: / User-agent: AlphaSeoBot Disallow: / User-agent: AlphaSeoBot-SA Disallow: / User-agent: Applebot Disallow: / User-agent: AspiegelBot Disallow: / User-agent: Blackboard Safeassign Disallow: / User-agent: BLEXBot/1.0 Disallow: / User-agent: BLEXBot Disallow: / User-agent: Bytespider Disallow: / User-agent: crawler4j Disallow: / User-agent: DotBot Disallow: / User-agent: Gigabot Disallow: / User-agent: LieBaoFast Disallow: / User-agent: MauiBot Disallow:/ User-agent: Heritrix Disallow:/ User-agent: MauiBot (crawler.feedback+wc@gmail.com) Disallow: / User-agent: MegaIndex.ru/2.0 Disallow: / User-agent: MqqBrowser Disallow:/ User-agent: Nimbostratus-Bot/v1.3.2 Disallow: / User-agent: Seekport Crawler Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: SeznamBot Disallow: / User-agent: SputnikBot/2.3 Disallow:/ User-agent: The Knowledge AI Disallow:/ User-agent: TurnitinBot Disallow: / User-agent: UCBrowser Disallow: / User-agent: yacybot Disallow: / User-agent: Yeti Disallow: / User-agent: YisouSpider Disallow: / User-agent: * Disallow: /search Disallow: /lxbcodes Disallow: /get-lexbase Disallow: /ajax Disallow: /logout