# Define access-restrictions for robots/spiders # http://www.robotstxt.org/wc/norobots.html #Set crawl delay so we do not get clobbered User-agent: YandexBot Crawl-delay: 1000 Disallow: / User-agent: BubiNG Disallow: /