User-agent: * Disallow: /search-item.php Crawl-delay: 20 User-agent: Riddler Disallow: / User-agent: Mail.Ru Disallow: /search-item.php Crawl-delay: 2 User-agent: Yandex Disallow: /search-item.php Crawl-delay: 0.5