User-agent: * Crawl-delay: 5 Disallow: /search Disallow: /*?*sort=