#robots txt info from SH1's `vnn-robots.txt` file User-agent: Teoma Disallow: / User-agent: twiceler Disallow: / User-agent: Gigabot Disallow: / User-agent: Scrubby Disallow: / User-agent: Robozilla Disallow: / User-agent: Nutch Disallow: / User-agent: baiduspider Disallow: / User-agent: naverbot Disallow: / User-agent: yeti Disallow: / User-agent: * Disallow: /?attachment_id=* Disallow: /ad-group-sitemap.xml Disallow: /files Disallow: /login Disallow: /wp-admin Disallow: /wp-content/blogs.dir Disallow: /wp-content Disallow: /wp-json Disallow: /wp-login.php*