# Block all other spiders User-agent: * Disallow: /customers/* Disallow: /customers/ Disallow: /faq Disallow: /api/* Disallow: /api/ Disallow: /api/insitefix Disallow: /terms-conditions Disallow: /r Disallow: /r/* Disallow: /r/ Disallow: /rr Disallow: /rr/* Disallow: /rr/ Disallow: /marketingoptout/* Disallow: /assets/* # Disallow: /learn/* # Disallow archive.org User-agent: ia_archiver Disallow: / User-agent: archive.org_bot Disallow: / # Disallow Vamoose User-agent: Vamoose-Spider Disallow: / User-agent: BUbiNG Disallow: /