# 3D Print Everything — https://3dprinteverything.org
#
# One group, every crawler. Search engines and AI assistants are welcome on
# every public page; the paths below are logins, customer links and the API.
# (2026-10-01: these used to be eight groups that had drifted apart — the AI
# crawlers' groups never got /quote-approval — and "Disallow: /portal" also
# blocked the public /portal-info page, because robots rules are prefixes.)
#
# The AI crawlers are named so that each has an explicit Allow of its own.
# Keep the rules below identical for all of them: a crawler obeys only the
# group that names it, so a separate group is a separate copy to keep in step.
#
# /api/ is closed except for two doors (2026-10-01). llms.txt and /ai-ordering
# tell AI assistants to start at GET /api/agent/capabilities, and the gallery's
# photos are served from /api/storage/gallery/ — "Disallow: /api/" shut both.
# The longest matching rule wins, so the two Allow lines beat the Disallow;
# every other /api/ path, customer files included, stays closed.
#
# The agent door is the one document, not the folder (2026-10-01). It was
# "Allow: /api/agent/", which also opened GET /api/agent/order/<id>?token=...,
# a customer's order status, to any crawler that came across a status link.
# No "$" on the end: not every crawler reads it, and one that doesn't would
# treat it as a literal character and lose the door. Nothing else starts with
# /api/agent/capabilities. The server also sends "X-Robots-Tag: noindex,
# nofollow" and "Cache-Control: private" on every other /api/agent/ answer
# (server/agentOrders.ts).

User-agent: *
User-agent: Googlebot
User-agent: Bingbot
User-agent: GPTBot
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: ClaudeBot
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: anthropic-ai
User-agent: PerplexityBot
User-agent: Perplexity-User
User-agent: Bytespider
Allow: /
Allow: /api/agent/capabilities
Allow: /api/storage/gallery/
Disallow: /api/
Disallow: /staff
Disallow: /portal$
Disallow: /portal/
Disallow: /quote-approval
Disallow: /showcase
Disallow: /rush-quote
Disallow: /pay
Disallow: /order-status
Disallow: /vendor
Disallow: /unsubscribe
Disallow: /refer/
Disallow: /appearance
Disallow: /beta
Disallow: /model-forge

Sitemap: https://3dprinteverything.org/sitemap.xml