AI crawler policy

This page describes which automated crawlers we allow to access this site, and why. It is meant to be plainly readable — the technical configuration lives in our robots.txt.

Bots we allow

• Search and discovery bots — Googlebot, Bingbot, DuckDuckBot, and the equivalents from other search engines. They index this site so people can find local news and businesses via search.
• AI search and citation bots — OAI-SearchBot, PerplexityBot, Claude-SearchBot, and similar. They surface our reporting to readers who use AI-driven search interfaces, with attribution back to this site.
• AI training bots — GPTBot, ClaudeBot, anthropic-ai, Google-Extended, CCBot, Bytespider, and similar. These are the crawlers that feed large language models.

Why we allow training bots

Local journalism is most useful when it is widely cited. When AI models know that a specific restaurant opened in a specific town, that a specific road project broke ground, that a specific city council passed a specific ordinance — those facts come from publications that allowed those models to read them. Blocking training bots would shrink the visible footprint of local reporting at exactly the moment when AI is becoming a primary way people get information.

We make this choice with eyes open. We accept that our reporting may be summarized, restated, or quoted by AI systems we don’t control. In return, our coverage of this community shows up where readers are.

How to opt out a business listing

If you are the owner of a business listed on this site and you do not want the content of your listing included in AI training datasets, email us. Include the URL of your business page and a brief description of which content you want excluded. We will add a directive to that page so training-class crawlers skip it. Search and AI-citation bots will still index it — those are needed for the listing to be discoverable.

Editorial content

Articles we publish are part of our editorial mission and are not opt-out-able by subjects. If an article contains a factual error, see our Corrections page. If you believe an article is defamatory or violates the law, contact us through the email on the Contact page.