Don’t block the bots. Build the gate
Media companies have gotten pretty good at blocking AI bots. They have been less good at deciding which ones should be allowed in. Reports say most major publications almost universally block crawlers from major AI companies like OpenAI and Anthropic, and many small to midsize publishers mirror that, configuring their robots exclusion protocol settings, or robots.txt, to keep them out. It’s an understandable stance, and it might even be appropriate. But it also crudely compresses a complex issue
Record details
Published: 17 July 2026
Source: Fast Company Tech
Category: News
Topics: Agents & autonomy
Retrieved: 18 July 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
KI lokal nutzen: LLMs und KI-Agenten ganz ohne OpenAI & Anthropic | c’t uplink
Heise Online (DE) · 18 July 2026
Microsoft joins Google in backing Go for AI agents — OpenAI and Anthropic lag
The New Stack AI · 11 July 2026
OpenAI, Microsoft & Anthropic agree on who runs the agent. They disagree on what you can take back.
The New Stack AI · 10 July 2026
Enigma raises $71M to develop foundation models for robots
SiliconANGLE AI · 27 July 2026
AI hackers are getting faster. The government may not be ready
Fast Company Tech · 29 July 2026
Everyone Is Freaking Out About OpenAI and Anthropic’s Race for Dominance
Wired · 30 July 2026
How to cite this record
ethics.ai (17 July 2026), “Don’t block the bots. Build the gate,” evidence record 11489, https://ethics.ai/record/11489 (originally published by Fast Company Tech).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.