Back to feed

datasette-ip-rate-limit 0.1a0

Simon Willison's Weblog

May 14, 2026

5/14/2026

Selective Exemption Of Static Assets And Turnstile Routes From Rate Limiting Improves Reliability

datasette-ip-rate-limit 0.1a0 · Simon Willison's Weblog

Science, Technology & Innovation · May 14, 2026

Production config exempts static assets (/static/*) and anti-bot challenge routes (/-/turnstile*) from rate limiting so core site delivery and verification flows aren’t throttled while IP-based limits target expensive or scrape-prone endpoints.


5/14/2026

AI Generated Configurable Rate Limiting Plugin Enables Faster Production Mitigation

datasette-ip-rate-limit 0.1a0 · Simon Willison's Weblog

Science, Technology & Innovation · May 14, 2026

An AI (Codex/GPT-5.5 xhigh) generated a configurable rate-limiting plugin that was deployed with a concrete YAML policy on datasette.io to mitigate crawler pressure, demonstrating AI-assisted creation of reusable production infrastructure and faster incident-to-mitigation workflows.


5/14/2026

Targeted Path-Specific Rate-Limiting Curbing Crawler Abuse On Specific Endpoints

datasette-ip-rate-limit 0.1a0 · Simon Willison's Weblog

Science, Technology & Innovation · May 14, 2026

A surgical, path-specific rate-limiting plugin reads client IPs from the Fly-Client-IP header and blocks any IP that exceeds 60 requests in 60 seconds for 20 seconds, but only for two hot URL families (/global-power-plants/* and /legislators/*) and up to 10,000 tracked keys to suppress scraper spikes while preserving access elsewhere.