The crawl token is rate-limited to keep it available to everyone and to stop it from turning into a bulk scraper. I don't have the exact request cap published yet. When the limit is known, it will be on the site with the token.
If your AI worker hits the limit, the fetch returns an error and the worker stops pulling that chapter or module spec. Nothing is lost. You can wait for the window to reset, or have the worker retry with backoff. The 206-page PDF is free and has the same content, so you can download it and let your worker read from that copy instead.
The kit is free. The AI crawl token is free too. What costs you is time: a stalled worker is a stalled build. Don't let a rate limit idle your evening.
If the token keeps failing, email me and I'll look at it.
Mouldi
