The doc had been growing incrementally as each host got probed; reshape
it as a polished benchmark with TL;DR top, methodology section
(including safety constraints + caveats), per-host detailed tables,
final crawler settings, batch-50 walltime breakdown, and a reproduce
recipe.
Five hosts fully covered:
pro.lceda.cn API 5.0s -> 0.5s (10×)
lceda.cn doc 5.0s -> 0.5s (10×)
oshwhub detail 2.0s -> 1.0s ( 2×)
oshwhub listing 2.0s -> 1.0s ( 2×)
modules.lceda CDN 0.2s (already optimized)
Net effect on batch-50 plan: sleep total ~32min -> ~3min, walltime
~2h -> ~10-15min.
Key finding: the original 5s/req on Pro was set out of "logged-in
account is precious" caution with zero empirical evidence. Sustained
burst probe (25 distinct UUIDs at 0.5s, no recovery) showed 0/25 errors
and median latency 410ms — the caution was unjustified.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Knowit
ed713fa557