|
Sure. You can configure your server to only accept X number of requests per IP which will effectively rate limits Googlebot if they'd been coming at you with 10+ clients at a time.
Or if you feel like getting more complex, you can have your app server (PHP, Python, Rails) handle the rate limiting and send a 504 header (indicating a temporary overload) to Googlebot when its making too many requests.
The downside of each is that it might hurt your SEO if you do it badly, and Google sees your site as 'slow' rather than rate limited.
|