Looking for Host Not Crushed by Search Engines
 

Amerinoc is apparently crushed server wise on VPS packages when Search Engines crawl sites.
Can any host make an offer to move?

What makes you say this? I am seriously curious. I have seen somme site ranking go down over the past few months (mainstream).

Anything to back it up?

Yeah, our server crashing last night. Its getting to be the weekly se crawl/crash syndrome. Was told by support SE's crawling our sites literally crashed the server. There were no other traffic spikes at the moment.

Just looking for a more robust option if available.

Server crash because a spider crawls it's sites...? wow... just wow.

Yes, very sad. Funeral is at 3pm

This doesn't sound right.

Have they provided you any other information ?

Agreed - that doesn't sound right. Sounds like you might have a script issue or misconfiguration in apache somewhere.

Webmaster tools lets you throttle the indexing, by the way...

hmmm this is highly unusual unless it is vps with 32mb of RAM.

Anyway by any chance you are running WP sites on that server? And must be one at least on the server, who dosen't have it :) nowdays. The real culprit could be xmlrpc.php in root of wp site. I have seen recently many attacks on this file and one Quad Core server which usual load is about 2-3 constantly, suddently go high up to 40 because of those attacks from many bad bots trying to hacks wp site thru xmlrpc.

Run this against access.log (apache or nginx) to see if there are many access attempts to xmlrpc.php

grep xmlrpc access.log | cut -d' ' -f1 | sort | uniq -c | sort -rn | head -n8

Those attacks are usually frequent on weekends when "hackers" are out from school and they are hacking sites from their free amazone instances ;)

I forbid access to xmlrpc.php in nginx and they are gone; since then server is cool.

More info here Disable XML-RPC in WordPress to Prevent DDoS Attack - BlogAid

Hope this will help someone

:winkwink::thumbsup

Thats pretty helpful. Thanks a lot. Yes there are some WP sites and bots have been hitting certain files, usually search or jpg related.

What are the specs of the VPS and how many wordpress sites do you have on it ?

come to talk to us, i am sure we can make you a good offer for a stable server for you. We have never had a server crash because of crawlers.

That may be true, but there are more than enough shitty reviews of your services. So I would suggest the OP decline your offer. :2 cents:

Yeah, must be good server. :)

Though I am not sure do I believe this shit. If it is Google they can for starters limit the crawling intensity from webmaster tools, but site crashing because of SE spiders.. come on.. :)

Make sure you have something setup to block wp-login.php bruteforce attacks, those can really cause cpu load to skyrocket. I run a lot of of Wordpress blogs on LEMP Stacks and have it set to only allow my IP through to wp-login.php and then I have Fail2Ban block any user who attempted to try to bruteforce (along with banning anyone who searches for Wordpress exploits). You could also install the popular Wordpress plugin "Limit Login Attempts" which will help against bruteforce attacks as well but I prefer the only allowing my ip method.

Is it a managed VPS?

I will get that to you. It does turn out we were being hit by UA and ID IPs at the same time the SE's crawled. I am guessing the bot hits were timed. They've been doing it off and on since late July.

ID - 202.67.41.51 ++ others
UA - 91.200.12.71 ++ others

Toss in HK, VN, CN and KR from time to time. We've had to add more than 70 block rules on the cloud service and add more each day.

We ramped up security on the wp log ins. Now they are shut off after 2-3 attempts.

Sent you a PM, AK.

Ukraine keeps pounding our site. Now its, 91.200.12.116 - which is being blocked in cloudflare but still gets through.

:1orglaugh:1orglaugh Ya Bad spiders!

I've been on all different kinds and sizes of servers on AmeriNoc for over 12 yrs and love the company.. Hence I list them down there in that area.. But as others said look for the real problem before jumping ship or you will be in the same boat on a different server :2 cents:

Replied. I'm sure this can be mitigated.

Maybe that you go bash some else instead of me. If you have a problem with me than talk to me directly do not go about and talk shit about our service. I have had plenty of clients coming from the big guys who deliver shitty service. Those three people who are not happy are not representing our service.

So take your petty shit else where.

Love the guys at Amerinoc. Great guys, helpful, supportive and knowledgeable. We've discovered that if you use Amerinoc and decide to add cloudflare you must sign up to cloudflare on your own to get full DNS protection.

Amerinoc did this as a favor to us because we are not tech heads (signing us up). However we are learning we have to do it ourselves and grunt it out learning the tech of it.

But you haven't made the switch to Cloudflare yet ? Shoot me another PM and I will invite you into one my my Slack channels and help you get this fixed.