Who to turn to when your VPS has been hacked...
We have run a successful IT support business for over 15 years! In which time we have always had the same hosting provider, not just for our own hosting but for all our customers.
As of December 2017 we ran the hosting for over 70 different domains and had never had a single problem. So guess my surprise when on 24th December (YES! Christmas Eve) I receive a text saying "Hey Will sorry to bother you on Christmas Eve but I think my websites been hacked, can you take a look please"...
Oh sh** here we go...
Yes the VPS has definitely been hacked!
Having a single website hacked isn't such a big deal, after all even though we provided the hosting for over 70 different domains we didn't necessarily do the website support for all these domains.
Plus fixing a single website that's been compromised can be as easy as just an hour or so work. However in this case it quickly became clear that our entire server has been maliciously hacked.
From my Mums house on Christmas Eve I quickly checked 1 websites, then the next and the next. All sites either didn't load at all or produced random popups for very random things, thankfully nothing too disgusting.
My VPS has been hacked and I have no idea what to do next...
I'll be honest, as I headed back home on Christmas day afternoon I was pretty much already out of ideas as for what to do. Even though we had used the same hosting company for 15 years in the last 18 months I had noticed their services had really been slipping. This whole thing already felt like it was going to be a mess!
On further checking when I got home in front of my laptop I had some really random results. Sometimes the sites worked fine, sometimes nothing at all and other times random re-directs and pop-ups to spammy sites.
It didn't matter if the sites where WordPress based or plain HTML sites. I accessed the FTP of my own site and I could clearly see random folders / files that I had never created. On top of this there was additional malicious code that had been added to core website files.
The problem at this point is I simply do not have the access needed to even begin fixing all 70 sites; I was going to have to rely on the hosting company, so I raised a support request (on Christmas day Ugh!) and had to call it a night.
My hosting company are useless!
You can probably already imagine the rants I went on during this time, none of which I really wish to re-live. If I was to spell out all the conversations I had with the support team this whole post would be way too long. The long short of it is this...
After 15 years of service, paying thousands of pounds worth of invoices bang on time EVERYTIME! They were unwilling / unable to rectify the hack.
But this is the kicker after 3 or 4 days of doing my best to hold everything together, trying everything I could with limited access to fix the issues, manually deleting files and folders, manually finding and removing malicious code I finally said the hosts "restore from the backup" to be clear restore the server backup from before Christmas eve.
The response I received to this simple request "sorry we don't have backups" my response to that "what the f***, what if you had a fire and the server was incinerated? Your seriously telling me you don't have server backups? Disaster recovery backups??"
Hosts response "No sorry we don't do that"...
What to do when your VPS has been hacked and you need to change VPS?
Needless to say this was the last straw I needed a new hosting provider. My first thought was bluehosts, I listen to a lot of podcasts and bluehosts are often recommended by show hosts that I feel I can trust.
I setup my new VPS with bluehosts and was quickly underway moving websites from old to new VPS provider. My main website guy was more than happy for extra work so the plan was that I would move the HTML sites and he would move the WordPress sites. Before moving we would take a local copy of each site and manually check for malicious code.
Heads down and we were off, it took about 3 weeks in total but we had everything moved over to our new superfast VPS and all was good. Well that's what I thought for about 3 hours...
My bluehost VPS has been hacked...
There were a couple of warning signs, in between moving all the websites I had noticed that the server would occasionally stop and the sites wouldn't load. As you can imagine I was constantly running security scans so figured the heavy work load might be the cause..?
Unfortunately although the people at bluehosts support were very nice people they were very VERY useless when it came to actually rectifying issues. I reported the server crashes at least 6 times, every time I was told someone would investigate and come back to me but they never did.
On top of this I could also see that every now and again files were still having malicious code uploaded. Again I reported this many times to support but they were never able to fully help or rectify my issues.
I was able to combat the hack by running my own scans daily and fixing things but this obviously wasn't ideal.
Finally about 6 weeks after moving to bluehosts one night I noticed none of the sites were working. I put in a call to support and they said that as the server had been compromised they had shut down all my services.
After much late night arguing I managed to get someone to allow me root access so I could access the server and remove all the malicious content I could find.
Honest final thoughts on bluehosts hosting, I was really surprised at just how bad it was, keep in mind I had paid 3 years in advance for their premium VPS hosting around $3000.00 and I was treated like a total mug!
There is no online support ticket system (only phone support) which I really love, well would love if it was any good. The first level support you get to speak with have very basic knowledge and are really not able to help with anything remotely sophisticated.
At least 10 times when various different issues came up I was told that the problem would be raised with a level of support higher and I would be contacted via email about the progress. Not once was I contacted after ending the call.
Worst of all is the amount of time you have to wait on the phone while calling the US. If you are in UK and you use bluehosts expect your phone bill to double in a hurry!
The only credit I can give to bluehosts is that when I did leave and speak to complaints they did refund my money which was a big relief given that I had paid so much up front.
Was this hack a personal attack on me..?
I was racking my brains at this point, I was minutes away from giving up. I had spent thousands on the new VPS (paid for 3 years in advance) spent another thousand on paying my website guy over time to move the WordPress sites and I was no further forward than i was on Christmas eve.
I was seriously considering that this might have been an attack on me, im a pretty straight talker but I couldn't remember pi**ing anyone off that much!
Welcome to 20i...
So after over 2 months, what at this point felt like a lot of money down the drain and a pretty crappy Christmas I was fed up and felt like I had no options left. Honestly I was really struggling for ideas of what to do next!
While all this madness was going on I could remember seeing an ad on facebook for something like reliable, fast, SSD hosting. Im not the type to trust a facebook ad but as I was so out of ideas I thought I'd have a look and see if I could find the ad again.
Luckily I did and that ad was for a new company called 20i. After reading through the website I thought I would pick up the phone and see how quickly someone in sales would answer. The phone was answered quickly and I had a great chat with one of the UK based sales reps.
I was advised to try 20i's 1 month free reseller hosting and wow what a blessing that facebook ad has turned out to be!
I am extremely happy to report that since moving to 20i roughly 5 months ago we haven't had a single problem with any of the websites we host.
On the odd occasion we have used the support ticket system to ask general support questions we have always received back a very quick concise response.
What to do if your VPS gets hacked?
I'm sure that hundreds probably thousands of VPS's get hacked every day the same way ours did. If your ever in the same or similar situation I would highly recommend turning to 20i's reseller hosting.
The list of unlimited features you receive are awesome and the same goes for the fast and useful support be it using the ticket system or by phone.
What we learned about offering hosting services...
We must have had an extremely lucky run, 15 years working with the same hosting provider who never took backups and luckily enough we didn't have any problems.
Although we tried our best to keep our customers websites working as much as possible of course we suffered some down time.
Unfortunately we even lost a few customers after bluehosts let us down I had no choice but to give everyone the option to move on. Luckily we only lost 2 customers out of a possible 70 and we parted on good terms whith us helping as much as possible to make it easy for them to move away from our services.
The services we use now at 20i include server side malware scanning and backups. We also take our own manual backups once per month just to be on the safe side.
We now take it upon ourselves to make sure that all our customers WordPress websites are up to date both their plugins and WordPress core files, we make manual checks every week.
We provide WordFence premium firewall and scanning services to our customers running WordPress websites (we won't host WordPress sites without it).
As much as it really sucked while the hacking was going on I am very happy to be moving forward with a good reliable hosing company I know we can trust with not just our customers websites but also our own.
↣ If you ever find yourself in what feels like a totally hopeless situation like we did don't worry! There are good businesses out there that are willing to help.
↣ If your hosting provider lets you down and are unwilling to go the extra mile to help you out, move on and take your business elsewhere.
↣ Moving WordPress sites from host to host isn't that tough, most can be moved in just a couple hours.
↣ Talk to your potential new hosting provider before you spend $3000! I took the word of a few podcast hosts and rushed into spending a lot of money. Test the water by calling sales see how quickly they pick up the phone and have a list of questions ready to go.
↣ Most hosts don't provide backup and malware scanning as part of the deal. Find a company like 20i that does all this work without you having to pay more.
↣ If you're looking after multiple WordPress websites at the very least make sure you install and regularly scan using WordFence.
Have you ever been in a really tough situation at work when you felt that no matter what you did you just couldn't win? Leave us a message below and let us know about your own experiences.
GoRazy.com Open Internet Promise...
We believe in a totally free and open internet where you can find what you're looking for, find great advice and helpful articles without being tracked or monitored in any way by the websites you visit.
So if you like the sound of an open internet that doesn't track you or serve up junk advertising share this and let's make the internet a better place!