Lessons learned from a hacker.

My wife and I were just settling in for our week-long 10th anniversary trip in the Ozark mountains. We had a great week planned, with lots of hiking and site seeing, even a trip to an old car museum which I love to go to when we visit there. I woke up the morning after we arrived ready to get started, only to see that there was a voice mail on my phone from 6 a.m. It was Amin Motin, my helpdesk manager, informing me that a number of my sites had been hacked into...
As you may be aware of, three of my servers were hacked into last week, resulting in a number of my sites displaying the hacker's "look at what I can do" message as the home page. Fortunately no data was lost from any of the databases, and the sites were restored back to their proper working order within a matter of hours after I reported the issue to my server host.
It would appear that my computer got a keylogger-style virus installed on it, despite the fact that I only use a Mozilla browser (which I didn't think was targeted by such things, but apparently due to its popularity growth this is no longer the case). I often scan the "black hat" and "software cracks" forums for references to my own software so I can close any loopholes those guys find to try and rip off my software, and I can only imagine that the virus came from one of those sites. Shame on me for getting busy and not updating my virus protection software.
Well, long story short, I had these 3 server's open in a shell (SSH) window, and next thing you know the hacker was logging in and doing his thing.
The virus has since been removed, the servers restored to their normal working condition and all passwords changed.
This is not the kind of thing I wanted to deal with while I was away on my 10th anniversary with my wife, and it's certainly not the kind of thing she wanted me to have to deal with on such a special occasion. However, the matter was resolved within a few hours of my reporting it to my web host, which allowed my wife and I to enjoy the rest of our trip in peace. It could have been a whole lot worse, to say the least, had I not had such fantastic support from Amin, my customers and my web host.
I wanted to share with you the lessons I have learned from this incident in case they will help you.
1. Having strong passwords is not enough.
It doesn't matter how "unguessable" your password is if a keylogger gets installed on your computer and the hacker gets the password sent to him by the virus. It's also not enough to rely on your virus protection software, because a brand new virus may not be detected.
That's why I've had my host block all shell (SSH) and FTP access from all IP addresses except my own. So unless the hacker manages to break into the data center where my servers are, or breaks into my house and sits at my desk, even if he has the passwords it won't do him any good.
Since I never do any SSH work away from home, this works just fine. But even if I did need to access the server via SSH from somewhere else, I can always log a ticket with my web host to get the IP address temporarily added to the "allow" list.
2. Having loyal customers is a beautiful thing.
Within an hour of the hack, there were a dozen emails sitting in my inbox from customers and colleagues informing me of the issue and offering to do whatever they could to help. Within two hours there were two dozen emails.
I can't even being to describe how good it made me feel to know that my customers and associates all have my back, and were willing to go the extra mile to help me resolve the problem if they could do anything to help. That really lets me know that I'm running this business the way I should.
Thanks to everyone who alerted me and offered to help. It is VERY much appreciated.
3. Having a responsive host is vital!
It wouldn't matter much if I knew about the hack while out of town if my web host was slow in responding. That certainly was not the case! Within a few hours of my notifying my host, they had the problem cleaned up and everything set right.
I was informed later that two of their top-tier systems administrators were set on the task. It wasn't long before they knew exactly how the hack occurred, what to do to close the hole, and how to tighten security to prevent any such incidents in the future.
I always knew that my web host's service was top notch, but you REALLY know you're in good hands when something like this happens and they're on the ball. I am SO glad I switched hosts a couple of years ago. My previous host was incredibly slow to respond no matter how urgent the matter.
In case you're wondering, my server host is:
I cannot recommend their services highly enough. Their response time even on trivial issues is incredible, and when it really matters, they're on top of things in a flash.
Disaster Averted
The situation certainly could have been a lot worse than it was. I cringe to think how things would have gone had I still been with my old web host (which shall remain nameless -- but it was a BIG host, which you think would have great support, but didn't). I could have been stressing it out all week long waiting for things to get corrected IF I had even known about what happened. I might have been obliviously hiking through the mountains while a dozen of my sites were down for a week. Just thinking about that grows gray hairs on my head!
But things didn't work that way. Everything was set right the day I reported it to my fantastic host, thanks to being informed by Amin and my customers and associates. Because all of the links in the chain were strong, disaster was averted, and the security of my servers are now far stronger than they were before.
I have certainly learned a lot from this situation, but the biggest lesson can be summed up in these smile-evoking words: I'm in good hands.
Please post your thoughts and questions in a comment below.


