I was having a swim in the community pool when Jitendra called me on my cell (I stepped out and picked up the call when the ringing became incessant).
Jitendra: Hey, Indus, where are you?
Me: Having a swim
Jitendra: Dude, sezwho.com seems to be down — There are tonnes of alerts, have you seen them?
Me: Well, I was at Midas, getting my brakes done and then jumped in the pool as soon as I reached home.
The heat of the issue dried up the swim pants and I was at home in no time. To our excitement, there were a few hosts continuously pounding our blog; making the rest of our services unavailable. On further investigation we found that we were under a SYN flood. Unfortunately, our blog is hosted on the same server where other webservices live. So, pretty much the core platform was unreachable for 3 hours or so. However, the image servers, databases, etc. were OK as they reside elsewhere.
We did a few things with our blog, watched the logs, but it continued as our current firewall was not able to block the SYN flood. Hence, we tried something simple by banning the IPs one at a time. Finally, we got the issue under control (for the time being). Here’s the graph from our server monitor.
Here’s our ToDo:
1. Upgrade the firewall
2. Move the blog elsewhere on a different host (and probably a different sub domain as well)
3. Monitor network -DPARANOIA
| 3.9 (2 people) |
Post a comment | Trackback URI | Comments RSS feed
No trackbacks or pings yet
Comment template by SezWho
Filter Comments
Filter by: is