Unresponsive site, downtime, etc

Tharic-Nar · 01-08-2013, 08:37 AM

For those that do not frequent twitter, during the latter half of yesterday and through to early today (and still ongoing), the site was under the effect of a non-malicious DDoS attack; put another way, we were Reddited and had thousands of visitors trying to read up on the Kingston 1TB flash drive. The site was not built to handle this kind of spike traffic and as such, became very unresponsive for a lot of people. For this, we sincerely apologies.

Fortunately, traffic is beginning to die down again and the site becoming more responsive. We are currently going through a number of solutions to implement to prevent this from happening again (Load balancing, CDN, tweaks to Apache, etc). There may be some downtime later as we enable any changes required, but I do not think we need to at present.

(Greg, you're awesome!)

Brett Thomas · 01-08-2013, 08:43 AM

We'll be looking into fixing this in the very near...now, but thank you to all of the new visitors who have come to join us!

DarkStarr · 01-08-2013, 02:34 PM

Your P4 server just couldn't keep up

Also, lol

Tharic-Nar · 01-08-2013, 03:12 PM

Nah, we moved away from the P4 a few months ago,

. We're on virtually anything these days, 8 of them actually, but with less memory than Apache thinks necessary.

Seriously though, what was thrown at us would have taken most single servers down. We beefed things up and made some tweaks, currently running 8 cores and 12GB of ram. At peak, we had something like 1500 concurrent users active, all on the one page, and that was only who managed to connect successfully.

Rob Williams · 01-08-2013, 04:08 PM

Quote:

Originally Posted by Tharic-Nar

Nah, we moved away from the P4 a few months ago,

. We're on virtually anything these days, 8 of them actually, but with less memory than Apache thinks necessary.

Exactly. We got multiple week's worth of traffic in a single day. We've gotten sweet traffic surges before, but this was unparalleled.

Brett Thomas · 01-08-2013, 04:21 PM

That peak lasted for several hours, by the way. :-)

Anyhow, we've done some tweaks. Should be smooth sailing now. I dunno if we'd survive quite THAT superstorm again but we should be able to handle anything close to that without even breaking a sweat.

01-08-2013, 08:37 AM	#1
Tharic-Nar Techgage Staff Join Date: Nov 2009 Location: UK Posts: 1,152	Unresponsive site, downtime, etc For those that do not frequent twitter, during the latter half of yesterday and through to early today (and still ongoing), the site was under the effect of a non-malicious DDoS attack; put another way, we were Reddited and had thousands of visitors trying to read up on the Kingston 1TB flash drive. The site was not built to handle this kind of spike traffic and as such, became very unresponsive for a lot of people. For this, we sincerely apologies. Fortunately, traffic is beginning to die down again and the site becoming more responsive. We are currently going through a number of solutions to implement to prevent this from happening again (Load balancing, CDN, tweaks to Apache, etc). There may be some downtime later as we enable any changes required, but I do not think we need to at present. (Greg, you're awesome!) __________________ PSU: Corsair 1000HX - Case: Thermaltake Xaser VI Full Tower - CPU: Intel Core i7 2600 @3.8GHz - Cooler: Thermaltake FRIO - Motherboard: ASUS P67 Sabertooth - Memory: 16GB Corsair Vengeance LP Arctic 1600 - GPU: 2x AMD Radeon HD 5870 - HDD: WD Caviar Black 1TB 6Gb/s - Audio: ASUS Xonar DX + Corsair HS1 USB Headset K/B: Corsair K90 - Mouse: Corsair M60 - Monitor: DELL U2410 - Speakers: Corsair SP2500 - Headset: Corsair HS1A - Mic: Blue Yeti USB Microphone - I need Fiber to fix my Irregular Bandwidth Syndrome - Last edited by Tharic-Nar; 01-08-2013 at 08:43 AM.

01-08-2013, 02:34 PM	#3
DarkStarr Tech Monkey Join Date: Apr 2010 Posts: 623	Your P4 server just couldn't keep up Also, lol __________________ Intel Core i7 2700k (4.8 @ 100x48) Watercooled - 16GB Crucial Ballistix @ 1600 Mhz 9.9.9.24 2T 8GB Corsair Vengeance @ 1600 Mhz 9.9.9.24 2T - Asus Sabertooth P67 - Asus Radeon 7970 Ref. (Non Ghz) Heatkiller 79xx Ni-Bl (Soon) @ 1050/1500 - 64Gb Crucial M4 SSD - 3x Hitachi 1Tb - Corsair TX950W Azza Genesis 9000B - 2x Samsung SyncMaster S27A550H - Vizio 32" LCD

01-08-2013, 03:12 PM	#4
Tharic-Nar Techgage Staff Join Date: Nov 2009 Location: UK Posts: 1,152	Nah, we moved away from the P4 a few months ago, . We're on virtually anything these days, 8 of them actually, but with less memory than Apache thinks necessary. Seriously though, what was thrown at us would have taken most single servers down. We beefed things up and made some tweaks, currently running 8 cores and 12GB of ram. At peak, we had something like 1500 concurrent users active, all on the one page, and that was only who managed to connect successfully. __________________ PSU: Corsair 1000HX - Case: Thermaltake Xaser VI Full Tower - CPU: Intel Core i7 2600 @3.8GHz - Cooler: Thermaltake FRIO - Motherboard: ASUS P67 Sabertooth - Memory: 16GB Corsair Vengeance LP Arctic 1600 - GPU: 2x AMD Radeon HD 5870 - HDD: WD Caviar Black 1TB 6Gb/s - Audio: ASUS Xonar DX + Corsair HS1 USB Headset K/B: Corsair K90 - Mouse: Corsair M60 - Monitor: DELL U2410 - Speakers: Corsair SP2500 - Headset: Corsair HS1A - Mic: Blue Yeti USB Microphone - I need Fiber to fix my Irregular Bandwidth Syndrome -

01-08-2013, 08:43 AM	#2
Brett Thomas Senior Editor Join Date: Apr 2009 Posts: 164	We'll be looking into fixing this in the very near...now, but thank you to all of the new visitors who have come to join us!

01-08-2013, 04:21 PM	#6
Brett Thomas Senior Editor Join Date: Apr 2009 Posts: 164	That peak lasted for several hours, by the way. :-) Anyhow, we've done some tweaks. Should be smooth sailing now. I dunno if we'd survive quite THAT superstorm again but we should be able to handle anything close to that without even breaking a sweat.