• 2 physical uplink switches in failover set-up;
• 3 physical uplinks in team per each vSwitch with beacon probing enabled;
• 2 of the uplinks (each going into different uplink switch) must be active and the third physical uplink should be set-up as a Standby adapter.

Beacon probing also works if you are using Vlan trunks. The only difference is that in case of any uplink failure beacon probes will be sent to all vSwitch VLANs increasing traffic a little.

Keep in mind:

You cannot advertise one MAC address to different uplink physical switches or Network flapping may occur. Do not use Beacon probing Network failover detection if you are using IP Hash Load balancing. Use IP Hash load balancing only with Etherchannel.

Digg frontpage listing usually sends 120.000  to 170.000 web visitors in two to three hours. It depends for how long you are listed on Digg’s frontpage and what is the topic.  In over 95% of cases this will probably be too much for simple dedicated server – if you are hosting Wordpress blog with some plugins activated (that require some extra load) and without tuning the server box.

You can activate some caching plugins (wp-cache, wp-super-cache, 1blogcacher), configure more efficient content distribution (for example, use lightweight web daemon to serve static files), use less includes and optimize plugin code for performance. This will probably handle the Digg frontpage load.

What if you land Yahoo! frontpage listing? This will be a real challenge for your systems and system administrator. Yahoo! is very popular and if your site gets listed in frontpage they can easily “send you” a few million of web visitors. Once again, depends for how long the listing will stay on the front page. Yahoo! traffic burst will definitely make your web server unreachable and I am saying this with 99.9999% confidence. You need multiple servers to handle  the load and do load balancing, and of course have upstream bandwidth available to serve the content. You need to employ perfect caching architecture, preferable keeping the page 100% static and load it in the server memory for super fast serving to web visitors. If you read it from the hard drive – your servers will experience very high IO and load, thus slowing down web responses.

Last time when one of our client’s got listed on Yahoo! frontpage the site (actually a blog) got around ~900.000 unique visitor landings, over 200.000 visitors browsed site further reading other articles and comments.  The traffic burst from two web front-end servers serving the content was close to 195Mbps (2 x 100Mbps links were fully maxed out) and I am talking about serving HTML/PHP content, not the images from these servers.

This is why you need to plan architecture to be scalable (horizontally) and easily affordable. One server box will not make it even if you get top-notch mega box for 20k USD, period.

For best performance we usually deploy Nginx and tie it together with Apache as a back-end for PHP processing, memcached, super cache and do a lot of rewriting rules and other optimizations.

The server is powered by one CPU Quad Core 5430 series processor, with 2GB RAM and SATA drives, runs on a SuperMicro server board. It’s a powerful box that can handle traffic well if correctly tuned and optimized.

We were running the latest Wordpress blog software with some custom rewrites done on the front end Nginx daemon (front end proxy). All static content was served by Nginx and all PHP queries were forwarded to Apache 2 version compiled from source as well as latest PHP 5.2.6 loaded as a module.

Nginx front end, Apache + PHP, Super Cache, some custom rewrite rules:
Requests done: 1000, concurrency 30 threads

Server Software:        nginx/0.7.11
Server Hostname:        www.neatorama.com
Server Port:            80

Document Path:          /
Document Length:        300421 bytes

Concurrency Level:      30
Time taken for tests:   0.691438 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      300687000 bytes
HTML transferred:       300421000 bytes
Requests per second:    1446.26 [#/sec] (mean)
Time per request:       20.743 [ms] (mean)
Time per request:       0.691 [ms] (mean, across all concurrent requests)
Transfer rate:          424678.70 [Kbytes/sec] received

Connection Times (ms)
min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       3
Processing:     4   20   6.1     19      41
Waiting:        0   16   6.6     14      28
Total:          4   20   6.2     19      41

Percentage of the requests served within a certain time (ms)
50%     19
66%     24
75%     26
80%     27
90%     27
95%     29
98%     30
99%     33
100%     41 (longest request)

Nginx + FastCGI, running spawn-fcgi from Lightspeed distribution with 30 child threads, all static content served by Nginx and PHP served by PHP 5.2.6 php-cgi version (custom compiled from the source of course).

Server Software: nginx/0.7.11
Server Hostname: www.*****.com
Server Port: 80

Document Path: /
Document Length: 170281 bytes

Concurrency Level: 30
Time taken for tests: 103.429538 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Non-2xx responses: 900
Total transferred: 170518100 bytes
HTML transferred: 170281000 bytes
Requests per second: 9.67 [#/sec] (mean)
Time per request: 3102.886 [ms] (mean)
Time per request: 103.430 [ms] (mean, across all concurrent requests)
Transfer rate: 1609.99 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 1
Processing: 1748 3076 492.0 3035 4844
Waiting: 768 1673 370.9 1659 2948
Total: 1748 3076 492.0 3035 4844

Percentage of the requests served within a certain time (ms)
50% 3035
66% 3248
75% 3405
80% 3496
90% 3732
95% 3943
98% 4192
99% 4398
100% 4844 (longest request)

As you can see Nginx + Apache, PHP, Super Cache was the clear winner – no questions asked.

Conclusion: we will stick with Nginx as a front-end proxy serving all static content while PHP processing forwarded to Apache web server running on the same server (using worker), running Super Cache and doing as much rewrites and file checking using Nginx for best performance. Apache is a big resource hog, however, our tests show it still outperforms running PHP scripts via FastCGI.

I would like to note that running PHP as a FastCGI required slightly lower memory usage, however the CPU load shoot up and I was not sure if we could handle Digg traffic that easily. Enjoy!

This works pretty well for stateless UDP-related services – for example, DNS UDP-based load balancing and distribution. This is what first-class providers such as Neustar Ultra Services (former UltraDNS), CriticalDNS and many root TLD operators use to offer first-class DNS services.

Many ISPs will block any route announcements that are smaller than /24.

We are currently preparing setting up our Linux based servers with 1Gbps connectivity/switching and tests with real results will follow shortly.