Load balancing with Nginx

Posted: June 15, 2014 in Uncategorized
Tags: , ,

 

I am using a simple HTTP server written in Python which will runs on the port given by the commandline argument. The servers will act as upstream servers for this test. Three servers are started
on port 8080,8081 and 8081. Each server logs its port number when a request is received. Logs will be written to the log file located at var/log/loadtest.log. So by looking at the log file, we can identify how Nginx distribute incoming requests among the three upstream servers.

Below diagram shows how Nginx and upstream servers are destrubuted.

Load balancing with Nginx

Load balancing with Nginx

Below is the code for the simple HTTP server. This is a modification of [1].

#!/usr/bin/python

#backend.py
from BaseHTTPServer import BaseHTTPRequestHandler,HTTPServer
import sys
import logging

logging.basicConfig(filename='var/log/loadtest.log',level=logging.DEBUG,format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p')

#This class will handles any incoming request from the browser.
class myHandler(BaseHTTPRequestHandler):

	#Handler for the GET requests
	def do_GET(self):
		logging.debug("Request received for server on : %s " % PORT_NUMBER)
		self.send_response(200)
		self.send_header('Content-type','text/html')
		self.end_headers()
		# Send the html message
		self.wfile.write("Hello World: %s" % PORT_NUMBER)
		return

try:
	#Create a web server and define the handler to manage the
	#incoming request
	PORT_NUMBER = int(sys.argv[1])
	server = HTTPServer(('', PORT_NUMBER), myHandler)
	print 'Started httpserver on port %s '  %  sys.argv[1]
	#Wait forever for incoming htto requests
	server.serve_forever()

except KeyboardInterrupt:
	print '^C received, shutting down the web server'
	server.socket.close()

Lets start the servers on port 8080, 8081 and 8081.

nohup python backend.py 8080 &
nohup python backend.py 8081 &
nohup python backend.py 8082 &

Check if the servers are running on the secified ports.

netstat -tulpn | grep 808
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      454/python
tcp        0      0 0.0.0.0:8081            0.0.0.0:*               LISTEN      455/python
tcp        0      0 0.0.0.0:8082            0.0.0.0:*               LISTEN      457/python

*Configure Nginx as a load balancer for above upstream server.

Create a configuration file in /etc/nginx/udara.com.conf with the below content. Above started servers are configured as upstream servers.

upstream udara.com {
        server udara.com:8080 ;
        server udara.com:8081 ;
        server udara.com:8082 ;
}

server {
           listen 80;
           server_name udara.com;
           location / {
                        proxy_pass http://udara.com;
           }

* Pick a client to send request. You can use Jmeter or any other tool. However I wrote a very simple shell script which will send given number of request to the Nginx

#!/bin/bash
c=1
count=$1
echo $count
while [ $c -le $count ]
do
     curl http://udara.com/
     (( c++ ))
done
 Round robing load balancing
upstream udara.com {
        server udara.com:8080 ;
        server udara.com:8081 ;
        server udara.com:8082 ;
}

Let’s issue 9 request.

./requester.sh 9

Logs written on var/log/loadtest.log log file.

06/15/2014 11:54:11 AM Request received for server on : 8080
06/15/2014 11:54:11 AM Request received for server on : 8081
06/15/2014 11:54:11 AM Request received for server on : 8082
06/15/2014 11:54:11 AM Request received for server on : 8080
06/15/2014 11:54:11 AM Request received for server on : 8081
06/15/2014 11:54:11 AM Request received for server on : 8082
06/15/2014 11:54:11 AM Request received for server on : 8080
06/15/2014 11:54:11 AM Request received for server on : 8081
06/15/2014 11:54:11 AM Request received for server on : 8082

Request are distributed evenly among all three servers in round robin fashion.

Session stickiness

Requests from the same client will be forwarded to the same server always.The first three octets of the client IPv4 address, or the entire IPv6 address, are used as the hashing key to determine which server to forward the request. In case the selected server is unavailable, the request will be forwaded to another server.

upstream udara.com {
	ip_hash
        server udara.com:8080 ;
        server udara.com:8081 ;
        server udara.com:8082 ;
}

All the requests are forwaded to the server running on 8082.

06/15/2014 11:54:55 AM Request received for server on : 8082
06/15/2014 11:54:55 AM Request received for server on : 8082
06/15/2014 11:54:55 AM Request received for server on : 8082
06/15/2014 11:54:55 AM Request received for server on : 8082
06/15/2014 11:54:55 AM Request received for server on : 8082
Weighted load balancing

By default Nginx equaly destribute the requests among all upstream servers. This is OK when all the upstream has the same capacity to serve the requests . But there are scenarios where some upstream servers have more resources and some resources have low resources compared to others. So more requests should be forwarded to high capacity servers and low capacity servers should be forwaded less number of requests. Ningx has provided the ability to specify the weight for every server. Specify weight propotional to the capacity of the servers.

upstream udara.com {
 server udara.com:8080 weight=4; #server1
 server udara.com:8081 weight=3; #server2
 server udara.com:8082 weight=1; #server3
}

Above configurations says server1’s capacity is four times of server3 and server 2 has thrice the capacity of server3. So for every 8 requests, 4 should be forwaded to server1, 3 should be forwaded to server2 and one request for server3.
Below logs shows that requests are destributed according to the weight specified.

06/15/2014 12:01:36 PM Request received for server on : 8081
06/15/2014 12:01:36 PM Request received for server on : 8080
06/15/2014 12:01:36 PM Request received for server on : 8080
06/15/2014 12:01:36 PM Request received for server on : 8081
06/15/2014 12:01:36 PM Request received for server on : 8080
06/15/2014 12:01:36 PM Request received for server on : 8081
06/15/2014 12:01:36 PM Request received for server on : 8082
06/15/2014 12:01:36 PM Request received for server on : 8080
 Mark a server as unavailable

“down” is used to tell Nginx that a upstream is not available. This is usefull when we know that the server is down for some reason or there is maintainance going on that server. Nginx will not forward request to the servers marked as down.

upstream udara.com {
        server udara.com:8080 weight=4;
        server udara.com:8081 weight=3 down;
        server udara.com:8082 weight=1;
}

 

06/15/2014 12:10:54 PM Request received for server on : 8080
06/15/2014 12:10:54 PM Request received for server on : 8080
06/15/2014 12:10:54 PM Request received for server on : 8082
06/15/2014 12:10:54 PM Request received for server on : 8080

No request has forwarded to the server running on port 8081.

High avalability/ Backup

When a upstream server node is marked as backup, Nginx will forward requests to them only when primary servers are unavailable.

upstream udara.com {
        server udara.com:8080 ; #server1
        server udara.com:8081 ; #server2
        server udara.com:8082  backup; #server3
}

Request will be sent only to server1 and server2. No requests will be sent to server3 since it is the backup node.

06/15/2014 02:57:40 PM Request received for server on : 8080
06/15/2014 02:57:40 PM Request received for server on : 8081
06/15/2014 02:57:40 PM Request received for server on : 8080
06/15/2014 02:57:40 PM Request received for server on : 8081

Stop the servers running on 8080 and 8081 so only server on 8082 is running.
Request are sent to the backup node.

06/15/2014 02:46:04 PM Request received for server on : 8082
06/15/2014 02:46:04 PM Request received for server on : 8082
06/15/2014 02:46:04 PM Request received for server on : 8082
06/15/2014 02:46:04 PM Request received for server on : 8082
Multiple backup nodes.
upstream udara.com {
        server udara.com:8080 ; #server1
        server udara.com:8081  backup; #server2
        server udara.com:8082  backup; #server3
}

Requests are directed only to server1 as long as server1 is available.

06/15/2014 03:03:02 PM Request received for server on : 8080
06/15/2014 03:03:02 PM Request received for server on : 8080
06/15/2014 03:03:02 PM Request received for server on : 8080
06/15/2014 03:03:02 PM Request received for server on : 8080

When server1 is stopped, requests are forwaded to both server2 and server3.

06/15/2014 02:57:40 PM Request received for server on : 8081
06/15/2014 02:57:40 PM Request received for server on : 8082
06/15/2014 02:57:40 PM Request received for server on : 8081
06/15/2014 02:57:40 PM Request received for server on : 8082

[1] https://github.com/tanzilli/playground/blob/master/python/httpserver/example1.py

[1] http://nginx.org/en/docs/http/load_balancing.html

Advertisements
Comments
  1. susinda says:

    Good post .. Thanks for sharing your knowledge and experience.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s