Challenge

We're experiencing longer than expected response times under load and we want to decrease that so we implement horizontal scaling with NGINX as a load balancer. A load balancer distributes a ton of incoming requests/traffic (the load) across multiple resources to process that traffic.

Actions

Lets horizontally scale

  1. Create multiple instances of T.2 Micros that all host your API and are all connected to your same DB (in other words, you should be able to access your API endpoints from multiple IP addresses).

    1. You will also need to open up port 80 on your EC2 instance through AWS security groups
  2. Create one more T.2 Micro instance (this will be our NGINX proxy server/load balancer)

  3. SSH into your NGINX instance ssh -i path/to/pem/key <username>@<public ip>

  4. Install NGINX sudo apt update sudo apt install nginx

  5. Run NGINX sudo systemctl start nginx

  6. Navigate to http://<your nginx instance public ip address> and you'll see something similar to the following

    https://s3-us-west-2.amazonaws.com/secure.notion-static.com/37fdc172-5a31-4d4a-a1c5-cd8736832bd6/Untitled.png

  7. Now, we have to configure NGINX to connect to our API instances. To do this, lets open up the config file sudo vim /etc/nginx/sites-enabled/default and change it to something like this

    proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:10m max_size=10g 
                     inactive=60m use_temp_path=off;
    
    upstream sdcgroup {
      least_conn;
      keepalive 500;
      server <one instances public ip>:<your api port>;
      server <another instances public ip>:<your api port>;
      # queue 100 timeout=100; # you may have to delete this line for it to work
    }
    
    server {
      listen 80 backlog=4096;
      gzip on;
    
      location /<your api endpoint> {
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_pass <http://sdcgroup>;
    
        proxy_cache my_cache;
    		proxy_cache_valid any 10m;
        add_header X-Cache-Status $upstream_cache_status;
      }
    }
    
  8. (optional) If you want your NGINX server to work with loader.io you should add an additional location directive to your config file to return the verification. I put an example verification token, but you can put whatever yours is.

    proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:10m max_size=10g 
                     inactive=60m use_temp_path=off;
    
    upstream sdcgroup {
      least_conn;
      keepalive 500;
      server <one instances public ip>:<your api port>;
      server <another instances public ip>:<your api port>;
      # queue 100 timeout=100; # you may have to delete this line for it to work
    }
    
    server {
      listen 80 backlog=4096;
      gzip on;
    
      location /<your api endpoint> {
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_pass <http://sdcgroup>;
    
        proxy_cache my_cache;
    		proxy_cache_valid any 10m;
        add_header X-Cache-Status $upstream_cache_status;
      }
    
      location /loaderio-95196c056520a04c106a81f0501ef836 {
        return 200 'loaderio-95196c056520a04c106a81f0501ef836';
      }
    }
    
  9. Our last setup step is to set this config file to exist in another location, why I do not know, but NGINX wants us to so to do that run this command: sudo ln -s /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default

Result

You can now run sudo systemctl restart nginx and your nginx proxy server/load balancer will be accessible at its public IP address to route your traffic at the endpoints you were using.

Look at those blazin speeds🔥 . Really quick response times and also in my console, you can see how traffic is properly getting routed to my 2 running instances

Look at those blazin speeds🔥 . Really quick response times and also in my console, you can see how traffic is properly getting routed to my 2 running instances

Additional Resources