Giovani Altelino

Giovani Altelino

Software engineering blog and other (un)related themes.

27 Dec 2020

Digital Ocean App Plataform Benchmark With Locust

I have moved this static blog to the Digital Ocean App Plataform, using their free tier, I’ve used it since the beta release, and had no issue so far, but also, not many visit my website….

I think that I can’t do better than their official documentation, so take a look there, if you have a small blog like this, I think that the main drawback is that for now it only has integration with Github.

Benchmark

Ok, now let’s do some benchmark, just to check how many users we could serve this website. For that I will be using locust.

With the configuration below and around 100 users and 18 RPS I got no error, I really don’t expect much more users than this, but still, I want to push it a little more.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
import time
from locust import task, between
from locust.contrib.fasthttp import FastHttpUser

class QuickStartUser(FastHttpUser):
    wait_time = between(1, 10)

    @task
    def project_page(self):
        self.client.get("/project")
        
    @task
    def my_journey(self):
        self.client.get("/about/about")

    @task
    def hacker_news_graphql(self):
        self.client.get("/project/hacker-news-graphql")

    @task
    def whatsapp_monitor(self):
        self.client.get("/project/whatsapp-monitor")

    @task
    def automation(self):
        self.client.get("/post/automation")

    @task
    def covid_api(self):
        self.client.get("/project/covid-api")

    @task
    def this_blog(self):
        self.client.get("/post/this-blog")

So let’s tweak it a little bit, I spin up two virtual machines and simulate 200 users in each one, with more 500 in my local computer, for five minutes… still no failure… It’s nice that we could have 900 users changing pages every 10 seconds or so and our website would still be running fine, but now I want to see it crash.

Instead of running everything on it’s own, I will use the master and worker tags, so I can control everything in a single UI and divide the workload between multiple machines. I will deploy a master and five workers to simulate the workload.

What is interesting about Locust is that it’s quite simple to deploy and destroy, so the cost of using it can be very low, using a git repository and user data to automate the startup of the virtual machines, I will be using a Ubuntu.

First we need a git repository with our data, here I’m leaving it open, but you will probably need to have it as private, so you would need to change the user data below.

1
2
3
4
5
6
7
8
9
#!/bin/bash

sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt install python3-pip -y
pip3 install locust
git clone https://github.com/giovanialtelino/Locust-Test.git
cd Locust-Test
locust -f second.py --master --expect-workers 5

This is our master, after starting it, we need to get the IP from our cloud provider, to update the user data of the workers machine’s.

1
2
3
4
5
6
7
8
9
#!/bin/bash

sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt install python3-pip -y
pip3 install locust
git clone https://github.com/giovanialtelino/Locust-Test.git
cd Locust-Test
locust -f second.py --worker --master-host=<master-ip-here>

Most cloud providers will allow you to create more than one virtual machine at the same time, so you would need to create five of those, and wait a little bit. After around 10 minutes, just go to the http://master-ip:8089 and the Locust window should be open with the workers already connected.

Keep in mind that you may need to adjust the networking settings, Digital Ocean leave it open by default, but Azure don’t, and if you plan to run it more than once in Azure, I recommend you to create a Network Security Group, so you don’t need to adjust the inbound and outbound security rules all the time.

In Azure after you create the Master using the basic Virtual Machine utility, you need to use the Virtual Machine Scale Set to create the other 5 at the same time, and remember to set all of them in the same resource group, so you can delete it easily after your tests. Digital Ocean allow you to scale using the same default window, but you can’t delete all the droplets at the same time.

azure resource group delete

Again, don’t forget about it, you will have a lot of machines running.

Tests

With around 247 RPS, no error.
With around 746 RPS, no error.
With around 1240 RPS, 1% error, and only after a short bust!
Around 2000 RPS, still 1% error after a short bust.
Impressive!
But it get’s even better, I downloaded some failures files, and found out that the error were from the VMs, all of them were “OSError(24, ‘Too many open files’)”.
99% of the response were below 30 ms, really impressive stuff, I guess the CDN is really doing wonders here. You can read some of them here.

I tried to go beyond but my VM were all crashing, so I decided to stop the test here, the total cost was around 0.36 USD, and this with some setup and testing, after you make everything work fine the next time it will be event faster.

Conclusion

Digital Ocean App Plataform is a simple way to host your static website for free, than can easily handle at least 2000 RPS, in the free tier.