As we were testing various VPN services, we noticed that no matter how many features a VPN provides, when it comes to usability, speeds is a real setback. To structure our VPN review efforts at Dealarious, we have developed an in-depth and detailed VPN speed test guide which we now follow for our reviews.
This comprehensive VPN speed test guide is easy to follow, and you can use it to test your VPN service too. Our guide has two sections. The first section explains the speed test tool of our choice and the second section explains exactly what we do with the tool.
Finding a Reliable Online VPN Speed Test
When we test for Internet speeds, we always measure the best speed. A consistent speed indicates a reliable VPN server, and this is true even for an ISP. Clearly, consistency in speeds is a key performance indicator for a reliable VPN service. But what tool should we use to test the speed of these VPN servers?
The most popular tool to test Internet speeds is undoubtedly SpeedTest.net, but is it the most accurate Internet speed test? To find out, we compared many back-to-back speed test results between:
- SpeedTest.net, the popular speed test by Ookla
- Fast.com, which was developed by Netflix
- SpeedOf.Me, which is an HTML5-based speed test
Speedtest.net vs. Speedof.Me vs. Fast.com
As we can see above, none of these speed tests agree with others, and they are not consistent themselves! The difference in these Internet speed-test results became clearer when we read the test methodology of Fast.com and SpeedOf.Me.
How Fast.com calculates download speed?
To calculate the download speed, Fast.com performs a series of downloads from Netflix servers.
How SpeedOf.Me calculates download speed?
SpeedOf.Me tests your internet connection by directly downloading sample files from your browser. SpeedOf.Me hosts its sample files on CDN.
Evidently, Netflix has faster and more reliable servers than any CDN that SpeedOf.Me is using.
We have decided to use Fast.com for all our speed tests, and we strongly recommend you use Fast.com. It is the most reliable service for VPN speed tests and general Internet speed tests too.
Calculating Overall VPN Network Speed
Now that we know Fast.com is the best Internet speed test method, it is time to devise a VPN speed test. Fast.com serves our needs accurately, and hence, we will design a speed test plan with Fast.com. The reason we have to come up with a plan is that VPN speeds are not always fast and consistent and the results that we report after these tests have to reflect an accurate picture.
VPN speeds vary with the time of the day and also across server locations. The above test graph shows how our VPN download speeds varied across the world at the same time. Besides, the graph looked significantly different after six hours with slower servers becoming fast and faster servers reporting slow speeds. Apparently, VPN speeds cannot be calculated accurately at one point in a day.
To accommodate for this behavior, we have spread our tests evenly through the 24 hours in a day. Also, we are following a sequence of steps for our tests to be more meaningful and realistic.
Steps to Test VPN Speed
1. Calculate a Median of Five Speed Test Results
We will start by taking five consecutive speed tests for each server and determine a median value.
The average is a widely used aggregation method. However, it has a problem. Any outliers (a significantly higher/lower number) in our data can skew the average and give a wrong picture. Instead, we will use the median for our speed indicators, as the median is less affected by outliers.
Location | Test 1 | Test 2 | Test 3 | Test 4 | Test 5 | Median |
|---|---|---|---|---|---|---|
Singapore | 8 | 5.1 | 6.6 | 8.3 | 5.4 | 6.6 |
Italy | 18 | 13 | 11 | 13 | 0 | 13 |
India | 9.5 | 9.9 | 9.6 | 7.8 | 12 | 9.6 |
Germany | 8 | 9.5 | 6.1 | 4.8 | 8.9 | 8 |
France | 19 | 16 | 12 | 14 | 13 | 14 |
Base | 20 | 18 | 21 | 21 | 23 | 21 |
2. Compute a Median of Medians
Once we have the median speeds for all servers, we will take another median of all these median speeds.
- This median of medians will give us an estimated speed for the entire VPN network at that time.
- Besides, it will help us filter out slow VPN servers and identify good servers. We will only consider servers that rank higher than the median of medians.
Location | Test 1 | Test 2 | Test 3 | Test 4 | Test 5 | Median | Median of Medians |
|---|---|---|---|---|---|---|---|
Base | 20 | 18 | 21 | 21 | 23 | 21 | 0.9 |
France | 19 | 16 | 12 | 14 | 13 | 14 | |
Italy | 18 | 13 | 11 | 13 | 0 | 13 | |
India | 9.5 | 9.9 | 9.6 | 7.8 | 12 | 9.6 | |
Germany | 8 | 9.5 | 6.1 | 4.8 | 8.9 | 8 | |
Singapore | 8 | 5.1 | 6.6 | 8.3 | 5.4 | 6.6 | |
Denmark | 1.7 | 4.9 | 5.7 | 4 | 2.4 | 4 | |
Sweden | 4.1 | 2 | 3.3 | 3.2 | 3.8 | 3.3 | |
Switzerland | 2.7 | 2.6 | 2.1 | 2.6 | 3.4 | 2.6 | |
Ireland | 1.9 | 1.9 | 0.82 | 0.92 | 2.1 | 1.9 | |
UK | 0.9 | 3.7 | 4.3 | 0 | 0 | 0.9 | |
Mexico | 1.4 | 0.9 | 0.9 | 2.4 | 0.6 | 0.9 | |
Norway | 0.5 | 1.6 | 0.8 | 1.1 | 0.3 | 0.8 |
3. Calculate Percentage of Base ISP Speed
As the next step, we will calculate the percentage of base speed (when not using a VPN) for all the good servers that we have identified. This will give us an estimate of drop in speeds for various servers.
Location | Test 1 | Test 2 | Test 3 | Test 4 | Test 5 | Median | Base Speed % |
|---|---|---|---|---|---|---|---|
Germany | 8 | 9.5 | 6.1 | 4.8 | 8.9 | 8 | 38 |
Singapore | 8 | 5.1 | 6.6 | 8.3 | 5.4 | 6.6 | 31 |
India | 9.5 | 9.9 | 9.6 | 7.8 | 12 | 9.6 | 46 |
Italy | 18 | 13 | 11 | 13 | 0 | 13 | 62 |
France | 19 | 16 | 12 | 14 | 13 | 14 | 67 |
Base | 20 | 18 | 21 | 21 | 23 | 21 |
4. Normalize VPN Speed Test Results
We will then repeat this test four times in a day to get some reliable data that is normalized to represent a more accurate picture. The servers that make it above the median regularly will make it to our graphs.
5. Create Graphs with Normalized Speed Values
Once we have a list of servers that are fast throughout the day, we will create charts to show how their speeds change through the day. The charts will be based on median speeds and will be a good indication of what you might experience if you connect to these servers at any time in the day.
VPN Speed Test
Identifying Fast VPN Servers
Each VPN service will finally get a speed- rating in our reviews based on:
- How many locations offered by the VPN service are fast
- What the VPN service's maximum median speed is across these locations
This series of steps and tests will finally help us identify fast servers for any VPN service and give an overall speed rating for the VPN's network.
These steps constitute the most stable VPN speed test methodology we could devise. We will update this guide if we find a better way to test VPN speeds. We would also love to hear your suggestions on improving these tests further. Leave a comments if you have ideas to enhance our tests in any way.
I take a deep interest in finding out why things work the way they work. I also write about VPN services, anonymity tools, and privacy tools here at Dealarious.com.