Can you trust VPN speed tests?

Using a VPN brings all kinds of benefits. You're protected on public Wi-Fi, more anonymous online, it's more difficult to track what you're doing, and you can bypass geographical blocks to access the sites you need.

There's also a performance price to pay, mostly due to the extra work a VPN adds to your internet access. All your traffic must be encrypted, maybe twice, then routed through an intermediate server (perhaps thousands of miles) away before finally heading off to your destination.

This is probably going to slow you down, and often by a huge amount. How much, exactly? Just about every VPN reviewer – including us – will try to help you find out by carrying out their own series of speed tests.

These reports can be very detailed, maybe with upload speeds, download speeds, latencies and more. But how much can you trust the individual figures, and what do they really mean to you?

• It’s also worth bearing in mind the good reasons why a VPN isn't enough

Test issues

Unpicking the results of a VPN speed test starts by understanding how it works, and what this is actually telling you.

The typical procedure involves running some kind of speed check – often a web-based service like speedtest.net – without the VPN, then repeating it with the VPN connected to one or more locations, and reporting any differences. This is simple enough, and can be accurate, but there are lots of points to consider.

Issue number one is the starting location. We connect to a VPN from the UK, for instance, which usually has several fast servers available. You may see very different results if you're connecting to the service from another country with its own level of servers and connectivity.

The next variables are the server locations being tested. If the reviewer isn't connecting to the countries you want to use, you're left to guess what this might mean. Terrible speeds from the UK to Netherlands could be interesting, but they don't necessarily mean you'll see the same when you access France and Germany.

There are more complications with the test setup. We test mainly on a Windows 10 laptop using OpenVPN UDP and the default Windows client settings, but if you're using different hardware, another client, and maybe tweaking the protocol or settings, that's likely to affect results.

Even if the test results were 100% accurate for the server locations tested, keep in mind that they can only measure speeds at a moment in time. These could be influenced by factors like the load on the reviewer's internet connection, how busy their ISP might be, maybe a temporary issue on the VPN server being accessed, or an overloaded test server.

Say we get lucky: there are no other factors involved and our figures really do represent VPN performance for the routes we've measured. That's great, but it could change tomorrow if the VPN adds a new server, upgrades its bandwidth, or perhaps drops its prices and sees thousands of new users arrive to overload the service.

This doesn't mean you should give up on speed tests entirely. If they consistently show one VPN is three times the speed of another, for instance, that's probably information worth knowing. Just treat the results as a broad indicator rather than the absolute truth.

Extended benchmarks

Some services and websites claim they can offer better and more accurate benchmarks of VPN speeds. There's some truth in that, too, although the end results may still not help you very much.

Suppose we set up five testing locations, for instance, based in the UK, US, India, Brazil and Australia. At regular intervals, each would automatically connect to several locations for every VPN. These could then run some custom speed test of their own (upload this, download that), record the results and calculate a final average.

This approach can seem very appealing. Having multiple testing locations gives you a more representative sample of service performance. Automating the tests reduces the chance of human error, and seems to ensure consistency in testing times and methods.

Unfortunately, this isn't quite as simple as it sounds. The more locations we have, the more variables we have to consider. We must ensure that all our five locations use identical servers, for instance, configured in precisely the same way and with the same level of internet connectivity. If they're not – if the European server is fast and the Asian server is slow, say – the final averages will be skewed.

Even if they're accurate, there's another problem. We suspect most people are interested in speeds from their home country to specific locations they need, maybe US to US, US to Canada, US to UK for the BBC, and the Netherlands for P2P. Our tests are averaging speeds starting from all continents to all other continents, and will be distorted by the many connections you'll never make. The end figure may be precise, but it's not telling most users what they need to know.

Choosing test locations is difficult, too, as VPNs can vary hugely in their network size. What happens if you've decided to test in Italy, for instance, but a VPN doesn't have a server there? You could pick somewhere close, in which case you're no longer making an exact comparison. Or you could simply choose the most common locations that everyone supports, like London, but then you're not fairly reflecting the benefits of choosing a VPN with a larger network.

There are other potential problems. For example, if we always used the same VPN servers, a provider could tweak them – add more RAM or system resources, reduce the number of other users on that location – to improve their performance and skew the results.

Even the basic test methodology has its issues. Running automated tests from our own servers is useful, but it doesn't exactly replicate what you're doing when you connect from home, using the official client, on a regular tablet. If you're planning to do something different, like choose a lesser protocol to improve speeds, or a more advanced one to ramp up security, the tests may again not match up with the speeds you'll see.

There is still some value to these extended tests, but they're no sort of benchmarking breakthrough, and on balance they're unlikely to give you the precise information you really need to know.

• Check out the best free VPN services of 2018

Do it yourself

Other people's speed tests can give you an idea of VPN performance, but there's only way to know for sure how a service will perform for you: try it for yourself.

If you've signed up for a trial, the best test is to run whatever applications you would use normally (games, video streaming and more) and see how they behave.

But if you're interested in more general tests to calculate an average download speed, you can do that, too. It's not straightforward and you'll be faced with many of the same issues as any other VPN reviewer, but these can be minimized with a little time and effort.

Use a couple of speed test sites, for instance. Speedtest.net is a good place to start, and ask your ISP to recommend another.

Don't run your tests over a wireless connection, if possible – this increases the chance of interference and cuts your maximum speeds. Use a wired connection instead.

Close all non-critical applications before you start testing, including everything (apart from antivirus) that might live in your Windows system tray. It's important that there's nothing else gobbling up system resources or using your internet connection. If you're unsure, use a network monitor to check. (In Windows 10, press Ctrl+Shift+Esc to launch Task Manager, click Performance and choose your network interface to look for unexpected network activity.)

Run both your speed tests several times at different times of day, with no VPN enabled, and record all the results. Keep going until you're happy that you understand how your connection and the speed test results might vary naturally.

Install, set up and enable the VPN. If you're testing multiple VPNs, ideally don't have them installed at the same time. Don't worry if that's inconvenient or impossible, as in theory you should be able to run them side-by-side without problems, but be careful. Don't set up both clients to launch when Windows starts, for instance; run one or the other, not both simultaneously.

With the VPN connected to your nearest server, try both the speed tests several times and calculate an average and range. If the test allows you to choose a test server, make sure you select the same one you used in the previous tests.

Switch to each of the most important VPN locations to you, repeat the tests and look for any changes.

Try other nearby servers, too, if possible. If your VPN has several New York servers, for instance, try two or three of them, and check out nearby locations such as New Jersey. Don't trust your VPN client's suggestion about what is the ‘fastest’ server, but try several for yourself. Geographically close servers can deliver very different speeds.

If you do get some unexpected results, don't just accept them – look for explanations. Run more tests, try another location, maybe run a test with the VPN disabled to confirm your connection is working as it should be.

At the end of this you'll have a very good idea of how a VPN will perform when connecting to and from your preferred locations. But even better, by seeing the variability involved in speed tests, you'll know just how much to trust them in the future, and that's well worth all the effort.

• Check out our best VPN services to get the best VPN deals around

from TechRadar - All the latest technology news http://www.techradar.com/news/can-you-trust-vpn-speed-tests