This is an overview of how we test modems, routers, and Wi-Fi extenders at Networks Hardware.
We here at Networks Hardware always test a device before recommending them. And here, you can learn exactly how we do this testing.
This process is created by me, Andreas Grant, Network Engineer and owner of Networks Hardware.
Let’s get started.
How We Test Performance
When testing performance, we always first make sure the firmware of the device’s test is up to date. We also make sure to remove as much interference as possible.
Then we start with the different steps in the testing. Which you can learn more about below.
Wi-Fi Throughput
Wi-Fi is one of those things that are hard to measure exactly. This is because so many aspects play a role in the measured speed.
Things We Consider Before Testing
First, we need to consider the Wi-Fi standard of the router itself and then match that with the client device we measure.
For example, testing a Wi-Fi 6E router needs a client device compatible with Wi-Fi 6E.
And it doesn’t end on whatever the device is Wi-Fi 6 or Wi-Fi 5. We also need to consider if the router and the client device are compatible with MU-MIMO 2×2 or 4×4.
The latter scoring faster speeds than the former.
Secondly, we need to consider the band we are testing. Something pretty easy on many devices but not all. It all depends if you can separate them or not.
If we cannot separate them (like in the Orbi series), we must use third-party apps to decide which band we use. In this case, we have used Wi-Fi Switcher.
Thirdly, we must consider the channel bandwidth. Because wider channels will be faster than smaller channels.
To keep things simple for the users, we have opted to use the 20 MHz on the 2.4 GHz band and the 40/80 MHz on the 5GHz band.
In our opinion, the most important thing isn’t the exact channel bandwidth used but rather that we stay consistent through all our tests. This is so we can fairly compare the devices with each other.
Wi-Fi Throughput Testing Procedure
For the testing itself, we use the Ookla speed test. The reason is that this has the closest server to our testing location.
Then we test the speed at 5 feet, 25 feet, 50 feet, and 75 feet from the router.
At every distance, we do at least three speed tests and then take the average result of the three.
We do this both on the 2.4 GHz and the 5 GHz band (and the 6GHz band if the router has one).
After the tests, we add all the data collected to our testing database. From there, we can compare the different results with other similar devices.
Cable Modems
Testing modems is hard.
This is because modems are mostly just either working or not working.
Nevertheless, we here at Networks Hardware do our best to test them as well as possible.
Our modem testing facility (sounds fancy, but it’s just a family member’s house 🙂 ) is in Boston. And until recently, we only had access to an 800 Mbps internet plan.
This has led us to test some of the modems with such a speed.
Nowadays, however, we have a 1200 Mbps plan with Comcast Xfinity.
Cable Modems Testing Procedure
For testing a modem, we start with as high an internet plan as the modem claims to support.
Then we make at least three speed tests and measure what speeds the modem delivers. We then switch to another modem and see what it provides.
Most of the time, they will provide the same speed. But it has happened that some models are slower than others.
We then log in to Xfinity’s website and switches the internet plan to a lower speed. Then we do the same test again.
We repeated this until we reached the lowest internet plan available.
The results are lastly added to our database to compare them to other similar modems.
Wi-Fi Range
Measuring range is another of those things that are very hard. Therefore, every range listed on the website is estimated based on what we got.
Different homes will get different results depending on the thickness and material of the walls, amount of interference, and home layout.
Nevertheless, when comparing networking devices, the most important aspect is consistency.
This means that we perform the tests the same way on every device. So that even if our estimation isn’t perfect, we can still tell you which is the best.
Wi-Fi Range Testing Procedure
For the testing, we use an app called Netspot. It’s a Wi-Fi analyzer that measures the signal strength of a Wi-Fi signal.
But to use Netspot, you also need a layout image of the place you are measuring the Wi-Fi in. So for this, we used SmartDraw.
Then we place the router at our usual spot (two floors down) and do around 10-15 tests in different locations on the floor. We always close every door to simulate the reality as much as possible.
Once finished, we look at the signal strength in the different spots and estimate the device’s coverage.
Mesh Wi-Fi
Mesh Wi-Fi follows the testing procedure for normal Wi-Fi routers pretty much spots on. With some extra caveats.
Mostly surrounding the satellites and how much they affect the speed and coverage.
Mesh Wi-Fi Testing Procedure
With Mesh systems, we always start with testing the main router alone.
This allows us to estimate how much every satellite really adds and how much slower they are compared to the main router (they are always a bit slower).
We then add one satellite at a time (sometimes there only is one). With every satellite added, we run further tests and see how much the signal gets increased (using Netspot).
Then we test the throughput at the same location as before and see how much faster speeds we get.
This way of testing Mesh Wi-Fi has been influenced by CNET, which explains how they do it here.
Latency (Ping)
Latency is tested when we run the speed test for the Wi-Fi throughput on the Ooakla speed test.
Latency will mostly depend on our ISP, but since we do all our router tests in the same place, we just compare the results to see which is best.
Unfortunately, we can’t compare routers and modem/router combos because they are tested at different locations.
Wi-Fi Bandwidth
Influence by PCMag, we also recently started to test the router’s bandwidth.
You might now as yourself, what is the difference between bandwidth and throughput?
We got the answer in this article. But if you want a short answer:
Bandwidth measures how much data can be sent through your network at any given time. While throughput measures how much data is being sent through your network at a particular moment.
So, to test this, we use a program called iPerf.
Every test runs for 60 seconds and gives us an estimation of the bandwidth the router can handle.
How We Test Features
Here is a short overview of how we test some of the different features that many routers have.
Quality of Service
Quality of Service, or in short QoS, allows you to prioritize bandwidth in your home networks.
When testing a router, we always check how extensive this feature is. For some context, some router only allows you to limit the data of specific devices.
In contrast, others allow you to show what type of applications (like gaming) should be prioritized (this is much better).
Then there is a wide range of levels of QoS between the two.
Parental Controls
Parental Controls are a feature that allows parents to control their kid’s activities online.
This feature also comes in many different ways. The best ones allow for blocking topics online, setting time constraints for specific devices, and having inbuilt modes.
For example, Study Mode, that only allows your kid to access informational resources.
Guest Networks
Guest networks either are or are not on a router. So there is not much to test with them.
However, we still turn them on and connect to them to make sure everything works properly.
Firewall/Security
Right now, we don’t have a good way of testing a device’s security more than looking at what types of security measures it has.
Things like WPA2 or WPA3 are things we look at.
We are working on it with ideas about using a fake virus and seeing if the device detects it or not. But this is still in progress.
How We Test Setups Modems & Routers
Most networks device has two ways of setting them up. One uses a computer, and the other uses the brand’s app. We always set up using both.
This ensures that we spot any difficulties and can report them to you before you buy that said device.
How We Test Setup Extenders
When we test Wi-Fi extenders, we also always test the setup with a router from a different brand than the extender.
This is to make sure that it is compatible with other router brands. Usually, this has been done with an ASUS router.Â
Database With Our Test Results
Here we plan to add our database with our testing results. But as it’s not complete yet, you will have to wait a bit longer.
What To Do Next?
Now that you know how we performed our test, you might want to know which devices performed best.
For that, check out our articles on:
