Instability and low speeds on my 1 Gbit/s connection

​@rvk01 I undersand your suggestion of the investigation path, and that has been mine for a long time, however moving away from the experia box was what brought *some* stability to the connection.

The experience was roughly this:

• Restarting the Experia Box whenever I had an issue
• Restart the ONT when that didn’t work
  • This is way more rare to happen, but I would say more than I would expect to
• Get a 5G router and KPN 5G (which is metered on a daily limit but should be fine if it’s just a backup)
  • It didn’t suffice, I’d run over constantly because of the many and long unstable moments the link had
• Realize the 5G was sustaining 40-60% packet loss
• Moving away from the Experia Box to make a direct OpnSense conection with the ONT VLAN
  • This essetially eliminated the problem with restarting the router, it’s a 4-core/16GB RAM machine instead of the experia hardware
• Hiring Odido’s Klik’n’Klaar as a backup link (realizing the packet loss was gone and stability of speed was higher)
• Measuring with Grafana/InfluxDB the performance of both links from a dedicated machine
• Realizing I have roughly daily outages around 4-6am 
• Questioning my life choices…

The amount of infrastructure fiddling was just growing because I could neither get a proper link working with relatively constant speeds around the hired plan nor be able to have support follow up on the instabilities (given the internet being up at the time of contact is enough to dismiss an outage)

As you can see the measurements and procedures are not coming from lack of knowledge (at least until now, I can always be missing a variable) but I’m really unhappy with both the hardware provided (the experia box requiring me to reboot every now and again) and the link quality (even bypassing most of the overhead from the router)

I would say my backup link has been probably more reliable than the main one

You can observe the constant dips on WAN (KPN) happen just as ofter or more than a 5G link.

Also switching to Experia (and having to restart the box over and over) will be a degradation of the service now (since I can’t have 2 ppoe flows over the same ONT) until we discover what might be going on with the link.

I understand if you have problems with the Experia hardware, that you moved away from it. But fact remains there are only few routers/hardware which can handle speed with PPPoE on the WAN side (which is needed for KPN).

So… even if the Experia box gave you instability… what was actually the maximum (or usual) speed you could get out of your 1Gbps line?

Even if KPN wanted to diagnose any line problems… the Experia box would need to be connected (because they can only diagnose with that hardware).

> I understand if you have problems with the Experia hardware, that you moved away from it. But fact remains there are only few routers/hardware which can handle speed with PPPoE on the WAN side (which is needed for KPN).

This is why I mentioned I have a 4-core / 16 GB / 4x 2.5gbps ethernet ports dedicated box for the router, which I assume is more than what the Experia box has…

> So… even if the Experia box gave you instability… what was actually the maximum (or usual) speed you could get out of your 1Gbps line?

To be honest while the internet was stable I didn’t bother to measure since it was ok, but I’ve been through a ton of instability since then

> Even if KPN wanted to diagnose any line problems… the Experia box would need to be connected (because they can only diagnose with that hardware).

If we managed to get a proper test in depth I’d be more than willing to turn it on for a limited period, however the update is usually “The link looks just fine” just to be met with another outage a couple of hours or days later

No, because the Experia box is specially geared towards the PPPoE link.

Even though you have 4 cores… OpnSense can only use 1 single core because PPPoE can’t be spread over multiple cores.

It’s just a test for a few minutes… just to see if you have the correct speed on the line. I expect you do and that it is the OpnSense that can’t handle 1Gbps on that single core.

And if that’s the case, you might want to return to the Experia box and diagnose again what makes it unstable. Or use another router which can handle the speed with PPPoE.

Switched for the test, but I assume 2GHz cores should be enough for a 1Gbps link... I'll post the results once they're in

No, not necessarily. If it’s a really modern CPU you might just reach above 900Mbps but that’s not guaranteed (and only when completely optimized which it’s probably not).

https://forum.opnsense.org/index.php?topic=26162.0

https://forum.opnsense.org/index.php?topic=27049.15

I’ll keep the box up for a couple of days and take the optimizations into account for the comparison later… hopefully both give both results, with the Experia box and ONT collaborating for a good experience :p

Now that you have connected the Experia box again... there are two tests you can do…

1. Connect a PC with a good network card directly to the Experia box and do a speedtest.net (via clean browser or via the Windows app).

What is the global speed you reach now. 

2. Connect the OpnSense with the WAN to a LAN port of the Experia box. Preferably with the OpnSense on it's own subnet mask (so not the same as the Experia box. Connect the same PC to the LAN of OpnSense and do the test again.

What is the global speed you reach now. 

Possibly, because OpnSense doesn't have to do PPPoE now, it could reach higher. It's not a permanent solution because you don't want double NAT (NAT after NAT) but it will show what's going on and possibltly why the OpnSense doesn't get the full stable speed.

Since the main issue is instability, I’ll keep the monitoring for a couple of days, reaching 900 on a single test means nothing if the internet is down daily from 4-6am (which is my main complaint)

It will confirm that your own hardware isn't strong enough to handle the speed via PPPoE. But ok. After that you would still need to deal with the possible instability on the Expedia box.

At least with the Experia box connected, KPN could do measurements and look at the issue (which isn't possible with other hardware).

Hi ​@hcassus and welcome to our KPN Community. I'm glad to see you’ve gotten some help already and curious about the results of the tests with the Experia Box after a few days. Do keep us posted! 

I have just switched back to the direct link so we can compare the performance, so far the results look promissing regarding stability

The results are in: The Experia box is indeed able to increase the speed average and reduce drops both in frequency and intensity, but we still see lots of them (which points to the link instability I mentioned)

Some highlights:

04/09/26 @ 5:36 - 211 Mbps

04/09/26 @ 7:12 - 134 Mbps

04/09/26 @ 11:55 - 646 Mbps

04/09/26 @ 16:54 - 627 Mbps

04/09/26 @ 19:36 - 655 Mbps

I just flagged the ones at ~40% or less of the hired speed, but there aare way more drops around the 800 Mbps marks (which I consider to be acceptable)

With a longer term metric collection, I’m expecting to find many more of those bigger gaps (as I already had in the past)

And that’s with the KPN Box and a PC directly connected to the router?

In that case you need to keep the box connected and a moderator should order a line-measurement.

We’ll wait on a moderator to do that.

I'm curious about the long term results as well. And about this: 

If that's the case and you keep it that way I can check the line for any instabilities. If there are steep drops happening at the same time every day again for example, could it also be something in your home network that boots up or starts downloading? Just to make sure it's not anything in your own network causing this. If that's something you’ve already checked, do let me know! 

Oh, and could you fill out your community profile with the name and date of birth of the subscriber and your zipcode + housenumber? Then I'll be able to run the checks. 

I’ve completed my profile with the data you asked, and managed to also get some more drops noted.

Since they don’t happen everyday, it’s also quite difficult to get support to check at the time they happen, so it would be nice if the history somehow helps.

13/09 07:30 - 191 Mbps

14/09 7:02 - 410 Mbps

14/09 7:42 - 152 Mbps

14/09 8:17 - 256 Mbps

14/09 8:38 - 237 Mbps 

I’m also trying to work on some improvements on the measurements to correlate signals of saturation and bandwith more precisely, but I’m assuming no process I have running would be able to saturate ~700mbps of the link

Thank you! I've checked your connection and here's a couple of my findings:

Firstly, there's scheduled maintenance to your connection on the 30th of September in the middle of the night (1-7 AM). Perhaps you’ve already received a message about this (or you will soon), but then now you know in advance. 

Secondly, in all the scans I can do and have done the connection appears to be stable. At least from our network up to your KPN router. If anything in between there is unstable or dropping frequently, the scan should have picked it up. 
However, this is only when the connection drops completely. The last drop was today, and before that I see disconnects on the 8th and 5th of September, which correlates with your messages here. 

All I can see from the connection drop today is that it was user requested, which generally means the hardware got restarted or reset. Did you switch back to your own router perhaps? 

The tricky thing is that I cannot measure drops in speed. This is partly because a lot of speed issues happen in the home network, and not necessarily in our connection. That makes it more difficult to figure out where the speed drops are coming from, but not impossible. To pinpoint where it originates, could you answer these questions? 

Just to check, when the Box 12 was connected it was directly to the fiber access point with no other appliances in between? Not the smart home plug that automatically restarted the NT point either? 
Could you tell me what kind of devices are connected to your home network? 

Has this ever been done exactly as described here? You can use the Ookla Speedtest in your webbrowser or use the Windows App | MacOS App. 
Please disconnect any other devices when you do measurements. I ask this because some (older) devices can cause interference and that can cause the internet speed to drop. 
Do you experience the drops on all devices at home? So any laptops, Smart TV's and phones experiencing drops in speeds? 

It's a lot of questions but this will hopefully help us pinpoint what is happening and how to solve it. 
And to add: because maintenance is scheduled on the 30th, should the mechanics then find someting in our network that isn't working right (and would be causing the issues you’re experiencing), they can fix it as well. 

> All I can see from the connection drop today is that it was user requested, which generally means the hardware got restarted or reset.

This is weird because I haven’t touched the router or switched back from mine.. at the same time I see no drop in my measurements, so it might have happened and recovered in between mesurements

> The tricky thing is that I cannot measure drops in speed. This is partly because a lot of speed issues happen in the home network, and not necessarily in our connection.

I’m slowly attempting to get a correlation between link saturation and speed drops to rule that out as a cause, but it’s taking a bit long for me to have time to set that up

> a single PC connected to port 1

The problem with it is having to unplug my whole infra to make one speedtest, this is shy I’m attempting to get that correlation metric… I might also try to have it done my switching the rest of the infra to the 5G backup link, but I’ll keep you posted

> Do you experience the drops on all devices at home? So any laptops, Smart TV's and phones experiencing drops in speeds? 

Yes, but I can understand those might be way more sensitive to the rest of the infra like wi-fi signals etc, this is why I am conducting the tests as close to the router as I can

> because maintenance is scheduled on the 30th, should the mechanics then find someting in our network that isn't working right (and would be causing the issues you’re experiencing), they can fix it as well. 

I might say that “the worst has passed” (which was some months ago with constant drops in actual connection, not just speed), but our constant debugging is helping eliminate a couple of confounding variables and helping tune the ones I can’t escape

That could be it. I saw no other odd connection drops in my measurements so it seems stable enough from our end thankfully. 

Do keep me posted on the measurements and results. I understand it's quite a hassle to disconnect everything and just keep one PC or laptop conected, but it does often provide valuable insight in whether other devices might be causing disturbances in the connection. 
If your current tests do not provide any correlation or eliminating more variables, I do suggest you give that a try. 

Hopefully we'll be able to figure out what's causing this and I'll await the results of your tests! 

​@Sanne van KPN 

I got some measurements and was able to get both a drop caused by load (the first drop around 16:00) and one not affected by it (the one around 22:00), so we can see the measurement of both dimensions are somewhat precise.

I’ll measure in the next few days the  total offload of the traffic (unfortunately I can’t keep this way for long) to see if we can measure anything different