What If Your 10-Gigabit Ethernet Is Unreliable? I Bought A 5-Gigabit SFP+ Module!

| Comments

I bought a couple of MokerLink 2.5-gigabit switches back in September. One lives in my office. The other lives in my network cupboard on the other side of the house. I plugged my desktop PC into a 10-gigabit copper SFP+ module, and I connected the two switches using another pair of Xicom 10-gigabit SFP+ modules.

I was a little worried that the 70’ of Cat 5e running through the attic might not manage to connect at 10 gigabit, but it worked just fine. It continued to work fine for a few months. Then things started becoming less reliable.

My 2.5-gigabit MokerLink switch and my old 1-gigabit switch in my office

I hate intermittent problems. I still haven’t correctly identified my problem. In fact, I have been back to having a flawless 10-gigabit Ethernet connection across my house for more than a month. That makes my problem even more difficult to troubleshoot!

I am going to walk through my troubleshooting steps, and tell you what I currently suspect is going on. Maybe you are having similar problems, and maybe something I have done or something that popped into my mind may be of use to you!

What do I think is going on? Here’s the tl;dr

I am extremely confident that my problem is related to the weather. It doesn’t get all that cold here in Plano, TX, but my problems started in the cooler months, got worse as the weather got colder, and the problem went away completely once the temperature outside stopped dropping below maybe 50F at night.

I don’t think it is just the cold. Things are more likely to be problematic when it is humid or raining.

Most of the length of the Cat 5e cable running from my office to my network cupboard is above the insulation in the attic.

I initially thought my SFP+ copper modules were overheating

Copper SFP+ modules do tend to get quite warm. They’re not going to burn you, but they’re hot enough that you think they might when you unplug them!

When the problems started occurring, I would move the cable in my office from the 10-gigabit SFP+ module to a 2.5-gigabit Ethernet port on the switch. That would always work fine. I would usually remember to move the cable back a few hours later, and I’d have a flawless 10-gigabit connection again. I assumed the SFP+ modules might be cooling down enough.

Smokeping graph

Each smokey point on my Smokeping graph represents me swapping ports or modules while doing heavy iperf3 testing

At first I was going almost a week between having to do this. Then every couple of days. Then several times a day.

I decided to take one of the switches apart to see what I might be able to do to keep the SFP+ modules cooler, and I thought I had a pretty good idea. Replacing the dried-out thermal compound in my cheap Intel 10-gigabit Ethernet card was necessary to keep them running at full speed. That was a good enough reason for me to give it a try here!

Attempting to keep the SFP+ modules cool in my MokerLink switches

I moved all my cables to the old gigabit switch, I ordered an assortment of thermal pads from Amazon, and I waited for the pads to be delivered. I figured I would stack several pads up between the PCB and the chassis to help the SFP+ modules to transfer more heat from the module to the shell of the switch.

My shipment got delayed a few days. I cut up some thermal pads when they finally arrived, installed them in the MokerLink switch, and moved all my cables back to the 2.5-gigabit and 10-gigabit ports.

Thermal pads installed in my MokerLink 2.5-gigabit Ethernet switch

My stacks of thermal pads installed under the SFP+ ports were inspired by the stock thermal pad that MokerLink installs under their CPU!

I couldn’t get a 10-gigabit connection at all. Did I break the switch? Did I mess up the SFP+ ports?

Probably not. I could get them to connect if the other end of the connection was a 2.5-gigabit port.

I wasn’t sure what was going on, but I figured I might as well modify both switches with the thermal pads for good measure.

Things just started working again one day!

I didn’t think about my 10-gigabit network much at all for a few weeks. This upgrade was just an inexpensive and fun experiment. My old 1-gigabit Ethernet gear was adequate for my daily needs, and the 2.5-gigabit upgrade was still working just fine. The 10-gigabit links were just a bonus. It didn’t hurt at all having them downgraded.

I don’t know what made me remember to try the 10-gigabit link again, but when I did, it was working perfectly. At least for a while. I’m not sure how long it was stable before I ran into trouble again.

This is when I started noticing the correlation to the weather. The 10-gigabit link was more likely to be problematic at night. It is colder at night. Especially since the sun wasn’t beating down on the attic all day.

Then I started being able to make reliable predictions. I would see rain in the forecast, then I would see network problems.

What do you think is happening in my attic? Are my cables contracting in the cold causing something weird to happen in a bend somewhere? Do I have a tiny tear in a jacket of a cable somewhere that is letting moisture in? Is that causing a short, or is the combination of cold and moisture doing something else?

If moisture is causing a short in a damaged cable, then why does it work perfectly at 2.5-gigabit speeds?

I ordered a 5-gigabit SFP+ copper module

One of the two switches involved in this problematic network link is managed. I can set the port to 1000, 2500, or 10,000 megabit. This does not influence the link speed between the SFP+ modules. The switch always detects a 10-gigabit full-duplex link even when the other end of the connection is a 2.5-gigabit port.

When I first set up the 10-gigabit connection across the house, I set up a long chain of couplers and extra patch cables in order to coax the Xicom SFP+ modules into connecting at 5-gigabit speeds. That worked great when the wiring was good, but I haven’t been able to get them to negotiate down to 2.5-gigabit or 5-gigabit on their own.

Lianguo 5-gigabit SFP+ module

Can you see that tiny switch labeled 2 and X on my Lianguo 5-gigabit module? That switches between 2.5-gigabit and 5-gigabit mode!

I wound up ordering a 5-gigabit SFP+ module from Aliexpress. I had two reasons for choosing this particular module. The first is that there really isn’t a big selection of 5-gigabit SFP+ modules. The best part is that this module has a tiny switch that you can flip to set it to 2.5-gigabit or 5-gigabit. I figured that might come in handy!

The 5-gigabit SFP+ module works perfectly. I plugged it in and immediately saw 5-gigabits per second on all my iperf tests.

I want to try the 5-gigabit module while the 10-gigabit link is having problems!

Nothing has gone wrong while I am here to do anything about it. My Smokeping graphs have been very nearly solid green for the last 6 to 8 weeks. We had a cooler night with some rain about three weeks ago, and I had a few cyan blips on the graph. A cyan blip means that one out of twenty pings over a five-minute period didn’t get a response.

I am pretty sure that singular blip happened before the 5-gigabit module arrived at my house.

I joked in Discord that since a potential solution is here in my hands that I won’t see another problem until November. That is feeling less like a joke now.

Conclusion?!

I don’t think we really get to see a conclusion until winter rolls around again. I think the 5-gigabit SFP+ module was a good purchase. Dropping down to a 5-gigabit Ethernet connection to the other side of the house is still a huge upgrade over my old 1-gigabit connection, and that is WAY less work than pulling a new cable.

Some of this is way more obvious with the benefit of hindsight. The temperature or humidity in the attic didn’t occur to me at all early on. When it get cold outside, we turn the heat on. Parts of the house that receive the most cooling in the summer also receive the most heat in the winter. My immediate assumption was that an extra few degrees of heat near my network cupboard was pushing things past the limit. That doesn’t seem to have been the case.

What do you think? Have you ever experienced strange network issues tied to weather or temperature? Could humidity or drops in attic temps be messing with your cabling? Maybe you have a different hypothesis to explain why my 10-gigabit link acts up when it’s cold. Share your thoughts (or war stories!) in the comments below, or jump into our Discord community to geek out over networking mysteries with fellow tech enthusiasts. Let’s solve this together—maybe before winter comes back! ❄️🔌

Comments