If you find that when some device on your network is uploading data, the entire network becomes unusable due to excess latency, this post is for you.
There's a term for this, BufferBloat, and it refers to the a phenomenon where your ping times increase due to excessive buffering at a choke point on your network.
In my case, I have a DSL network connection with 800kbps upload. When the network is idle, I have pretty good ping times to the outside world, with typical values being around 20ms.
But if some device on my network is uploading data, this can increase to 2000ms or more.
There's an easy test for this now. Go to DSLReports Speed Test and watch the Buffer Bloat score. This test measures ping times during the download and upload portion of the bandwidth metering tests, and if it goes up significantly while testing vs idle, that's buffer bloat.
In my case, quite often some device on the network would be uploading something, and this was killing my ability to play Overwatch on the PS4. It's hard to track down what's uploading. It could be that my wife has recorded a video clip on her phone and it's syncing to iCloud. Maybe I dropped a big file into a Dropbox folder. All I'd know is I couldn't play right now because "something" was killing the network.
I tested my network connected directly to the SpeedStream 6250 DSL modem, and connected through an AirPort Extreme Base Station. Neither made a big difference. But after doing some reading, I discovered there's a QoS algorithm for prioritizing packets that helps with this problem, available for integration into networking equipment firmware. The "codel" algorithm is implemented into newer versions of the DD-WRT custom router firmware project.
I'm not using the WiFi on the router, so the wireless stats didn't matter to me. The cheapest DD-WRT compatible router that supports code is the ASUS RT-AC56U. The ASUS RT-N66U also runs DD-WRT, but the codel algorithm isn't enabled in firmware for that router, so it won't help.
I set up that router, installed a beta DD-WRT (the one I used is here), and configured it as my PPPoE gateway. The SpeedStream DSL modem is still there, but it's just acting as a dumb modem, and the RT-AC56U is taking care of PPPoE, and local network services like DHCP.
The last step was enabling QoS on the router. This is in the DD-WRT web interface. Choose the fq_codel queueing discipline, and enter a value in the uplink field that's slightly lower than your connection's actual bandwidth speed. For my 800kbps-rated upload, which typically sees about 80k/second upload speeds, I entered 650 here. You can experiment by changing the numbers and repeatedly running the speed tests.
As you can see, once all this was done, the buffer bloat problem was solved. It worked better than I expected.
Ping times are great in theory, so here's some concrete evidence: I've had Photos.app uploading some large video files for the last few days. Meanwhile, I can go play Overwatch and not see the Ping or Latency indicators at all. It really works!
Thanks to the bufferbloat project for relentlessly pursuing this and developing the algorithms to combat bufferbloat. As of February 2017 none of the mainstream routers have these algorithms built in, but hopefully in the future this will become a standard feature.