BufferBloat, or Gaming on DSL while Uploading

February 27th, 2017

If you find that when some device on your network is uploading data, the entire network becomes unusable due to excess latency, this post is for you.

There's a term for this, BufferBloat, and it refers to the a phenomenon where your ping times increase due to excessive buffering at a choke point on your network.

In my case, I have a DSL network connection with 800kbps upload. When the network is idle, I have pretty good ping times to the outside world, with typical values being around 20ms.

But if some device on my network is uploading data, this can increase to 2000ms or more.

There's an easy test for this now. Go to DSLReports Speed Test and watch the Buffer Bloat score. This test measures ping times during the download and upload portion of the bandwidth metering tests, and if it goes up significantly while testing vs idle, that's buffer bloat.

In my case, quite often some device on the network would be uploading something, and this was killing my ability to play Overwatch on the PS4. It's hard to track down what's uploading. It could be that my wife has recorded a video clip on her phone and it's syncing to iCloud. Maybe I dropped a big file into a Dropbox folder. All I'd know is I couldn't play right now because "something" was killing the network.

I tested my network connected directly to the SpeedStream 6250 DSL modem, and connected through an AirPort Extreme Base Station. Neither made a big difference. But after doing some reading, I discovered there's a QoS algorithm for prioritizing packets that helps with this problem, available for integration into networking equipment firmware. The "codel" algorithm is implemented into newer versions of the DD-WRT custom router firmware project.

My Solution

I'm not using the WiFi on the router, so the wireless stats didn't matter to me. The cheapest DD-WRT compatible router that supports code is the ASUS RT-AC56U. The ASUS RT-N66U also runs DD-WRT, but the codel algorithm isn't enabled in firmware for that router, so it won't help.

I set up that router, installed a beta DD-WRT (the one I used is here), and configured it as my PPPoE gateway. The SpeedStream DSL modem is still there, but it's just acting as a dumb modem, and the RT-AC56U is taking care of PPPoE, and local network services like DHCP.

The last step was enabling QoS on the router. This is in the DD-WRT web interface. Choose the fq_codel queueing discipline, and enter a value in the uplink field that's slightly lower than your connection's actual bandwidth speed. For my 800kbps-rated upload, which typically sees about 80k/second upload speeds, I entered 650 here. You can experiment by changing the numbers and repeatedly running the speed tests.

As you can see, once all this was done, the buffer bloat problem was solved. It worked better than I expected.

Ping times are great in theory, so here's some concrete evidence: I've had Photos.app uploading some large video files for the last few days. Meanwhile, I can go play Overwatch and not see the Ping or Latency indicators at all. It really works!

Thanks to the bufferbloat project for relentlessly pursuing this and developing the algorithms to combat bufferbloat. As of February 2017 none of the mainstream routers have these algorithms built in, but hopefully in the future this will become a standard feature.

Occasionally Connected

February 16th, 2017

I think it's easy for people who have a reliable always-on internet connection to forget that much of the world does not. Even those of us who do may prefer not to use cellular data as much as possible.

What I'd like is for the phone to sync as much data as it can while it has a Wi-Fi connection, on the assumption that I'm going to need that data when I'm offline.

iOS is pretty good at this, but as I recently discovered on a trip where the hotel Wi-Fi only worked in the common areas and I had no cellular connection at all, both iOS itself and the apps I use had some big problems working offline.

Ideally, Background App Refresh would launch every app that wants background refreshes whenever I'd encounter a WiFi network after having been offline for a while, but that's not how it seems to work. I couldn't predict which apps would have fresh data and which would not.

Here's a quick rundown of the apps I was using and my experiences during an occasionally-connected week.

Mail is the gold standard for apps that work either online or offline. When I was within WiFi range, mail would download, and I could then read it and reply whether I was online or offline. Any mail I wrote while offline would send the next time I was online. This is exactly how it should work.

TweetBot would pull down the timeline if I launched it while in WiFi range, but wouldn't go get it on its own, I had to manually launch it.

Safari Reading List seems like an ideal offline feature, but in practice, worked terribly.

When you add a page to the Reading List, if you're online when you do it, Safari will go and cache the page so you can read it offline. But if you're adding pages while you're offline, Safari doesn't proactively go and download the content of those pages when you do come back online. It also doesn't download pages that you added from a different device. This renders Reading List useless as you just can't predict what pages will be available.

Music lets you sync playlists for offline playback, and this works well.

The Washington Post app would download stories if I launched it while online, and keep them cached so I could read them when offline.

Amazon Prime let me download shows for offline playback, but when I tried to play them in the hotel room, it had actually downloaded a French language version of The Grand Tour even though none of my preferences were set that way. Fail.

Plex is awesome. I downloaded some shows to play in the hotel room when offline, and they played perfectly.

Ulysses (and iCloud Drive sync in general) would sync if I was online, but didn't sync unless I was running the app. This is different from iCloud Documents and Data, which I believe would sync in the background even when you weren't running the app.

The Apple Watch Weather complication would just show stale data while offline, with no indication the data was stale. Not good.

I mostly place the blame on Apple here, for not aggressively triggering Background App Refresh for all enabled apps when the device found itself online.

Emotions by Hodelpa

February 6th, 2017

We went to the Dominican Republic for a week, and stayed at Emotions by Hodelpa. ([TripAdvisor]). Here’s some information about the hotel that you might find useful if you’re considering this resort for your trip.

The place is always referred to as “Emotions by Hodelpa”. Hodelpa is a chain, and they recently bought this resort and have put a lot of resources into renovating it.

“Essentia by Emotions” is part of the hotel, but shows up as a separate hotel in the booking sites. It’s actually just one of the Emotions buildings, the closest one to the beach. Essentia guests get some nice perks, including being steps from the beach, the 24 hour beach bar and one of the buffet restaurants.

The beach is beautiful, and there’s a 24-hour all-inclusive bar right on the beach. Most of the guest rooms are across the street from the beach, but it’s a short walk and there are guards at the crossing 24 hours a day so it’s never a problem getting to the beach.

NewImage

The primary language spoken at the resort is Spanish. They will try to accommodate English-language speakers, but it would be worth brushing up on some basic Spanish phrases. All the signage is in English, menus and printed materials are available in English, but the staff all speak Spanish.

The common areas, restaurants and some of the buildings with the guest rooms are new (in January 2017). The renovations are ongoing, and not everything is perfect yet but the design is nice (if stark) and I expect the rough edges will be smoothed out. The rooms are large, but could use more storage.

A lot of reviews mention a lack of hot water. The buildings have solar water heaters on the roof, and I suspect they just aren’t up to the needs of everyone showering at the same time. Try again later, I guess.

Emotions is a small resort. It has one main pool, the one you see in the photos on their website. There’s an adults-only pool, a small Essentia-only pool, and of course the beach.

There’s a nice pond with two large pink flamingos that are always there. I don’t know why they don’t fly away, but they seem perfectly happy there and it was cool seeing them every time we walked past.

It’s in a small town (Juan Dolio) with nothing to do. If you’re looking for adventures or excursions, expect a long drive to one of the larger nearby towns.

WiFi is not available in the rooms, at least, not in our room. There are WiFi repeaters at each building, but the signal doesn’t make it through the concrete walls into the rooms. You can get online on the balcony, but it’s frustratingly slow.

The coffee shop is great. Hand-made espresso drinks including alcoholic coffee drinks, and pastry snacks, no extra charge. The coffee shop doesn’t open until 9am, but there are in-room coffee makers.

There are some specialty restaurants that are included in the all-inclusive package, but you have to make a reservation one day in advance. There’s a lineup in the lobby right at 9am. The restaurants were a nice change from the buffet, but didn’t have a kids menu.

This isn’t a full review, just some notes from our trip. We enjoyed it, but did find it a bit lacking in entertainment and things to do. If you’re looking for a quiet week, this is your place.

Controllers vs Mouse and Keyboard

November 20th, 2016

This was (and is, until tomorrow, November 21st, 2016) a free weekend in Overwatch, so you can download and play the game for free on PC, PS4 and Xbox One. I play a lot of Overwatch on the PS4, so I thought I’d take the opportunity to try it on the PC.

I’m not very good at aiming using a game controller. I find it constantly frustrating trying to line up a shot and overshooting the target. I’ve been trying to get better, by using techniques like roughly aiming with the aiming controls and then moving my player for the fine aiming. This works, but it never feels precise.

I installed Overwatch on the PC and spent a few hours playing it, and first impression is, wow. I can aim! What a difference.

Second impression is: Uh oh, so can everyone else. Snipers seem a lot more dangerous on the PC than on PS4.

When you get killed in Overwatch, the game shows you a replay from the perspective of the person who got you, and it makes me feel better to see, on the PS4, other players having the same trouble. Lining up a shot, overshooting, slowly repositioning … sometimes people make a lucky shot but often it shows other people struggling with the same imprecision I do. 

That levels the playing field, and probably explains why they’ll never give us cross-platform play (PC and console gamers in the same game). The PC gamers would run away with it.

I’d switch back to the PC, but main reasons for going console in the first place still apply: That’s where my friends are, cheaper long term hardware cost (no buying new hardware as minimum requirements rise), and RSI.

I use a trackpad on my Mac, because when I used to use a mouse, I’d start to experience carpal tunnel syndrome symptoms. Wrist pain. Switched to the trackpad and the symptoms vanished. Game controllers are fine, trackpads are fine, but for some reason, I just can’t spend a lot of time gaming with a keyboard and mouse anymore.

But it was nice to jump back into that world for a few hours. Yes, console gamers are at a disadvantage, but we’re all at the same disadvantage, which evens it out. Makes me feel a little less bad about my crummy aim.

Xcode Errors in Source Editor

November 19th, 2016

Here’s a problem I was having recently. My project would build and run fine, but the source editor was showing errors.

Sometimes the errors wouldn’t be there, sometimes they wouldn’t. Sometimes they’d interfere with things like autocomplete, and it made working with the affected source files rather frustrating.

The problem was that the source file was used in more than one target, and one of the targets in a scheme other than the one I was using to develop the app had build settings that were causing the file to not build successfully.

In my case, I’d added some Swift source to the project, and configured the bridging header in my main target, but not in the UI Tests target. This is hard to discover, because there’s no way that I can see to build that target directly from Xcode.

But there is an indirect way to get it to build, and it’s probably a good idea to enable this anyway:

NewImage

Pick “Manage Schemes…” in the dropdown that appears when you click on your project name in the picker on the Xcode toolbar, 

Napkin 16 11 19 6 42 38 AM

Now when you Analyze your project, you’re analyzing not just the main target, but any other targets you select as well. This will show the build errors in Xcode and make it easy to go fix them.

More RAM Please

November 1st, 2016

Here’s my two cents on the new MacBook Pro’s, not that anyone is asking.

This tweet sums it up:

“I’m struck by the cultural divide between dismissive Apple defenders and people who buy an expensive mac for real work every four-five years” – @pinboard

The main reason I’m uncomfortable dropping that much money on a new laptop right now is RAM. My current Mac Pro, late 2010, has 20GB RAM, and my current laptop, a 2012, has 16GB.  I use a lot of RAM, because I use big, RAM-hungry tools, and I’m frequently near using all of it.  I just checked and right now I’m using 15.59GB.

When I bought my laptop, four years ago, 16gb was way more than I needed. But usage increases over time. I’m uncomfortable buying a new computer, expecting it to last another 4 years, with the same amount of memory in my current hardware.

I know it’s probably Intel’s fault, and that’s fine.  What matters to me is that these new laptops aren’t the computer I want to upgrade to, so I will wait.

 

 

“Free” Photo Storage

October 25th, 2016

Google is giving free, unlimited, full-resolution photo storage to all customers of their Google Pixel phone, and giving unlimited storage of “optimized” photos to everyone. This is an attractive deal, and Google using the storage warnings on iOS in their advertising will resonate with a lot of people. I think this will be a real, long-term threat.

How much storage is a Pixel user going to use, over the life of the phone? It really depends on the user, of course, but since the Pixel is a high-end, expensive phone, I’d expect the people who buy them are going to be heavy photo and video users. I could easily see using a few hundred gigabytes of photos and video over a couple of years. Video is large.

Apple added 4K video and Live Photos to the iPhone, features which more than double the per-item storage requirement. In a world where Apple makes a profit from either users buying higher-capacity devices, or paying for iCloud storage, that’s great for Apple. But it makes it more difficult to compete with Google’s free storage option, since it would simply cost Apple more to store the data generated from the same amount of customer usage.

But why is Google doing this? I don’t believe it’s simply to sell more phones.

Google is making hardware as a way to protect their ad business. Everything Google does can be viewed through this lens. Chrome was a way to keep the browser vendors from using an alternate default search engine. Android was a way to keep Apple from cutting Google out of mobile. Google needs you to be using Google services, and is systematically removing anything that gets in the way of that. You can have Google fiber to your Google Wi-Fi to your Google phones, and Chromebooks. It’s Google all the way down.

But, specifically, why photos? I don’t have any inside info here, but from looking at a few obvious trends, I have a hunch.

Recognizing things in images is becoming easier. The search capability that Google has introduced for Google Photos lets you search for photos that contain whatever terms you want to type in. Your phone knows where you are when you take these photos, so Google can tell a lot about the places you take pictures.

For example, it’s obvious from my photos that I have a dog. Why wouldn’t Google use that signal in their ad-selection algorithm? It makes sense, and it’s feasible, so they will.

There’s so much Google could learn about your home, your style, colour preferences, clothing, furniture, and so much more just by analyzing your photos.

That’s the price for free photo storage.

Determine form-sheet or navigation controller presentation

June 29th, 2016

I recently ran into a situation where I needed to determine if a view controller was in a form sheet, or pushed onto a navigation stack.

A pattern I use for things like Settings in a universal app is to push the Settings view controller onto the stack, when on the iPhone, and wrap it in a UINavigationController and present that as a form sheet on the iPad. This works well, but I’ve been working on making my apps adaptive, and my old method of deciding which to use based on the UIUserInterfaceIdiom is not going to cut it in an adaptive world.

One odd thing about UIPresentationController is it doesn’t seem to have a way to determine how a view controller is currently presented. You can find out how the view controller would like to have been presented, but this doesn’t always match what it actually did.

In the end I kind of cheated, but I feel like it’s a good cheat so I’m sharing it.  It boils down to this:

    if (self.navigationController.viewControllers.count > 1) {

        // No “Done” button when in a navigation controller, use the Back button

        self.navigationItem.rightBarButtonItem = nil;

    }

If the view is the only view in its navigation controller, then I want the done button, otherwise, the user will use the navigation controller’s Back button to navigate away.

Writing with Tools

November 28th, 2015

I have always wanted to write more. Every time I get the itch to write something substantial, there is a pattern that plays out. The first thing I do is think “I need to find a great app to write in” and get totally sidelined by downloading and trying out writing apps.

This makes no sense, of course. I know most of what I read is written in BBEdit, Scrivener, or Microsoft word, and that it doesn’t matter. It’s all text, and all a writing app needs to do, most of the time, is make sure the keys you press on the keyboard show up as words on the screen.

But, as I am typing this in Editorial on the iPad, I am looking for a word count and not seeing one. Now I am distracted. It also doesn’t support iPad multitasking. That’s no good. Hmm, what else do I have.

So, now I am in Byword, which supports multitasking and which has an on-screen word count indicator, although it takes up a lot of space. Must resist.

The next challenge is how do I get text from Byword onto my blog. It would be nice if I could use a WordPress extension. Let’s see if that works.

Nope, although I could post directly to Tumblr if I wanted to. Maybe this is why people rave about the Workflow app. Back in a minute.

Playing with Workflow. Hang on.

Ok I think I have a workflow that will let me post this, and Workflow is cool, but it seems that if I want to edit this post later that would be a whole different process. Still, if this makes it to the blog then at least my tool indulgence has resulted in some output.

Nope, when I tried to run the workflow, I got a cryptic “unsupported URL” alert.

But I see Byword has direct support for publishing to WordPress. Let’s try that.

Does anyone else find the tools this distracting?

I published this in Byword, but I don’t see how I can update a post in Byword, so here I am I the WordPress app updating this post.

Darkness under Navigation Controller

September 13th, 2015

Translucent navigation bars are cool. The content that scrolls up off the visible area alters the look of the navigation bar. But they do introduce some complexity, and sometimes you just don’t need it, because your content doesn’t scroll.

So let’s say you’ve got an app that has some content that does scroll, like a UITableView, so you turn on the translucent navigation bar. But then you want to push a view controller that doesn’t scroll.  

So you go into Interface Builder, select your view controller, and turn off “Extend Under Top Bars”.

Under Top Bars Off

But now you get this:

Dark Navigation Bar

It’s almost as the invisible content underneath the translucent navigation bar was black. And that’s because it probably is.

This looks ugly, and it looks especially ugly when it’s animating in, because the white turning black is part of the push animation.

The best way I’ve found to fix this is to put a view above your main view, that acts as a white filler that the navigation bar can use for its translucency.

Drop in a new view, and update its size and position so that it’s above the navigation bar.  You may want to calculate the navigation bar height at runtime, but here I’m just using the size that’s applied since the dawn of iOS, 64 points:

Filler View Size

Make sure to set up the autoresizing mask, or constraints, so it adjusts automatically for different device sizes.

End result?  No more smudge underneath the navigation bar.

After

Much nicer.