Building a Slack bot in Node

November 16th, 2018

I was looking into how to build a simple Slack bot. I ran across various bot kits and frameworks and whatnot, that seemed to really overcomplicate the issue. Went back to the Slack API and discovered that it’s actually quite straightforward.

This particular code listens for messages in a private channel called ‘dev’. Private channels are called ‘Groups’ in the Slack API.

The bot listens for messages, looks fo the text ‘ping’ and responds ‘pong’.  The rest is up to you!

Local Siri

May 17th, 2018

Here’s my WWDC prediction for 2018:

Siri will be significantly replaced with a new, on-device Siri.

A few attributes of this new Siri that would make it interesting, and give it a shot of coexisting in a world with Google Assistant, Alexa and Cortana:

  • Requests will run directly on your device. No more waiting for your voice to be uploaded and the response downloaded, no more spotty Siri when your network connectivity is down. This will significantly increase response time and reliability.
  • Siri will treat your devices as a single cloud, fetching information from other devices or sending a request to the device best suited to process it.

This goes a step beyond where Siri is today regarding your privacy, and from that perspective it’s totally in sync with Apple’s privacy stance. But does doing things locally mean worse results?

I don’t think it has to. Some questions will require a network request, like asking how long it would take to drive somewhere, but if you’re an Apple user who’s all-in with iCloud, all your data is already on your device. Your documents, your calendars, messages, emails, can all be scanned by machine learning right on your device. Spotlight already does this for indexing.

So asking “What movies are playing nearby” will result in an internet request, but “Open the garage door” would not, since that can be completely handled inside your house on your local LAN. If you’re out, it will send a message to one of your other devices that is at home, like the Apple TV or HomePod. This is how HomeKit works today.

This is what I’d like to see, but since I’ve never been right with my WWDC predictions, I’d suggest not getting your hopes up.

Formatting CVaListPointer

April 9th, 2018

So you’re using libtiff in Swift, and need to implement the TIFFErrorHandler.

Or, for whatever reason, you have a CVaListPointer (the args parameter, below) and a format string, and you want to format. Here’s how:

let errorHandler:TIFFErrorHandler = {
(moduleNamePtr, formatPtr, args) in
guard let formatPtr = formatPtr else {
// No format string - this is unexpected and there's
// not much we can do here, perhaps log that it happened.

// Convert the format pointer into a Swift String
let formatStr = String(cString:formatPtr)

// Create the output buffer
var buff = Data.init(count: 1024)

// Accessing the buffer's bytes...
buff.withUnsafeMutableBytes({ (ptr) -> () in
// Format into them
vsnprintf(ptr, 1024, formatStr.UTF8CString, args)

guard let formatted = String(data: buff, encoding: .utf8) else {
// This would only happen if the logging produced an invalid string,
// we don't have a string we can display

NSLog("TIFF error: \(formatted)")



The Amazon Web Services Platform

October 19th, 2017

I’ve been working with Amazon Web Services as part of a new job, and I was surprised to see how many rich services Amazon is offering under the AWS banner now.  


I made a point of looking at every one of them (as of October 2017) and reading enough about it to get what it’s for. The scope of these services is incredible, and I get what the “serverless” trend is about now.  You can build and deploy complete application back ends in AWS, start to finish. No surprise to many, I’m sure, but last time I looked at AWS it was mostly storage and VMs.


Here’s my cheat sheet.


Code Hosting


VMs in the cloud.

EC2 Container Service

Docker containers in the cloud.


Cheap, simple VMs in the cloud. Backed by EC2. Check out this StackOverflow post for some details.

Elastic Beanstalk

App hosting. You supply the your application code (Java, PHP, .NET, Node, Tomcat, Ruby, etc), and they run it on a managed server.


Function hosting. You write backend logic code in JS, C#, Java or Python, and it hooks up to an endpoint that you can call or trigger from other AWS activity. “Serverless”.


Manages running large jobs in parallel Docker containers across many EC2 instances.

File Storage


Simple file storage with an HTTP API. Pay for the amount of storage and transfer you use.


Provides filesystem semantics on top of S3.


Slow, low-cost storage, for backups and archiving.

Storage Gateway

Software you run that acts as a local cache for your EFS data. Mount as iSCSI or as a share on your network.



Hosted SQL databases (hosting MySQL, PostgreSQL, or a number of others).


Amazon’s own MongoDB style noSQL database.


Hosted redis / memcached compatible in-memory caching.

Amazon Redshift

“a fast, fully managed, petabyte-scale data warehouse solution”


Artificial Intelligence


Speech recognition and natural language understanding. Build a smart Slack bot.


Text to speech.


Imagine analysis and object detection.

Machine Learning

Machine learning. Upload models, run them on data.



Simple Queue Service

Message queueing service. Create queues, add messages to them, and have various things pull messages and act on them.

Simple Email Service

Send and receive email. SMTP gateway or API.

Simple Notification Service

Pub/Sub messaging, for delivering notifications to different parts of your application, and SMS notification service. (Those two don’t seem that similar to me).


Developer Tools


Sort of a New Project wizard for creating projects that use Amazon services, in a large number of languages. Generates a Git repository and the provisions the services your project type needs.


Amazon does GitHub.


Amazon does continuous integration. 


Automated deployment of built applications to EC2 instances.


Visual workflow tool for orchestrating builds and deployments.


Log and trace collection and analysis for distributed applications.



Database Migration Service

Move your MySQL data into Amazon’s Aurora SQL database.

Server Migration Service

Move your physical PC into an EC2 VM.


Move your local data into and out of Amazon by copying it to physical devices and shipping them around. 


That’s the core stuff. I’m not going to list every other service, just the ones that I think might be of interest to app developers.



CDN. Make your content available quickly worldwide.


Amazon sells domains?


Monitoring for your AWS services.


Create templates for sets of services that you typically deploy together, with a visual editor for adding services and relationships.


Audit trail for things that happen across your AWS services.


Watches your AWS services and makes security, cost and other recommendations.


Automated security analysis of your services.

Certificate Manager

Free SSL certificates for your AWS-hosted services.


User account management services for your apps. Good blog post here.


Web application firewall.


Collect, process and analyze streaming data.


Online game server hosting, with game engine integration.

Mobile Hub

Helps you provision AWS services and integrate them into your mobile app. Not really a service itself.

Device Farm

Deploy your app to real physical devices for testing. “AWS Device Farm tests are run on real, non-rooted devices”


Spam your users with push notifications. I’m sorry, I mean “Engage Your Audience with Messaging Campaigns”. Also includes app usage analytics.

Step Functions

High-level service that lets you create workflows by stringing together Lambda functions.


Another workflow service, which on the surface seems pretty similar to Step Functions. 


Stream desktop applications to a web browser.

Elastic Transcoder

Video format conversion.

API Gateway

Define your own external API to give to your developers as a front end for the services you’ve built in AWS.


There are also services under the AWS banner, like WorkDocs, WorkMail, WorkSpaces and Chime, that seem more like Office365 type user services than developer services.

Parsing Hex Strings in Swift

August 12th, 2017

I want to turn the string "0x02 0x03 0x04" into an array of bytes ([UInt8]). Seems simple enough right?

let hex = "0x02 0x03 0x04"
let components = hex.components(separatedBy: " ")
let array = { UInt8($0) }

But the UInt8 constructor that takes a string doesn’t understand the “0x” prefix to indicate hex strings. There’s a handy UInt8("3F" radix:16) constructor that does understand hex but we have to remove the prefix first.

Swift doesn’t have an obvious way to remove the first 2 characters from a string.

This works:

let hexStr = "0x3F"
let hexValue = UInt8(hexStr.replacingOccurrences(of: "0x", with: ""), radix: 16)

But this will scan the entire string (admittedly only two more bytes in this case). But is there an easy way to just skip the first two bytes?

String has substring(from:String.Index) which seems like what we want, but "0x3F".substring(from:2) doesn’t compile. The String.Index is a type that’s managed by the String, to account for Unicode glyphs that are longer than one byte.

hexStr.substring(from: hexStr.index(hexStr.startIndex, offsetBy: 2)) will give us the correct substring but this is a mouthful – it’s hard to tell from scanning that line that all we want to do is remove the first two characters.

So we have these two options to do this with Swift string methods:

let hex = "0x3F"
let hex1 = hex.replacingOccurrences(of: "0x", with: "")
let hex2 = hex.substring(from: hex.index(hex.startIndex, offsetBy: 2))

As expected, a quick benchmark shows the second version is about 4 times faster than the first version.

There’s also Scanner, which can take the hex string with the 0x prefix and return a hex value, but it works with UInt32 and UInt64 types so there’s some extra gymnastics to get the result into a UInt8. Scanner benchmarks to be almost as fast as using substring, but using a separate class for this operation feels like more overhead than I want.

One of the great things about Swift is we can drop down to C code, and for this particular operation, there’s a C function that fits the bill exactly.

let hex3 = UInt8(strtoul(hex, nil, 16))

The C strtoul function converts a string given a radix, and for radix 16, ignores the prefix. And it’s 3x faster than the Swift substring version above.

For readability let’s wrap this up into a UInt8 extension:

extension UInt8 {
 static func from(hexString: String) -> UInt8 {
   return UInt8(strtoul(hex, nil, 16))

With that in place we finally have:

let hex4 = UInt8.from(hexString: "0x3F")

Since making it easy to apply to other int sizes veers off into generics, I’ll leave that as an exercise for the reader.

Always-On iOS

June 23rd, 2017

The iPad is a perfectly capable computer, but one thing you can’t use the iPad for are jobs that take a long time.

If you’re on a desktop or laptop PC and you need to kick off a job that’s going to take a half hour, such as rendering a 3D video, you just do it. The machine sits there and does the job until it’s done. The computer may go to sleep, which will pause the job, but you can control if and when this happens. The laptop may run out of power, but plug it in and the job continues.

On iOS, you can kick off a long-running job, but you have to keep that app in the foreground until it completes. You can’t hit the Home button and go surf the web until it’s done, because iOS will kill the backgrounded app after 10 minutes, whether it’s done or not. Need to upload a large file to an FTP server? Better hope you can get it done in 10 minutes, or make sure you switch back to the FTP tool before the 10 minutes is up, or the app is killed.

This also means you can’t use an iOS device as a server. The hardware is more than capable of acting in a server role, listening for connections and responding to them, but because hitting the Home button would wind up eventually killing your server, you can’t reliably use an iOS device this way.

iOS has the ability to run background jobs. Photos, for example, downloads and processes your photos in the background. iCloud does this for documents. But no third party applications are allowed to do this.

Suppose I wanted to build an app that would monitor photos from a webcam over wifi, do image detection with the vision API, and take some action depending on what objects are detected. Totally possible and totally reasonable app to build, but the app would have to be launched interactively by the user and kept in the foreground. If the device were to lose power, a user would have to tap on the app to re-launch it.

iOS does seem like an odd platform for this sort of server application, but it’s the platform where all the innovation is happening. Some Apple technologies, like CoreML and HomeKit, are only supported on iOS.

I don’t mind that iOS is so aggressive with background processes when the device is running on battery power. That seems like the right tradeoff. But when plugged in, apps should be able to request, and be granted, the ability to run in the background as long as they need to.

OperationQueue and dynamic priorities

May 7th, 2017

I had a question about OperationQueue that I couldn’t find an easy answer to: Can you modify the priority of an operation once it’s been added to the queue?

My situation is that I have some background operations that are executing, but the user may choose to view one of these items interactively, and at that point, I’d like to bump that item’s priority so the user doesn’t have to wait for it to get its turn in the queue. I want that item to jump to the head of the line.

Many of the StackOverflow responses I looked at said that the queuePriority had to be set before the operation was added to the queue, but I couldn’t find a definitive answer in Apple’s documentation, so I put together a short playground to test it. Here’s the code:

import UIKit

let queue = OperationQueue()

// Serial queue, start suspended
queue.isSuspended = true
queue.maxConcurrentOperationCount = 1

// Create three operations
let operations = [
BlockOperation() {
BlockOperation() {
BlockOperation() {

// Add all the operations
operations.forEach { queue.addOperation($0) }

// Resume (one of the items will start executing immediately)
queue.isSuspended = false

// Now dynamically change the priority of one of the items
operations[2].queuePriority = .high

// Must wait for competion or you won't see the result

Play with which operation you’re adjusting the priority on and you can see the that the priority changes are respected. You can adjust the priorities of operations on a queue.

BufferBloat, or Gaming on DSL while Uploading

February 27th, 2017

If you find that when some device on your network is uploading data, the entire network becomes unusable due to excess latency, this post is for you.

There's a term for this, BufferBloat, and it refers to the a phenomenon where your ping times increase due to excessive buffering at a choke point on your network.

In my case, I have a DSL network connection with 800kbps upload. When the network is idle, I have pretty good ping times to the outside world, with typical values being around 20ms.

But if some device on my network is uploading data, this can increase to 2000ms or more.

There's an easy test for this now. Go to DSLReports Speed Test and watch the Buffer Bloat score. This test measures ping times during the download and upload portion of the bandwidth metering tests, and if it goes up significantly while testing vs idle, that's buffer bloat.

In my case, quite often some device on the network would be uploading something, and this was killing my ability to play Overwatch on the PS4. It's hard to track down what's uploading. It could be that my wife has recorded a video clip on her phone and it's syncing to iCloud. Maybe I dropped a big file into a Dropbox folder. All I'd know is I couldn't play right now because "something" was killing the network.

I tested my network connected directly to the SpeedStream 6250 DSL modem, and connected through an AirPort Extreme Base Station. Neither made a big difference. But after doing some reading, I discovered there's a QoS algorithm for prioritizing packets that helps with this problem, available for integration into networking equipment firmware. The "codel" algorithm is implemented into newer versions of the DD-WRT custom router firmware project.

My Solution

I'm not using the WiFi on the router, so the wireless stats didn't matter to me. The cheapest DD-WRT compatible router that supports code is the ASUS RT-AC56U. The ASUS RT-N66U also runs DD-WRT, but the codel algorithm isn't enabled in firmware for that router, so it won't help.

I set up that router, installed a beta DD-WRT (the one I used is here), and configured it as my PPPoE gateway. The SpeedStream DSL modem is still there, but it's just acting as a dumb modem, and the RT-AC56U is taking care of PPPoE, and local network services like DHCP.

The last step was enabling QoS on the router. This is in the DD-WRT web interface. Choose the fq_codel queueing discipline, and enter a value in the uplink field that's slightly lower than your connection's actual bandwidth speed. For my 800kbps-rated upload, which typically sees about 80k/second upload speeds, I entered 650 here. You can experiment by changing the numbers and repeatedly running the speed tests.

As you can see, once all this was done, the buffer bloat problem was solved. It worked better than I expected.

Ping times are great in theory, so here's some concrete evidence: I've had uploading some large video files for the last few days. Meanwhile, I can go play Overwatch and not see the Ping or Latency indicators at all. It really works!

Thanks to the bufferbloat project for relentlessly pursuing this and developing the algorithms to combat bufferbloat. As of February 2017 none of the mainstream routers have these algorithms built in, but hopefully in the future this will become a standard feature.

Occasionally Connected

February 16th, 2017

I think it's easy for people who have a reliable always-on internet connection to forget that much of the world does not. Even those of us who do may prefer not to use cellular data as much as possible.

What I'd like is for the phone to sync as much data as it can while it has a Wi-Fi connection, on the assumption that I'm going to need that data when I'm offline.

iOS is pretty good at this, but as I recently discovered on a trip where the hotel Wi-Fi only worked in the common areas and I had no cellular connection at all, both iOS itself and the apps I use had some big problems working offline.

Ideally, Background App Refresh would launch every app that wants background refreshes whenever I'd encounter a WiFi network after having been offline for a while, but that's not how it seems to work. I couldn't predict which apps would have fresh data and which would not.

Here's a quick rundown of the apps I was using and my experiences during an occasionally-connected week.

Mail is the gold standard for apps that work either online or offline. When I was within WiFi range, mail would download, and I could then read it and reply whether I was online or offline. Any mail I wrote while offline would send the next time I was online. This is exactly how it should work.

TweetBot would pull down the timeline if I launched it while in WiFi range, but wouldn't go get it on its own, I had to manually launch it.

Safari Reading List seems like an ideal offline feature, but in practice, worked terribly.

When you add a page to the Reading List, if you're online when you do it, Safari will go and cache the page so you can read it offline. But if you're adding pages while you're offline, Safari doesn't proactively go and download the content of those pages when you do come back online. It also doesn't download pages that you added from a different device. This renders Reading List useless as you just can't predict what pages will be available.

Music lets you sync playlists for offline playback, and this works well.

The Washington Post app would download stories if I launched it while online, and keep them cached so I could read them when offline.

Amazon Prime let me download shows for offline playback, but when I tried to play them in the hotel room, it had actually downloaded a French language version of The Grand Tour even though none of my preferences were set that way. Fail.

Plex is awesome. I downloaded some shows to play in the hotel room when offline, and they played perfectly.

Ulysses (and iCloud Drive sync in general) would sync if I was online, but didn't sync unless I was running the app. This is different from iCloud Documents and Data, which I believe would sync in the background even when you weren't running the app.

The Apple Watch Weather complication would just show stale data while offline, with no indication the data was stale. Not good.

I mostly place the blame on Apple here, for not aggressively triggering Background App Refresh for all enabled apps when the device found itself online.

Emotions by Hodelpa

February 6th, 2017

We went to the Dominican Republic for a week, and stayed at Emotions by Hodelpa. ([TripAdvisor]). Here’s some information about the hotel that you might find useful if you’re considering this resort for your trip.

The place is always referred to as “Emotions by Hodelpa”. Hodelpa is a chain, and they recently bought this resort and have put a lot of resources into renovating it.

“Essentia by Emotions” is part of the hotel, but shows up as a separate hotel in the booking sites. It’s actually just one of the Emotions buildings, the closest one to the beach. Essentia guests get some nice perks, including being steps from the beach, the 24 hour beach bar and one of the buffet restaurants.

The beach is beautiful, and there’s a 24-hour all-inclusive bar right on the beach. Most of the guest rooms are across the street from the beach, but it’s a short walk and there are guards at the crossing 24 hours a day so it’s never a problem getting to the beach.


The primary language spoken at the resort is Spanish. They will try to accommodate English-language speakers, but it would be worth brushing up on some basic Spanish phrases. All the signage is in English, menus and printed materials are available in English, but the staff all speak Spanish.

The common areas, restaurants and some of the buildings with the guest rooms are new (in January 2017). The renovations are ongoing, and not everything is perfect yet but the design is nice (if stark) and I expect the rough edges will be smoothed out. The rooms are large, but could use more storage.

A lot of reviews mention a lack of hot water. The buildings have solar water heaters on the roof, and I suspect they just aren’t up to the needs of everyone showering at the same time. Try again later, I guess.

Emotions is a small resort. It has one main pool, the one you see in the photos on their website. There’s an adults-only pool, a small Essentia-only pool, and of course the beach.

There’s a nice pond with two large pink flamingos that are always there. I don’t know why they don’t fly away, but they seem perfectly happy there and it was cool seeing them every time we walked past.

It’s in a small town (Juan Dolio) with nothing to do. If you’re looking for adventures or excursions, expect a long drive to one of the larger nearby towns.

WiFi is not available in the rooms, at least, not in our room. There are WiFi repeaters at each building, but the signal doesn’t make it through the concrete walls into the rooms. You can get online on the balcony, but it’s frustratingly slow.

The coffee shop is great. Hand-made espresso drinks including alcoholic coffee drinks, and pastry snacks, no extra charge. The coffee shop doesn’t open until 9am, but there are in-room coffee makers.

There are some specialty restaurants that are included in the all-inclusive package, but you have to make a reservation one day in advance. There’s a lineup in the lobby right at 9am. The restaurants were a nice change from the buffet, but didn’t have a kids menu.

This isn’t a full review, just some notes from our trip. We enjoyed it, but did find it a bit lacking in entertainment and things to do. If you’re looking for a quiet week, this is your place.