Fragmentation on Mac OS X: Time for a Change

October 2nd, 2010

Windows systems suffer from file system fragmentation and benefit from defragmentation. Windows 7 includes a defragmenter (as did previous versions of Windows) and automatically defragment the file system from time to time.

The Mac takes a different approach. There are quite a few mechanisms built into the HFS+ file system to keep the system working smoothly (check out this post for some details), but the most interesting one in terms of fragmentation is on the fly defragmentation.

In a nutshell, when any file under 20mb in size is opened, the system checks to see if it's fragmented. If it is, and some other conditions are met, then the file is automatically, instantly defragmented.

This works great for keeping those "small" files defragmented, but back when this feature was first added to HFS+ about 8 years ago, a 20mb file was pretty large. Not so anymore.

Another change in the demands on the file system is the advent of iTunes, and large media files. It's not uncommon today to be downloading podcasts that are more than 20mb, in addition to video podcasts that are hundreds of megabytes, and movies that are a gigabyte or more. It's also quite easy with iTunes to fill your hard drive with downloaded media.

Check out this fragmentation report of the hard drive in my laptop:

201010020858

That 1st file, a recent download, is split into 12,599 fragments. To put that in perspective, the drive I have in that laptop has a random seek time of about 12ms. 12ms x 12599 is 151 seconds. That's how much time the drive could spend just seeking to read this one file!

The 2nd file is an iPad application. The 4th, 5th, and 6th files are also iTunes downloads. In fact, most of the fragmented files are.

There were only a couple of files under 20mb that had any fragmentation at all, so the auto-defrag is doing its job. But 20mb is too small a size for automatic defragmentation in 2010. In fact, as far as I'm concerned, any attempt to open a fragmented file, no matter how big it is, should be considered - if not for instant defragmentation, then for being added to a queue of files scheduled for background defragmentation. But even if that's out of the picture, the 20mb size should be increased respective to the size and performance increase of computers since 2003, to at least 200mb.
(It's not configurable - you can see in the source here that it's a constant (20 * 1024 * 1024).

Netflix NewWatchInstantlyRSS feed

September 28th, 2010

Now that I'm a new Netflix fan (since they just opened the service up in Canada) I went looking for a way to be notified of interesting new movies added. I found an RSS feed. Cool.

Here's the Canadian one:
http://www.netflix.ca/NewWatchInstantlyRSS

Finally, MST3K in Canada!

Cross Device Apps

September 28th, 2010

Now that Flash is again welcome on iOS devices, and new tablets and phones are showing up with Flash runtime support (and AIR support), app developers have a real choice when it comes to app development technologies.

Let's say you've got an idea for a killer app and you want to build it. Do you target one device or many? And do you develop native, develop in HTML5, or develop for the Flash runtime?

RIM just announced a new OS for their new tablet, the Tablet OS. Developing native and supporting the range of devices that are going to be on the market over the next six or twelve months means supporting iOS, Android, possibly Windows Phone 7, and now the Tablet OS. On top of PC and Mac, of course (hey there are still a few people that run desktop operating systems .. mobile first makes a lot of sense but it doesn't hurt to bring your app to the desktop).

Native can give the best app on every target runtime. Can't argue with that, but at what cost? Even if you have the resources to develop for all these targets, managing that kind of software development (simultaneously shipping 4 implementations of the same app) is difficult and frankly it's better to spend your time working on your app's magic than on platform support.

So what cross-platform options are there?

HTML5, Java, and AIR.

HTML5 works, but the tools for creating applications built on HTML5 technologies aren't there yet. It's an uphill battle building an app that targets HTML5. It's possible, but in my experience, it's not enjoyable (okay so maybe that's not a criteria you care about), and it doesn't convey any real advantages other than that your app runs on multiple targets. There are better ways of achieving that.

Java isn't there on iOS, and the Java runtimes that are there on Android are different enough from what you can do on the desktop that I don't think that's really a viable option. Java itself isn't a problem - you can write classes and run them anywhere - but the framework you need to build the user experience isn't there for cross platform Java apps.

AIR lets you choose whether you want to build your apps using Flash, an animation-focused tool, or Flex and Flash Builder, a developer-oriented tool. It's nice to have that option, since these are very different models and call for different tooling. Flex is built from the ground up for building apps; Flash for building games and a different sort of rich interactivity. Pick whichever works for you.

Twitter Weekly Updates for 2010-09-19

September 19th, 2010

One Year at Home

September 11th, 2010

One year ago today I moved from Ottawa to Brantford, switching from working in the office to working from home full-time.

I'd always wanted to work from home, but I'd never asked an employer if I could. At Adobe, and at previous jobs, my employers were okay with employees working from home one or two days a week so I often did that, but this time I made the leap to not having an office "at the office" at all.

My wife is from Brantford, and moved to Ottawa to be with me. Four years ago we had our first child, and the value of having family nearby started to become apparent. We didn't have any in Ottawa, so every birthday and holiday involved a 12 hour round-trip to Brantford. Plus, there was no grandma dropping by to see her grandson. No playing with the cousins. We eventually decided that it was important to be closer to family.

So we've been here a year. So, how's it working out?

It's been a very good move for my son and my wife. Brantford is a much smaller town than Ottawa (about 100k vs close to a million) but big enough that there isn't much you can't get here. It's got a Future Shop, a decent theatre, and a Starbucks. What more do you need?

Brantford's main industry has been manufacturing, and that's been struggling in recent years. On the positive side, this means housing prices are much more reasonable than in a lot of big cities, and I think that in general, telecommuting means being able to buy a better house for the same dollars, because you don't need to compete for prime real estate.

My commute time has dropped from about an hour a day to the time it takes to walk down the stairs to my office.

Staying Motivated

One question people ask me is how do I stay focused and motivated. There are a few parts to that answer.

If you're not self-motivated to start with, don't try working from home. If you're the sort of person that has a hard time quitting at 5pm, then this is for you.

But keeping in close contact with your co-workers is also important and motivating. At Adobe, we use a development methodology called Scrum. A key part of Scrum is that every day, you have a stand-up meeting with your team (and stand-up literally means you stand during the meeting, which helps keep the meeting short). Knowing that tomorrow morning you need to answer the questions "what did I do yesterday" and "what am I going to do today" helps keep you focused.

And the third part of staying focused, for me anyway, is a good GTD (Getting Things Done) methodology. I use Things, a Mac app that tracks tasks, both for work tasks and for non-work tasks around the house. Every day when I figure out what it is I want to get done, I highlight those things in Things and that helps keep me focused. Scrum has tools for this as well - there's a backlog of tasks that are assigned to me that I can check - but the Things UI and workflow just works better for me.

Keeping in Contact

Staying in touch with the people I work with is incredibly important, and fortunately, Adobe is already very well set up for this.

We use a tool called Adobe Connect that enables easy audio and video conferencing. Most people at Adobe are used to working with Connect for most meetings anyway, so including a Connect URL in a meeting invite is a common thing to do. This lets me see whatever's on the screen in the other meeting room(s), and share my own screen if I want to.

Adobe uses Cisco IP telephony, and thanks to the VPN, the phone on my desk at home has a 613 area code and rings when you dial my extension the same way it would if I was in the office in Ottawa.

Home and Work

There are benefits and drawbacks to being "at home" all day. My wife is at home with Matthew during the day so everyone is here in the house, but the house we moved into is a former duplex and the two sides are completely separated from each other save for one door. That door has a child lock on it, and my son knows that Daddy spends his day over there and he's not allowed over there.

Occasionally if there's some emergency Matthew wants me to attend to, like putting new batteries in his fire truck, he'll ask Mommy to email Daddy to come up and fix it. Seriously. It works.

On the other hand, sometimes when I'm working I like to wander around and think about problems or whatever it is I'm working on. In the office, this usually meant a trip to the kitchen or downstairs to Starbucks. At home, it just means stepping out onto the patio outside my office for some fresh air, or walking upstairs to say Hi to my son.

It's been a great year for me, and looking back my only regret is that I didn't start doing this sooner.

Human Spam

September 9th, 2010

The spammers are paying humans to spam instead of having bots do it. This means the spam is time-consuming to detect as spam, because it isn't obviously spam. And it almost always gets past the spam filters.

This is typical:

201009090945

The comment is perfectly reasonable, almost useful, and posted to an article where it's relevant. But the link? Spam.

Unless you get enough commenters to make croud-sourcing spam filtering work (think Slashdot or Digg), it's up to the blog owner to do it.

I'd just shut off comments altogether, but some of my posts have dozens or hundreds of valuable comments and I'd hate to lose that.

BioShock 2

August 31st, 2010

I'm playing through BioShock 2 on the PC. This is one of the first PC games I've played in a while - I've mostly been playing on the Xbox 360 the last few years.

I've been enjoying the game, but I've reached a point where it just suddenly stopped being fun. The whole game is a bit too linear/scripted, but the difficulty has been pretty reasonable. Now, all of a sudden, I have a fight that I just can't win, after maybe 50 tries.

The part of it that really bugs me, though, is that the game is spawning new enemies as the fight goes on. It's been a long time since I played a game that did that, and I didn't expect it of a game like BioShock 2. It doesn't matter how many enemies you kill, there are always more.

And in the context of BioShock 2, where you're in an underwater city where presumably resources are scarce, it just doesn't make any sense to have an unending horde of bad guys coming at me one at a time. What, are they all just right around the corner?

I'm tempted to just leave it and play something else, but I have been enjoying the story and would hate to miss the ending..

Twitter Weekly Updates for 2010-08-29

August 29th, 2010
  • Working in Flex is so much more fun than working in Java. A big part of it is it feels like me and the framework are on the same side. #
  • I keep Visual Studio around for keyboard macro recording and playback, a feature they nailed about 10 years ago. #
  • The RAID volume is synchronizing now. Please wait for 234.2 minute(s). #

Google Maps Routing

August 24th, 2010

Check this out:

Routing from one arbitrary destination to another isn't easy. Think about how long it takes your in-car GPS navigation device. "Recalculating".

And Google is doing it on every mouse move. Wow.

(Sorry about the video quality - YouTube mashed my video. And apparently if you upload a video that isn't in the aspect ratio it wants, it just scales your video. That's ugly.)

Twitter Weekly Updates for 2010-08-22

August 22nd, 2010
  • i7-930 tjmax is 98 degrees. Ouch. My cooler is keeping it at ~70 under load, which is plenty of headroom but still seems hot. #
  • Good Oracle vs Google analysis: http://redmonk.com/sogrady/2010/08/14/oracle-v-google #
  • I'm surprised nobody's released a tablet to compete with the iPad yet. #
  • @derekcunningham Thanks Derek - doesn't look like they support Java server stuff either (like JBoss). That's what I was looking for. in reply to derekcunningham #
  • Is an annotated Java object still a POJO? It's neither plain nor old. #
  • Companies that want customers to move to online billing need to have simple, reliable websites. Seems obvious, doesn't it? #
  • Bell's site is broken. I think my bill is wrong, and the email summary just includes the amount, no detail. Now what? #
  • I just got a nice "Dyson Wash Day" reminder email from Dyson Canada. Apparently it's time to clean the filter. #
  • Would it be illegal for NBC/Fox/ABC to short Hulu stock after the IPO, and then pull their content? #
  • Lost a 1.5tb hard drive. RAID makes this a :| instead of a :( #
  • EJB messaging seems a lot less intuitive than .NET messaging. So much XML. #
  • Building an MBean is a lot easier than I thought. #
  • I've had the iPhone 4 a couple weeks now, and even still every time I pick it up and look at the screen I'm amazed at the resolution. #
  • Lots of rumours of a Chrome OS tablet release in November. Seems way too late to target Christmas. #
  • Games for Windows Live is terrible. It's hard to believe it's from the same people as Xbox Live, which is awesome. #
  • Every time I use a unibody MacBook Pro I'm reminded how much I hate the buttonless trackpad. I can't buy one until they change that :( #