MealPlan Sync with Ensembles
MealPlan is a meal and grocery planning app that, up until recently, was only available on the iPad. I wanted to bring it to the iPhone, but I had to implement sync first.
I started with iCloud Core Data Sync from Apple, back before iOS 7, and ran into nothing but troubles. It was seriously broken, and that fact has been documented on dozens of sites so I won't address it here. I lost about two months worth of development time to trying to get iCloud to sync using Core Data before abandoning that.
I assumed I'd have to write my own sync back end. I didn't really want to do that for a number of reasons. One of the main ones is that, given the amount of revenue the app generates, it doesn't make sense for me to be "on call" 24/7 in case there's a problem on the server side. I wanted something that would just work, with no ongoing involvement from myself.
Drew McCormack's Ensembles project provided the answer.
Ensembles provides the solution that iCloud Core Data sync should have been. It uses a similar mechanism: Sync changes from your application's database into iCloud and then merging changes from other devices back into your local database. But Ensembles is simply a better design.
I want to talk a bit about implementing Ensembles in MealPlan.
Since MealPlan started with the intention of using iCloud to sync, the application's main data is stored in a class derived from UIManagedDocument. UIManagedDocument is a UIDocument subclass that is conceptually a document, but is actually stored as a Core Data database.
In retrospect, this is a leaky abstraction that shows through in the mismatch between a database API and a document API. But UIManagedDocument does two things that I found useful.
First, it takes care of setting up the Core Data stack. This isn't a big deal, but it does create a parent / child context where the child context is on the main thread and the parent context runs on a private queue. This is an efficient arrangement because it means database operations on the main thread often happen very quickly since they're just querying or pushing data to the parent context, which does the actual database operations in the background.
And second, UIManagedDocument changes the way you manage saving data, so that instead of explicitly calling save whenever you make a change, you instead call [document updateChangeCount:UIDocumentChangeDone] to tell UIManagedDocument that you've made a change.
UIManagedDocument doesn't immediately translate this into a save. Instead, it sets a timer for about 5 seconds, and if no other changes happen in that time, then the save occurs. This means that as the user is working with the app, saves happen less frequently. Whether this is a good thing or not, in retrospect, I can't really say. Seems like a wash to me. But because MealPlan was built around this save mechanism, it would have been a lot of work to switch away from it.
One useful tip when working with UIManagedDocument is to override contentsForType:error: and print out that the document is being saved. This is called when UIManagedDocument needs to "save" the document.
- (id)contentsForType:(NSString *)typeName
error:(NSError *__autoreleasing *)outError
{
DDLogInfo(@"Probably Auto-Saving Document");
return [super contentsForType:typeName error:outError];
}
The API here looks like you'd expect the document content to be returned from this method, which would make no sense for a database, and in fact if you look at the data returned from the superclass it's simply a dictionary with some keys that identify the database.
Ensembles, on the other hand, needs data to be on disk when a sync happens.
One of the nice things about Ensembles, compared to Core Data sync, is that it's essentially deterministic. You drive the sync process, in that sync happens only when you request it. From there it's an asynchronous operation, but there are enough callbacks that you can have a pretty good idea what's going on.
In MealPlan, I use the contentsForType:error: as a save notification from UIManagedDocument, and queue up a sync operation there:
- (id)contentsForType:(NSString *)typeName
error:(NSError *__autoreleasing *)outError
{
DDLogInfo(@"Probably Auto-Saving Document");
// Only queue up a sync if there were changes to save
BOOL shouldSync = (self.managedObjectContext.hasChanges);
id dataToSave = [super contentsForType:typeName error:outError];
if (shouldSync) {
[self performSelector:@selector(triggerSync)
withObject:self
afterDelay:1.0];
}
return dataToSave;
}
The one second delay before sync gives UIManagedDocument a chance to finish the save before syncing. Although this looks like a race condition, if the save isn't complete, that's okay, we'll pick up on that later.
I'm getting a bit ahead of myself .. let's start back at the beginning. The first thing we need to do is initialize Ensembles, but we need to do this after UIManagedDocument has had a chance to set up the Core Data stack. The place to do this is in the open completion handler.
Here's the code I use to open the document:
- (void)setupDocument
{
self.document = [MPDocument documentWithName:self.docFileName completionHandler:^(BOOL success) {
if (!success) {
DDLogError(@"Document failed to open.");
return;
}
// Set the merge policy
self.document.managedObjectContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy;
NSManagedObjectContext *parentContext = self.document.managedObjectContext.parentContext;
[parentContext performBlockAndWait:^{
parentContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy;
}];
[[NSNotificationCenter defaultCenter] addObserver:self
selector:@selector(handleManagedObjectContextDidSaveNotification:)
name:NSManagedObjectContextDidSaveNotification
object:self.document.managedObjectContext];
[self setupEnsembles];
CDECodeBlock documentReadyBlock = ^{
[self watchEnsemblesNotifications];
[self syncNow];
};
if (!self.ensemble.isLeeched) {
if ([MPSettings syncEnabled]) {
DDLogInfo(@"Leeching Store");
[self.ensemble leechPersistentStoreWithCompletion:^(NSError *error) {
if (error != nil) {
DDLogError(@"Error leeching: %@", error);
} else {
DDLogInfo(@"Leech complete");
}
documentReadyBlock();
}];
} else {
documentReadyBlock();
}
} else {
documentReadyBlock();
}
}];
}
There are a few items of note here.
Leeching is the Ensembles term for when your database is being synced by Ensembles. You must leech your database once, and it will stay leeched until you deleech it, or a deleech happens due to an error.
In MealPlan, whether or not sync is enabled is a user setting. If the user has not enabled sync, then although MealPlan still initializes Ensembles, the database is not leeched and no syncing happens. If sync is enabled, this means the user would like the database to be leeched, so attempt to leech it. If the leech fails, that's okay (this can happen for various reasons, that I'll mention later), but there will be no sync happening.
The merge policy is being set for both the parent and child contexts. This is required.
Let's take a look at setupEnsembles.
- (void)setupEnsembles
{
DDLogVerbose(@"Setting up Ensembles");
NSPersistentStore *store = [self.document.managedObjectContext
.persistentStoreCoordinator.persistentStores lastObject];
NSString *storePath = store.URL.path;
// Setup Ensemble
self.cloudFileSystem = [[CDEICloudFileSystem alloc]
initWithUbiquityContainerIdentifier:@"TEAMID.com.mycompany.myappid"];
NSURL *modelURL = [[NSBundle mainBundle] URLForResource:@"Model" withExtension:@"momd"];
self.ensemble = [[CDEPersistentStoreEnsemble alloc] initWithEnsembleIdentifier:@"MainStore"
persistentStorePath:storePath
managedObjectModelURL:modelURL
cloudFileSystem:self.cloudFileSystem];
self.ensemble.delegate = self;
}
The only thing that's unique here for working with UIManagedDocument is where to find the NSPersistentStore.
Ensembles imports data from iCloud, and is watching the iCloud container, but won't act unless you tell it to. But Ensembles will tell you when new files have appeared in the iCloud container, which is a good time to sync. The CDEICloudFileSystemDidDownloadFilesNotification is your cue that you should sync.
Syncing looks like this:
- (void)syncNow
{
if (![MPSettings syncEnabled]) {
return;
}
if (self.ensemble == nil) {
// Not ready yet - this can happen if, for example, you trigger a sync when restoring from background, but
// the initial document open isn't completed yet.
return;
}
if (self.ensemble.isMerging) {
DDLogVerbose(@"Skipping sync because Ensembles is already syncing");
return;
}
if (!self.ensemble.isLeeched) {
// Not leeched
return;
}
DDLogInfo(@"Initiating sync: Saving Document");
[self.document.managedObjectContext performBlockAndWait:^{
NSError *error = nil;
NSSet *objects = self.document.managedObjectContext.registeredObjects.allObjects;
if (![self.document.managedObjectContext obtainPermanentIDsForObjects:objects
error:&error]) {
DDLogError(@"Error obtaining permanent IDs");
}
}];
[self.document saveContextsWithCompletion:^(BOOL success) {
if (!success) {
DDLogError(@"Saving document failed");
return;
}
DDLogInfo(@"Save complete, initiating sync.");
[self syncNowWithoutSaving];
}];
}
What's happening here is the app determines if it really wants to sync now by checking some conditions, asks Core Data to assign permanent IDs to all the objects it's managing (since Ensembles needs these), forces both the child and parent context to save, and then calls syncNowWithoutSaving.
Saving is straightforward, but asynchronous, so it signifies completion using a callback.
- (void)saveContextsWithCompletion:(void (^)(BOOL success))completionHandler
{
NSError *error = nil;
if (![self.managedObjectContext save:&error]) {
DDLogError(@"Error saving document context: %@", error);
completionHandler(NO);
}
[self.managedObjectContext.parentContext performBlock:^{
NSError *error = nil;
if (![self.managedObjectContext.parentContext save:&error]) {
DDLogError(@"Error saving document parent context: %@", error);
completionHandler(NO);
} else {
completionHandler(YES);
}
}];
}
And this brings us to the syncNowWithoutSaving method, which actually makes the sync happen.
- (void)syncNowWithoutSaving
{
[MPSettings setSyncStatusLine:@"Sync in progress"];
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
if (!self.ensemble.isMerging) {
[self.ensemble mergeWithCompletion:^(NSError *error) {
if (error) {
if ([error.domain isEqualToString:CDEErrorDomain]) {
if (error.code == CDEErrorCodeSaveOccurredDuringMerge) {
DDLogError(@"Save occurred during merge; retry in 5 seconds");
} else if (error.code != CDEErrorCodeCancelled) {
[self updateSyncStatusLineFromError:error];
DDLogError(@"Sync failed: %@", error);
}
} else {
[self updateSyncStatusLineFromError:error];
DDLogError(@"Sync failed: %@", error);
}
// Try again in 5 secs
[self performSelector:@selector(syncNowWithoutSaving) withObject:nil afterDelay:5.0];
} else {
[MPSettings setSyncStatusLine:nil];
[MPSettings setLastSyncDate:[NSDate date]];
if (self.bgTaskIdentifier != UIBackgroundTaskInvalid) {
[[UIApplication sharedApplication] endBackgroundTask:self.bgTaskIdentifier];
self.bgTaskIdentifier = UIBackgroundTaskInvalid;
}
DDLogInfo(@"Sync complete");
}
}];
}
}];
}
Part of what I wanted to achieve with my integration with Ensembles is keeping the user informed, without being overwhelming. In the app's Settings, there is a status line below the iCloud switch that shows a short message. This message will be either "Last sync: " and the date, or one of the messages you see above. Typically, it will be "Sync in progress" for a short while, "Importing", and then back to the idle message.
Because the rest of the app is alive while the sync is happening, the user may well have done something, and because of how UIManagedDocument works, the change the user has made may not yet be saved. Ensembles gives the application a chance to handle this:
- (BOOL)persistentStoreEnsemble:(CDEPersistentStoreEnsemble *)ensemble
shouldSaveMergedChangesInManagedObjectContext:(NSManagedObjectContext *)savingContext
reparationManagedObjectContext:(NSManagedObjectContext *)reparationContext
{
if (self.document.managedObjectContext.hasChanges) {
DDLogVerbose(@"Should merge? No, there are unsaved changes.");
return NO;
}
if (self.document.managedObjectContext.parentContext.hasChanges) {
DDLogVerbose(@"Should merge? No, the parent context has unsaved changes.");
return NO;
}
return YES;
}
And then once the sync has completed, Ensembles gives you the notification that you must use to merge the changes into your own context:
- (void)persistentStoreEnsemble:(CDEPersistentStoreEnsemble *)ensemble
didSaveMergeChangesWithNotification:(NSNotification *)notification
{
NSAssert(self.document != nil, @"Expected a document");
NSAssert(self.document.managedObjectContext != nil, @"Expected a context");
[self.document.managedObjectContext.parentContext performBlock:^{
[self.document.managedObjectContext.parentContext mergeChangesFromContextDidSaveNotification:notification];
[self.document.managedObjectContext performBlock:^{
[self.document.managedObjectContext mergeChangesFromContextDidSaveNotification:notification];
[self.document updateChangeCount:UIDocumentChangeDone];
[[NSNotificationCenter defaultCenter] postNotificationName:MPSyncCompletedNotification object:nil];
}];
}];
}
Notice the changes being merged into both the parent context (first) and then the child context.
So how does the user turn on Sync? I have a switch in Settings, and when the user toggles it, that's when the database is leeched or deleeched.
- (IBAction)enableiCloudSwitchChanged:(id)sender
{
if (self.enableiCloudSwitch.on) {
[self updateFromSettings];
[MPSettings setSyncEnabled:YES];
if (self.ensemble.isLeeched) {
DDLogVerbose(@"Ensemble is already leeched, syncing now");
[[MPAppDelegate instance] syncNow];
} else {
DDLogVerbose(@"Saving document in preparation for leech");
[MPSettings setSyncStatusLine:@"Preparing to sync"];
[self.document saveContextsWithCompletion:^(BOOL success) {
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
[self attemptLeech];
}];
}];
}
} else {
[MPSettings setSyncEnabled:NO];
[self.ensemble deleechPersistentStoreWithCompletion:^(NSError *error) {
[self updateFromSettings];
}];
}
}
The attemptLeech function actually has a fair bit of work to do, since this is where the user is trying to turn on sync for the first time, and many things can go wrong.
- (void)attemptLeech
{
[self.ensemble leechPersistentStoreWithCompletion:^(NSError *error) {
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
if (error != nil) {
if ([error.domain isEqualToString:CDEErrorDomain] &&
error.code == CDEErrorCodeSaveOccurredDuringLeeching) {
// Silently try leech again
[self performSelector:@selector(attemptLeech) withObject:nil afterDelay:1.0];
} else if ([error.domain isEqualToString:CDEErrorDomain] && error.code == CDEErrorCodeCancelled) {
// Canceled also means a save happened during the leech, and we should just try again
[self performSelector:@selector(attemptLeech) withObject:nil afterDelay:1.0];
} else if ([error.domain isEqualToString:CDEErrorDomain] &&
((error.code == CDEErrorCodeFileCoordinatorTimedOut) ||
(error.code == CDEErrorCodeMissingDependencies))) {
// These are errors we recognize as transient, so just update the progress messages
if (error.code == CDEErrorCodeMissingDependencies) {
[MPSettings setSyncStatusLine:@"Waiting for data"];
}
if (error.code == CDEErrorCodeFileCoordinatorTimedOut) {
[MPSettings setSyncStatusLine:@"Waiting for iCloud sync"];
}
[self performSelector:@selector(attemptLeech) withObject:nil afterDelay:1.0];
} else {
// This is a failure
[MPSettings setSyncEnabled:NO];
[MPSettings setSyncStatusLine:[NSString stringWithFormat:@"Error initiating sync."]];
NSString *format = @"Check your connectivity and iCloud status and try again.\nError %@:%d";
NSString *message = [NSString stringWithFormat:format, error.domain, error.code];
UIAlertView *alert = [[UIAlertView alloc] initWithTitle:@"Error initiating sync" message:message
delegate:nil
cancelButtonTitle:@"OK"
otherButtonTitles:nil];
[alert show];
}
} else {
[[MPAppDelegate instance] syncNow];
}
}];
}];
}
One nuance here is that again because of the asynchronous save nature of UIManagedDocument, there may be changes that you need to save before you can leech. Leeching a document with unsaved changes will lead to problems.
And that, in a rather large nutshell, is the basics of syncing a UIManagedDocument-based application using Ensembles. Some of this seems obvious in retrospect, but it took me some time to tune sync so that it felt right, including getting the status reporting right. I hope you found this useful.
Questions? Comments? I'm @stevex on Twitter.