Problem with relationship one-to-many when documents are created simultaneously on multiple devices

[mastohhh]

I don't know how UnitTest this, so I'll try to explain my problem here.

I have an Entry document that has many Subentries.

To display all subentries of an entry, I use a NSFecthedResultsController on Subentry entity with this predicate :

[NSPredicate predicateWithFormat:@"entry == %@", entry];

When the controller triggers delegate with :

- (void)controllerDidChangeContent:(NSFetchedResultsController *)controller

I can loop on controller.fetchedObjects or entry.subentries. This works fine while subentries are created one after an other on each devices.

But when I create them simultaneously, controller.fetchedObjects contains good subentries, but entry.subentries is not updated. One subentry is missing on each device.

I'm using master branch of CouchbaseLite and NSIncrementalStore.

I would like to UnitTest it for you but I didn't find a similar one that simulates 2 devices.

[pasin]

Thanks for reporting the issue. Normally when we need to simulate a behavior that involves syncing between 2 devices, we create 2 different CBL databases replicating to the same remote sync-gateway URL. If you are interested in creating the unit test for this, you could make it that way.

Anyway, I would like to make sure that I understand the issue correctly.

1. Does this mean that 2 subentries were synced correctly into both devices?
2. I understand that calling controller.fetchedObjects will execute a fetch request. Does this mean that fetching with the predicate @"entry == %@", entry on the Subentry entity yielding the correct results?
3. Does entry.subentries miss an entry created on another device?

[mastohhh]

This is it. But in the next runloop everything comes back to normal.

I think the FRC is triggered just before the relationship is filled.

If you can't reproduce that, I'll try a UnitTest.

[mastohhh]

I have more info for you.

I disabled cache that was introduced with : 4f852bc

(I just commented the line _relationshipCache = [[NSCache alloc] init];)

It works way better. I think this could explain my problem : when FRC is triggered with new entries, the entry.subentries calls cache instead of querying again.

I hope this will help you !

[pasin]

Thanks a lot for the info and your patient. I have been working on the other issue, and I will tackle this issue this week.

[mastohhh]

I found something interesting.

When I got the bug with an Entry without subentries (but subentries pointing to the good entry), everything comes back to normal when I call refreshObject:mergeChanges: method.

To deliver app to my testers, I did an ugly hack in the -(void)informManagedObjectContext:updatedIDs:deletedIDs: CBLIS method : I call refreshObject:mergeChanges: on every changed objects.

- (void) informManagedObjectContext: (NSManagedObjectContext*)context updatedIDs: (NSArray*)updatedIDs deletedIDs: (NSArray*)deletedIDs {
    [context performBlock:^{
        // ...
        // CBLIS code removed for this example
        // ...
        dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(2 * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{
            [context performBlock:^{
                for (NSManagedObject *moc in inserted) {
                    [moc willAccessValueForKey: nil];
                    [context refreshObject: moc mergeChanges: YES];
                }
                for (NSManagedObject *moc in updated) {
                    [moc willAccessValueForKey: nil];
                    [context refreshObject: moc mergeChanges: YES];
                }
            }];
        });
    }];
}

I don't know what happens in the CouchbaseLite stack, but maybe documents comes from the replication in different order and sometimes objects are not refreshed. Especially for objects with one-to-many relationships. (I don't have many-to-many in my project).

I hope this will be helpful.

[mastohhh]

Ok I think I found it.

Replication gives batches of documents to the context. And sometimes Subentries comes in a separate batch of documents than their Entries.

So, when a Subentry is inserted, his Entry has already been updated and refreshed in the previous batch and at that time, the refresh didn't see his subentries.

What I do now, when an object is inserted, in addition to refresh it, I refresh all his toOne relationships. On every new subentries, I refresh their entry.

- (void) informManagedObjectContext: (NSManagedObjectContext*)context updatedIDs: (NSArray*)updatedIDs deletedIDs: (NSArray*)deletedIDs {
    [context performBlock:^{
        NSMutableDictionary* userInfo = [NSMutableDictionary dictionaryWithCapacity: 3];
        NSMutableSet* updatedEntities = [NSMutableSet set];

        if (updatedIDs.count > 0) {
            NSMutableArray* updated = [NSMutableArray arrayWithCapacity: updatedIDs.count];
            NSMutableArray* inserted = [NSMutableArray arrayWithCapacity: updatedIDs.count];

            for (NSManagedObjectID* mocid in updatedIDs) {
                NSManagedObject* moc = [context objectRegisteredForID: mocid];
                if (!moc) {
                    moc = [context objectWithID: mocid];
                    [inserted addObject: moc];

                    // Ensure that a fault has been fired:
                    [moc willAccessValueForKey: nil];
                    [context refreshObject: moc mergeChanges: YES];

                    /*
                     REFRESH TO-ONE OBJECTS
                     */
                    for (NSString *relationshipName in moc.entity.relationshipsByName) {
                        NSRelationshipDescription *relationshipDescription = moc.entity.relationshipsByName[relationshipName];

                        if (NO == relationshipDescription.toMany) {
                            NSManagedObject *destinationObject = [moc valueForKey:relationshipName];
                            if (nil != destinationObject) {
                                // Ensure that a fault has been fired:
                                [destinationObject willAccessValueForKey: nil];
                                [context refreshObject: destinationObject mergeChanges: YES];
                            }
                        }
                    }
                    /*
                     END
                     */
                } else {
                    [updated addObject: moc];

                    // Ensure that a fault has been fired:
                    [moc willAccessValueForKey: nil];
                    [context refreshObject: moc mergeChanges: YES];
                }

                [updatedEntities addObject: moc.entity.name];
            }
            [userInfo setObject: updated forKey: NSUpdatedObjectsKey];
            if (inserted.count > 0) {
                [userInfo setObject: inserted forKey: NSInsertedObjectsKey];
            }
        }

        if (deletedIDs.count > 0) {
            NSMutableArray* deleted = [NSMutableArray arrayWithCapacity: deletedIDs.count];
            for (NSManagedObjectID* mocid in deletedIDs) {
                NSManagedObject* moc = [context objectWithID: mocid];
                [context deleteObject: moc];
                // load object again to get a fault
                [deleted addObject: [context objectWithID: mocid]];

                [updatedEntities addObject: moc.entity.name];
            }
            [userInfo setObject: deleted forKey: NSDeletedObjectsKey];
        }

        // Clear cache:
        for (NSString* entity in updatedEntities) {
            [self purgeCachedObjectsForEntityName: entity];
        }

        NSNotification* didUpdateNote = [NSNotification notificationWithName: NSManagedObjectContextObjectsDidChangeNotification
                                                                      object: context
                                                                    userInfo: userInfo];
        [context mergeChangesFromContextDidSaveNotification: didUpdateNote];

        [[NSNotificationCenter defaultCenter] postNotificationName: kCBLISObjectHasBeenChangedInStoreNotification
                                                            object: self
                                                          userInfo: @{
                                                                      NSDeletedObjectsKey: deletedIDs,
                                                                      NSUpdatedObjectsKey: updatedIDs
                                                                      }];
    }];
}

I didn't reproduce the bug anymore. I think I solved it. I just need you to tell me if this is good. Do you want a pull request ?

I don't know how to reproduce it with a UnitTest. It would be great to have one for that.

[mastohhh]

And _relationshipCache was not causing the problem ...

[pasin]

@mastohhh look like that is the right fix. Could you please submit a PR?

Thanks a lot for your contribution.