Crash during replication

[lucatorella]

During replication I'm getting EXC_BAD_ACCESS and app is crashing

-[CBLRestReplicator removeRemoteRequest:]:
    0x1053181e3 <+0>:   pushq  %rbp
    0x1053181e4 <+1>:   movq   %rsp, %rbp
    0x1053181e7 <+4>:   pushq  %r15
    0x1053181e9 <+6>:   pushq  %r14
    0x1053181eb <+8>:   pushq  %r13
    0x1053181ed <+10>:  pushq  %r12
    0x1053181ef <+12>:  pushq  %rbx
    0x1053181f0 <+13>:  pushq  %rax
    0x1053181f1 <+14>:  movq   %rdi, %rbx
    0x1053181f4 <+17>:  movq   %rdx, %rdi
    0x1053181f7 <+20>:  callq  *0x7ee28b(%rip)           ; (void *)0x000000010a0a1d00: objc_retain
    0x1053181fd <+26>:  movq   %rax, %r14     <-------  **EXC_BAD_ACCESS here**

This is the log:

...
SyncVerbose: CBLRestPuller[https://xxx-data.yyy.com/customer_1]: postProgressChanged (0/14012, active=1 (batch=11, net=13), online=1)
SyncVerbose: CBLRestPuller[https://xxx-data.yyy.com/customer_1]: Received #17203 {4c1585d087fa540a835a2f01dc10a83e #2-dc8a73bc3cc8f88d099ac717b940e33c}
SyncVerbose: CBLRestPuller[https://xxx-data.yyy.com/customer_1]: postProgressChanged (0/14013, active=1 (batch=12, net=13), online=1)
SyncVerbose: CBLRestPuller[https://xxx-data.yyy.com/customer_1]: Received #17204 {4c1585d087fa540a835a2f01dc10abb8 #2-7e827d84beaa0d0ff2d3e515e95d56e6}
Sync: CBLReplication[from https://xxx-data.yyy.com/customer_1]: active, progress = 0 / 14013, err: (null)
Sync: CBLRestPuller[https://xxx-data.yyy.com/customer_1]: Caught up with changes!
SyncVerbose: CBLSocketChangeTracker[0x7ff8ebef6c40 customer_1]: POST //xxx-data.yyy.com/customer_1/_changes
SyncVerbose: CBLSocketChangeTracker[0x7ff8ebef6c40 customer_1]: POST //xxx-data.yyy.com/customer_1/_changes
RemoteRequest: CBLMultipartDownloader[/customer_1/44b696191a87d4235373fb5c8601b4a0]: Got response, status 401
CBLMultipartDownloader[/customer_1/44b696191a87d4235373fb5c8601b4a0]: Got error Error Domain=CBLHTTP Code=401 "401 unauthorized" UserInfo={NSURL=https://xxx-data.yyy.com/customer_1/44b696191a87d4235373fb5c8601b4a0?rev=2-4a660877547983a42de8255ff1f40376&revs=true&attachments=true, NSLocalizedFailureReason=unauthorized, NSLocalizedDescription=401 unauthorized}
Sync: CBLRestPuller[https://xxx-data.yyy.com/customer_1] Progress: set error = 401 unauthorized
Sync: CBLRestPuller[https://xxx-data.yyy.com/customer_1] STOPPING...
Sync: Stopping 12 remote requests
RemoteRequest: CBLMultipartDownloader[/customer_1/44b017bde7bf383ff0abb49cd3e3f6e7]: Stopped
RemoteRequest: CBLMultipartDownloader[/customer_1/44b696191a87d4235373fb5c86216c16]: Stopped
RemoteRequest: CBLMultipartDownloader[/customer_1/44b696191a87d4235373fb5c863fb8f8]: Stopped
RemoteRequest: CBLMultipartDownloader[/customer_1/44b696191a87d4235373fb5c865f07cd]: Stopped
RemoteRequest: CBLMultipartDownloader[/customer_1/44b696191a87d4235373fb5c867ed61d]: Stopped
RemoteRequest: CBLMultipartDownloader[/customer_1/44b696191a87d4235373fb5c869e434b]: Stopped
RemoteRequest: CBLMultipartDownloader[/customer_1/44b696191a87d4235373fb5c86bde017]: Stopped
RemoteRequest: CBLMultipartDownloader[/customer_1/44b696191a87d4235373fb5c86dc3bbe]: Stopped
RemoteRequest: CBLMultipartDownloader[/customer_1/44b696191a87d4235373fb5c86fb82ff]: Stopped
RemoteRequest: CBLMultipartDownloader[/customer_1/4508dcc231f1ca69d5b33c0201170462]: Stopped
RemoteRequest: CBLMultipartDownloader[/customer_1/4508dcc231f1ca69d5b33c02013736cd]: Stopped
Sync: CBLRestPuller[https://xxx-data.yyy.com/customer_1] Progress: set active = 0
SyncVerbose: CBLRestPuller[https://xxx-data.yyy.com/customer_1]: postProgressChanged (0/14013, active=0 (batch=0, net=0), online=1)
Sync: CBLRestPuller[https://xxx-data.yyy.com/customer_1] STOPPED
Replication: CBLRestPuller[https://xxx-data.yyy.com/customer_1] took 12.635 sec; error=Error Domain=CBLHTTP Code=401 "401 unauthorized" UserInfo={NSURL=https://xxx-data.yyy.com/customer_1/44b696191a87d4235373fb5c8601b4a0?rev=2-4a660877547983a42de8255ff1f40376&revs=true&attachments=true, NSLocalizedFailureReason=unauthorized, NSLocalizedDescription=401 unauthorized}

---

• Version: 1.2
• Client OS: 9.2.1
• Server: CouchDB 1.6.1

[snej]

This is similar to #1015, but in this case it's clearly the CBLRemoteRequest reference that's bad. I think this is actually the same bug, but this time it got a little farther because the (deleted) block variable storage was still accessible memory, and it was able to read a (bogus) CBLRemoteRequest pointer from it.

Luca, could you try merging in commit e7d17cb, or at least copying in the change to CBLRemoteRequest.m from that commit? Or you could download the latest nightly build.

[snej]

I'm going to close this on the assumption that it's the same bug as #1015. If you still see the problem after getting a fixed build, please comment and I'll reopen it.

[zygoat]

I seem to be observing a similar (if not same) problem, which unfortunately is not fixed by e7d17cb.

We have a continuous replication running on iOS. Simply making ANY change remotely will immediately crash the iOS app in this way as soon as the replication attempts to pull down the changes. (An assertion is thrown in -[CBLRestReplicator asyncTasksFinished:] as a result of _asyncTaskCount being decremented below zero.)

It appears as though the NSURLConnection received in the delegate callback in frame 10 has come back with a 401 (unauthorized) despite the request including a SyncGatewaySession cookie. I don't know why that's failing (which the main problem), but it also appears the failure is mis-handled.

Rolling back to CBL 1.1.1 fixes the problem, while 1.2, e7d17cb and current HEAD all seem to be afflicted.

[zygoat]

It seems that this is related to continuous replication. Switching to one-shot appears to avoid the crash, and replication completes as expected.