RangeError:Maximum call stack size exceeded

[lincolnthree]

Possibly related to #589 since I do use Ionic & Angular:

"dexie": "^3.1.0-alpha.9",

I don't have a whole lot of info on this since the stack trace is not very descriptive unfortunately. I saw it once and not since.

[dfahlander]

The code that gets into this error is the new rangeset implementation used for live queries in dexie 3.1.0-alpha. I don't think it's related to anything else than the rangeset code itself. I hope it's a bug that could be tracked down if we found a way to reproduce it. The code is rangeset.ts.

[lincolnthree]

Yeah, if I find a way to reproduce it I'll definitely pass it along. Sorry again I don't have more for you :( But figured you'd want to know about it just in case.

[staff0rd]

I'm getting this multiple times a day during development of this - source here. I ran the app all last night, no error, then today while i was running it, i got the error three or four times. I get the feeling the likelihood of this error occurring increases with activity to/from the db, i'm on ^3.1.0-alpha.8 but started to decrease my usage of useLiveQuery because of this (and some other reasons).

Happy to help isolate it and/or produce a repro if I can - right now the stacktrace is pretty long and vague - I'm not certain even which query it was executing when it occurs...

[dfahlander]

Thanks @staff0rd for sharing code that could hopefully be to nail down the issue. I'm quite confident we will find the issue and solve it eventually. Right now I have limited time to look into it in April/May (until Dexie Cloud beta is out).

Anyone brave enough to dive into this??? My gut feeling is that the issue lies in RangeSet.ts in one of these functions: addRange(), mergeRanges() or rebalance().

• either because of a bug in balancing the tree (so certain branches gets extremely long)
• or if the tree can end up in a state where nodes references each other circular.

Since @staff0rd's app may decrease the use of useLiveQuery(), I assume we need a historical commit instead of using the latest. Does @staff0rd have a commit to recommend (where you were still making heavy use of useLiveQuery())?

[staff0rd]

I am still getting it now on the latest commit, tho, it may be difficult for others to use that code to produce it - the app runs a bunch of statemachines in the background which also make calls to dexie as they need to. I'll see if I can dive into it a bit on my own, tho i'm new to dexie.

The latest time it threw (a few minutes ago) has only a single reference to useLiveQuery:

const goods = useLiveQuery(() => db.markets.orderBy("good").uniqueKeys());

I'll see if I can catch it before it throws and inspect the branches.

[jordoh]

As another data point: I'm seeing this in 3.1.0-alpha.9 as well (after upgrading from 3.0.x), using the undocumented "txcommitted" event instead of liveQuery itself. Unfortunately I've been unable to reproduce it on demand, and it's pretty intermittent in the wild. It has happened in Chrome and Safari, using indexeddbshim - I'm not sure if that makes an issue with indexedDB.cmp a more, or less, likely culprit.

[lincolnthree]

The only calls I'm using are:

export class CardDatabase extends Dexie {
...
constructor() {
  super()
  ...
  this.version(CardDatabase.LATEST_VERSION).stores({
        cards: '&id,collector_number,name,oracle_id,set_code,[set_code+collector_number],queried'
  });
  ...

this.cards = this.table('cards');

database.cards.bulkPut(input)
database.cards.where(input).first()
database.cards.where(input.property).equals(input.id).first()
database.cards.where(input.property).anyOf(input.ids).and(()=>...).toArray()

[staff0rd]

Of course, the moment I started watching, it stopped throwing. All I have so far is a screenshot of the react dev screen when it failed.

[dfahlander]

Thanks @staff0rd I think I'll try to stress Rangeset with random values to see if I can reproduce.

[dfahlander]

Not able to reproduce it. Trying with your requests and filling db with 100,000 of rows with random ids and index values. I'm wondering whether it could occur only if the key is a mutable key (array or binary key) and is mutated so that the binary tree gets confused? Does any of your indexed properties (id,collector_number,name,oracle_id,set_code,queried) contain mutable keys such as array, ArrayBuffer, UInt8Array etc?

[yulolimum]

Adding more info to this thread to hopefully help out.

Here's the error captured in Sentry with a stacktrace: https://sentry.io/share/issue/f0666217026d4a448adc153d9768595f/

Happened for only one user (out of hundreds) so far on: Chrome 89.0.4389 / Windows 10
dexie: 3.1.0-alpha.8
dexie-react-hooks: 1.06

All indexed keys are either integer or string.

[jordoh]

Only strings and (auto-incremented PK) integers indexed in our application as well. We also have whatever dexie-observable adds (though we're no longer importing dexie-observable).

[lincolnthree]

@dfahlander negative. Just strings and I think queried is a date or Unix timestamp/number

[dfahlander]

I've reproduced it in PR #1290. Added tests for RangeSet (had them before in another repo) and also found a set of ranges that reproduces this issue. Had to create 10,000 sets with 20 random ranges each to find one or two crashing combinations. One of those are defined in a repro test. It generates a circular tree for some reason. Will try fixing it soon.

[dfahlander]

an issue in rebalance made the tree corrupt in certain conditions. Will release new version soon and looking forward for feedback from you all 😉

[dfahlander]

Except from crashing, this bug did also lead to infinite loops when traversing a currupt and circular referencing tree.

[staff0rd]

i really appreciate the tree-rotation ascii diagram

[lincolnthree]

@dfahlander Out of curiosity, could this have resulted an error something like the following?

Failed to put in database: Object could not be cloned.? Failed serialization usually happens with self-referential object structures. Sounds like that may have been happening?

I see this error sometimes as well, but don't currently have a trace available.

[dfahlander]

@dfahlander Out of curiosity, could this have resulted an error something like the following?

Failed to put in database: Object could not be cloned.? Failed serialization usually happens with self-referential object structures. Sounds like that may have been happening?

I see this error sometimes as well, but don't currently have a trace available.

No. That error is a result of having non-clonable data in the object to store. Functions, promises and some other types must not be part of the object being stored. This limitation lies in the structured cloning algorithm implemented in IndexedDB.

[lincolnthree]

@dfahlander That makes sense. I've been somewhat befuddled about this error since all of this data should be pure JSON. So I'm not really certain where these other data-types are coming from. I'm actually wondering if Zone.js is patching the data erroneously somehow.

[dfahlander]

Dexie 3.1.0-alpha.10 is released just now.

If you'd have a chance to give me a comment in this issue after having upgraded and run it for a while, that would be terrific! :)

Thanks!

[lincolnthree]

Looks like this might have taken care of it. Numbers appear to be headed in the right direction:

No reports from the latest build. Continuing to monitor.

[lincolnthree]

Still no new reports, over 4,000 sessions.

[staff0rd]

I've been using Dexie 3.1.0-alpha.10 since yesterday, ran it over night, have not seen a single crash since updating. lgtm!