[BUG] Unstable behaviour when inserting from multiple threads

[domibies]

Version : LiteDB v5.0.3, tested on both Windows & Macos, both show unstable behaviour, but expose different errors

When inserting (a lot) from different threads, after a while, the operations on the Db lock (timeout after 1:00), mostly on Windows, or crash (null reference exception), mostly on MacOS
(edit : now crashes also on windows, as on Mac OS, see *note below)

I have witnessed the behaviour in a more complex scenario involving receiving messages multithreaded from Azure service bus, but I could reproduce it as follows.

I have written a small program that shows this behaviour. On windows, this programs shows some stats in the console for a while (as little as 30 seconds, or sometimes several minutes), until all threads lock on any insert or delete action, for one full minut, and then the code throws an exception.

you can find the program here :
https://github.com/domibies/litedb-mt-test

On MacOS, it crashes after a while.

I would expect the process to stay stable in this scenario.

a stack trace from macOS :
Unhandled exception. Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
at LiteDB.Engine.TransactionMonitor.b__15_0(UInt32 id)
at LiteDB.Engine.TransactionService.Commit()
at LiteDB.Engine.LiteEngine.AutoTransaction[T](Func2 fn) at LiteDB.LiteCollection1.Insert(T entity)

(*note : Hmm, weird, I just cloned the above repository on a windows machine, and now I'm getting the same behaviour on Windows as on MacOS (the NullReferenceException). I'm pretty sure that exactly the same source code (but project initially created on windows) showed other behaviour on windows (hanging threads, followed by a timeout exception "Timeout.DataBase is locked for more than 00:01:00. ...."))

[JensSchadron]

@domibies I've just cloned your repo and let it run on my machine as well, and I got the NullReference as well. The main question is now, what exactly is causing it 😓

[domibies]

Ok, try this : set DeleteEveryXSeconds to 10 instead of 3, and then (on windows) you see the behaviour I was first getting. First the inserting threads block on their db operations (screenshot below), and then they throw on EnterTransaction()

Tried the same on macOS, and after more than 10 minutes, it's still running...

LiteDB.LiteException
HResult=0x80131500
Message=Database lock timeout when entering in transaction mode after 00:01:00
Source=LiteDB
StackTrace:
at LiteDB.Engine.LockService.EnterTransaction()
at LiteDB.Engine.TransactionMonitor.GetTransaction(Boolean create, Boolean& isNew)
at LiteDB.Engine.LiteEngine.AutoTransaction[T](Func2 fn) at LiteDB.LiteCollection1.Insert(T entity)
at LiteDb.MT.Test.Program.InsertRecords(CancellationToken cancelToken) in C:\Users\dominique\Source\Repos\domibies\litedb-mt-test\Program.cs:line 124
at LiteDb.MT.Test.Program.<>c__DisplayClass18_0.b__0() in C:\Users\dominique\Source\Repos\domibies\litedb-mt-test\Program.cs:line 107
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)

[domibies]

Tried the same on macOS, and after more than 10 minutes, it's still running...

crashed 5 minutes later (null reference exception)

[lbnascimento]

@domibies Could you test with the current master? The transaction model was entirely rewritten.

[domibies]

@lbnascimento With the current master it does seem to be stable running for more than 1h almost 25M inserts & deletes from 32 threads. The 'auto checkpoint' behaviour seems to be gone? If I don't call LiteDataBase.Checkpoint() explicitly the *.db stays small, and the *-log.db keeps growing. Regulary calling CheckPoint() works fine.