[BUG] Multi-process application locking database file

[hillsydev]

LiteDB 5.0.3, .NET 4.6, Winfows 10 1809 64bit.

We have a single EXE .NET application which can be run as a windows service and as standard UI Winforms application. The purpose of the application is to run various tasks such as pulling and pushing data from API's and updating SQL server databases. These tasks are run within a Task thread within the application. The application contains a 'Settings' class which stores various strings and values for the application. The application also contains a 'Flags' class which stores the running state of the application, such as when the service starts the 'Running' flag gets set to 'True'. This 'Running' flag can be read from within the UI layer to indicate to an administrator that the task processes are running. Both classes are persisted to the same LiteDB file using a connection string in shared mode with password. When the service and the UI instances access the LiteDB file at the same time, a file lock will occur while which results in a file lock error.

Have tried adding a static lock object around the method to prevent any threads within the instance accessing the same method, but this does not work either.

The error is occurring in the following method:

Code to Reproduce

public void LoadSettings() {

            lock (_settingsDBLock) {

                try {

                    using (var db = new LiteDatabase(GetLiteDBSettingsConnectionString())) {

                        var col = db.GetCollection<AppSettings>(typeof(AppSettings).Name);

                        if (col != null) {

                            Settings = col.FindOne(Query.All(Query.Descending));

                        }

                    }

                    // make sure we have a Settings object instantiated 
                    if (Settings == null) {
                        Settings = new AppSettings();
                    }

                }
                catch (Exception ex) {

                    if (Globals.Core.Logging != null) {
                        Globals.Core.Logging.AddErrorLogEntry("Error in AppCore.LoadSettings.", ex);
                    }
                }
            }
        }

Expected behavior
After reading this documentation: https://github.com/mbdavid/LiteDB/wiki/Concurrency
... it mentions that LiteDB is thread and process safe, so I expected that multiple applications could access the same LiteDB file at the same time without file locking.

Screenshots/Stacktrace

Error in AppCore.LoadSettings. at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) at LiteDB.Engine.FileStreamFactory.GetStream(Boolean canWrite, Boolean sequencial) at LiteDB.Engine.StreamPool.<>c__DisplayClass3_0.<.ctor>b__0() at System.Lazy1.CreateValue() at System.Lazy1.LazyInitValue() at System.Lazy1.get_Value() at LiteDB.Engine.DiskService..ctor(EngineSettings settings, Int32 memorySegmentSize) at LiteDB.Engine.LiteEngine..ctor(EngineSettings settings) at LiteDB.SharedEngine.OpenDatabase() at LiteDB.SharedEngine.Query(String collection, Query query) at LiteDB.LiteQueryable1.d__26.MoveNext() at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext() at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable1 source) at LiteDB.LiteCollection`1.FindOne(Query query)

Additional context

[maxkatz6]

Hi @hillsydev
For multiprocess access you need to create connection to your database file with Shared type like below. Or, if you need write access from one process and read access from other, you can leave Direct connection type, which is by default, and make connection readonly where you need it.

var database = new LiteDatabase(new ConnectionString
{
    Filename = fileName,
    Connection = ConnectionType.Shared,
    ReadOnly = isReadOnly
});

[hillsydev]

Hi @maxkatz6

Yes, I included the 'shared' switch in the connection string but still getting locks. I suspect it's my application architecture trying to run the same EXE as a UI and as a service which is causing the lock.

[lbnascimento]

@hillsydev Are both processes running in the same machine?

[hillsydev]

@lbnascimento yes, it’s a single EXE, with a service process added. When exe launches in normal mode, the UI appears. When the same exe launches with a command line switch /servicemode it runs as a service. I’ve split the application now so the UI is in one exe and the service runs from another exe. Appears to be stable.

[lbnascimento]

@hillsydev Could you test with the current master? The transaction model was entirely rewritten.

[hillsydev]

Hi @lbnascimento, thank you for doing that re-write on the transaction model. Unfortunately it hasn't resolved the problem. I don't think it's a problem with LiteDB, I think it may have something to do with the way Windows handles processes running from the same EXE file because the other day when I found this problem, I tried on SQL server using the same application and it had locks as well. Since then, i've split the application into separate EXE files, one for the UI and one for the service and this works as expected now without locking.

In saying that, i've mocked up a test app here that contains the UI and the service in the same app which results in the same error. If you would like to investigate this further then let me know and I'll be happy to post the VS app in here so that you can replicate the error on your side.