Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.HeaderPage'

[lidanger]

This came again, 10 minutes after power on.

System.InvalidCastException: Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.HeaderPage'.
at LiteDB.PageService.GetPage[T](UInt32 pageID)
at LiteDB.TransactionService.PersistDirtyPages()
at LiteDB.LiteEngine.Transaction[T](String collection, Boolean addIfNotExists, Func2 action) at LiteDB.LiteEngine.Insert(String collection, IEnumerable1 docs, BsonType autoId)
at LiteDB.LiteEngine.Insert(String collection, BsonDocument doc, BsonType autoId)
at LiteDB.LiteCollection`1.Insert(T document)
at scsoft.wrySite.Acquisition.DaqManager.???
??????()

LiteDB version:4.1.1.0
OS:Windows 7
.NET Framework:4.0

[lidanger]

Now I can copy data from broken data file to a new file to handle the problem.But I think it's necessary to fix it thoroughly.

[mbdavid]

What did you mean you can copy data from broken file to new file? After power on, first connection must run recovery.....

But, last time 4.1.1 I think that I back file write again to default write..... because performance issues (in HDD disks)... I think I will use this as a connection string parameter... can be some difference in OS or HDD/SSD

[lidanger]

The recovery process is not very thoroughly, some collection structure is still broken.So I have to use LiteDB to copy data of collections to another file, then the new file can be used normally.
I think so, too. With 4.1.0, it seems OK on storage.Maybe it's a good potion to add a new connection string parameter.

[mbdavid]

Hi @lidanger, parameter added: "flush: true" will use FileStream.Flush(true) as used in 4.0 - will slow down in HDD disks but avoid this king of problem in power failure

Is in master branch, if you can test I appreciate. Will release soon on nuget

[lidanger]

Sorry, I went out with a business trip and I saw your words just today.

I tested it just now on win 7 in a virtual machine.It seems there is no effect using litedb with option flush:true .
@mbdavid

.net framework 4.0

Read - 读取数据出错System.InvalidCastException: Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.IndexPage'.
at LiteDB.PageService.GetPage[T](UInt32 pageID)
at LiteDB.IndexService.GetNode(PageAddress address)
at LiteDB.IndexService.d__13.MoveNext()
at LiteDB.QueryCursor.Fetch(TransactionService trans, DataService data, BsonReader bsonReader)
at LiteDB.LiteEngine.d__9.MoveNext()
at LiteDB.LiteEngine.d__13.MoveNext()
at LiteDB.LiteCollection1.<Find>d__17.MoveNext() at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable`1 collection)
at Test.Program.readThread() in E:\LiteDB\Test\Program.cs:line 93

[lidanger]

test code:

using LiteDB;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;

namespace Test
{
class TestItem
{
public string Id { get; set; }
public string Name { get; set; }
public DateTime Time { get; set; }
public double Value { get; set; }

    public TestItem()
    {
        Id = Guid.NewGuid().ToString("N");
    }
}

class Program
{
    static LiteDatabase _db;

    static void Main(string[] args)
    {
        log4net.Config.XmlConfigurator.Configure();

        _db = new LiteDatabase(Properties.Settings.Default.ConnectionString);

        Thread thread = null;

        for (int i = 0; i < Properties.Settings.Default.WriteThread; i++)
        {
            thread = new Thread(writeThread);
            thread.Start();
        }

        for (int i = 0; i < Properties.Settings.Default.ReadThread; i++)
        {
            thread = new Thread(readThread);
            thread.Start();
        }

        while (true)
        {
            Thread.Sleep(1000);
        }
    }

    static void writeThread()
    {
        var logger = log4net.LogManager.GetLogger("Write");
        var rand = new Random(DateTime.Now.Millisecond);

        while (true)
        {
            var item = new TestItem
            {
                Name = "adb",
                Time = DateTime.Now,
                Value = rand.NextDouble() * 1000
            };
            try
            {
                var col = _db.GetCollection<TestItem>();
                col.Insert(item);
            }
            catch (Exception ex)
            {
                logger.DebugFormat("写入数据出错{0}", ex);
            }

            Thread.Sleep(Properties.Settings.Default.WriteInterval * 1000);
        }
    }

    static void readThread()
    {
        var logger = log4net.LogManager.GetLogger("Read");
        var lst = new List<TestItem>();

        while (true)
        {
            try
            {
                var col = _db.GetCollection<TestItem>();
                var items = col.Find(Query.GT("Value", 800));

                lst.Clear();
                lst.AddRange(items);
            }
            catch(Exception ex)
            {
                logger.DebugFormat("读取数据出错{0}", ex);

                Thread.Sleep(1000);
                continue;
            }

            foreach (var item in lst)
            {
                Console.WriteLine("{0}\t{1}\t{2}", item.Name, item.Time, item.Value);
            }

            Thread.Sleep(Properties.Settings.Default.ReadThread * 1000);
        }
    }
}

}

[lidanger]

config:

    <Test.Properties.Settings>
        <setting name="ConnectionString" serializeAs="String">
            <value>Filename=db.dat;Flush=true;</value>
        </setting>
        <setting name="WriteThread" serializeAs="String">
            <value>10</value>
        </setting>
        <setting name="ReadThread" serializeAs="String">
            <value>5</value>
        </setting>
        <setting name="WriteInterval" serializeAs="String">
            <value>5</value>
        </setting>
        <setting name="ReadInterval" serializeAs="String">
            <value>10</value>
        </setting>
    </Test.Properties.Settings>

[Youwannadie]

Hello @mbdavid , i think there are still concurrency problems like were in version 3.x.x.
I am using version 4.1.4 and still got exceptions:
LiteDB.LiteException - Cannot insert duplicate key in unique index '_id'. The duplicate value is '3929'
System.InvalidCastException - Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.IndexPage'
I have a singleton where LiteDatabase instance is created with Exclusive mode. That singleton is shared in few Timers (System.Timers.Timer) so in few threads.

[chenhuang444]

I too believe there might still be problems in 4.1.4. Our logs indicate around 2000 crashes from 150 users.

InvalidCastException: Unable to cast object of type “LiteDB.EmptyPage” to type “LiteDB.HeaderPage”。
  Module "LiteDB.PageService", line 90, col 0, in GetPage
    T GetPage[T](UInt32)
  Module "LiteDB.PageService", line 3, col 0, in DeletePage
    Void DeletePage(UInt32, Boolean)
  Module "LiteDB.DataService", line 105, col 0, in Delete
    LiteDB.DataBlock Delete(LiteDB.CollectionPage, LiteDB.PageAddress)
  Module "LiteDB.LiteEngine+<>c__DisplayClass8_0", line 365, col 0, in Delete { <lambda> }
    Int32 <Delete>b__0(LiteDB.CollectionPage)
  Module "LiteDB.LiteEngine", line 78, col 0, in Transaction
    T Transaction[T](System.String, Boolean, System.Func`2[LiteDB.CollectionPage,T])
  Module "LiteDB.LiteEngine", line 90, col 0, in Delete
    Int32 Delete(System.String, LiteDB.Query)
  Module "LiteDB.LiteCollection`1", line 0, col 0, in Delete
    Int32 Delete(LiteDB.Query)
  Module "Tutor.Logic.Storage.DocumentDb", line 19, col 0, in DeleteAll
    Int32 DeleteAll[T](System.Nullable`1[System.Int32])
  Module "Tutor.Logic.Frog.FrogHelper", line 59, col 0, in PostFrog
    Void PostFrog()
  Module "Tutor.Student.App", line 299, col 0, in OnStartup { <lambda> }
    Void <OnStartup>b__9_1()
  Module "System.Threading.Tasks.Task", line 15, col 0, in InnerInvoke
    Void InnerInvoke()
  Module "System.Threading.Tasks.Task", line 16, col 0, in Execute
    Void Execute()

InvalidCastException: Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.DataPage'.
  Module "LiteDB.PageService", line 90, col 0, in GetPage
    T GetPage[T](UInt32)
  Module "LiteDB.QueryCursor", line 119, col 0, in Fetch
    Void Fetch(LiteDB.TransactionService, LiteDB.DataService, LiteDB.BsonReader)
  Module "LiteDB.LiteEngine", line 282, col 0, in Find
    Boolean MoveNext()
  Module "LiteDB.LiteEngine", line 484, col 0, in Find
    Boolean MoveNext()
  Module "LiteDB.LiteCollection`1", line 186, col 0, in Find
    Boolean MoveNext()
  Module "System.Linq.Lookup`2", line 91, col 0, in Create
    System.Linq.Lookup`2[TKey,TElement] Create[TSource](System.Collections.Generic.IEnumerable`1[TSource], System.Func`2[TSource,TKey], System.Func`2[TSource,TElement], System.Collections.Generic.IEqualityComparer`1[TKey])
  Module "System.Linq.GroupedEnumerable`3", line 0, col 0, in GetEnumerator
    System.Collections.Generic.IEnumerator`1[System.Linq.IGrouping`2[TKey,TElement]] GetEnumerator()
  Module "System.Collections.Generic.List`1", line 84, col 0, in .ctor
    Void .ctor(System.Collections.Generic.IEnumerable`1[T])
  Module "System.Linq.Enumerable", line 14, col 0, in ToList
    System.Collections.Generic.List`1[TSource] ToList[TSource](System.Collections.Generic.IEnumerable`1[TSource])
  Module "Tutor.Logic.Frog.FrogHelper", line 59, col 0, in PostFrog
    Void PostFrog()
  Module "Tutor.Student.App", line 299, col 0, in OnStartup { <lambda> }
    Void <OnStartup>b__9_1()
  Module "System.Threading.Tasks.Task", line 15, col 0, in InnerInvoke
    Void InnerInvoke()
  Module "System.Threading.Tasks.Task", line 15, col 0, in Execute
    Void Execute()

We use a single instance of LiteDb for access from different threads.
.NET version 4.0

PostFrog basically collected accumulated logs and uploads it to our database.

public static void PostFrog()
{
      var frogGroups = StorageHelper.Global.GetAll<FrogItem>().GroupBy(f => f.UserId).ToList();
      StorageHelper.Global.DeleteAll<FrogItem>();

      // Upload to server
}

public static void AddFrogItem(string url, int duration = 0, string custom = null)
{
    var item = NewFrogItem();
    item.Url = url;
    item.Duration = duration;
    item.Custom = custom;
    item.SeqId = -1;

    var task = Task.Factory.StartNew(
        () =>
        {
            StorageHelper.Global.PUtOne(item);
            if (PendingFrogCount > FrogUploadThreshold)
            {
                PostFrog();
            }

            FrogTasks.Cleanup();
        });

    FrogTasks.Add(task);
}

We add a few logs and then do a manual PostFrog when our application starts and this is the case where these specific exceptions occur. But I have also seen exceptions happen during the lifetime of the application during periodic log uploads

[lbnascimento]

Hi! With the objective of organizing our issues, we are closing old unsolved issues. Please check the latest version of LiteDB and open a new issue if your problem/question/suggestion still applies. Thanks!