[BUG] Fail: invalid page position when INSERT/DELETE #1762
Comments
|
@gasblack Could you send me your datafile? |
|
@lbnascimento unfortunately I deleted the database when I tried to replicate the issue |
I have a corrupted database available, where can I send it to you in private? In addition, it seems that it's not the datafile to be corrupted but the log file, in fact, if I delete only the log file all seems to work correctly. |
|
@lbnascimento You could send me both files at lbnascimento@inf.ufrgs.br |
|
@gasblack I have identified the issue with the log file that you sent me by email. In this specific case, all the 15 pages in the log file were from uncommited transactions, so you can safely discard the log file and use the data file as if nothing happended. LiteDB data files are divided in 8192-byte chunks called "pages". For an unencrypted data file, page 0 (the header page) starts at byte 0, but encrypted data files have a "hidden page" before the header contaning the salt used in the encryption and some other stuff. Only the first 64 bytes of this hidden page are used, the rest being filled with zeroes. For whatever reason, this page was written incorrectly in your log file, only the 64 bytes were written, without the padding zeroes. I manually added the padding zeroes to the page, which fixed the log file, but all of its pages were from uncommited transactions anyway, so that's why it's safe to discard the log file. Are there any steps to reproduce this issue consistently, or is it intermittent? What OS and .NET environment were you using when the log file was corrupted? I will work on a fix. |
|
Hi, It is not possible to simply delete the log file, because we can't know beforehand if it contains any data that has not yet stored. |
|
@nicolo-coolshop You are correct in saying that it is not always correct to discard the log file. What I meant is that in his specific situation, the log file (which was corrupted and which I fixed) happened to only have data from uncommited transactions, so the correct thing to do was to discard it. If you send me your data and log files, I could check (and potentially fix) if it's the same type of problem. |
|
I work with @gasblack, so it's the same problem. How can we prevent that from happening? How can we "detect" and "restore" programatically when it happens?
We are using WPF and Xamarin IOS/Android platform happens on all platforms. |
I'm not sure yet, I don't know why this is happening. If you have a consistent series of steps to make this kind of corruption happen, please let me know.
The correct structure for a LiteDB encrypted file (both data or log) is as follows:
What happened in your corrupted log file was that the "Bytes 64-8191" section was ommited, so the data incorrectly started at byte 64. In order to detect and fix such files, you could do something like this: using (var fs = File.Open(filePath, FileMode.Open, FileAccess.ReadWrite))
{
var bytes = new byte[128];
fs.Read(bytes, 0, 128);
if (bytes[0] == 1 && bytes.Skip(17).Take(15).All(x => x == 0)) //making sure that the file is encrypted
{
if (bytes[63] != 0 && bytes[64] != 0) //checking if padding bytes weren't added
{
var fixedBytes = new byte[fs.Length + 8192 - 64];
fs.Position = 0;
fs.Read(fixedBytes, 0, 64);
fs.Read(fixedBytes, 8192, (int)fs.Length - 64);
fs.Position = 0;
fs.Write(fixedBytes, 0, fixedBytes.Length);
}
}
}I tested this solution with your corrupted log file and it fixed the problem. Of course, you're encouraged to do your own testing. Also, there are probably more efficient ways to achieve the same effect, but I believe the general idea is clear. Given that I don't know the root cause of this issue, I don't know if it can also affect data files, but the solution that I proposed should work with data files too. |
|
@gasblack @nicolo-coolshop Could you test with the latest master? I believe it is fixed. |
|
Hi, thank you for the great database. But I need some help. I am also getting the same exception. But not sure why. I tried to open the same db using LiteDB Studio which works fine without any issue. But the application throws exception. I copied the same db to different machine and debugged, but no issue there. Why is it happening on a specific machine. Once the issue come in a machine, until I delete that db, I cannot use it anymore. Is the fix or any option to find the root cause? I see this issue frequently. Please help. I am using the litedb version 5.0.10 from Nuget 2021-10-13 11:09:36.715 +05:30 [ERR] Failed |
|
@rajapc Are you using LITEDB with single process or multiple process are accessing the DB. We are also facing similar issue with LiteDB @mbdavid Can you please help us with this exception. We are observing this very frequently System.Exception: LiteDB ENSURE: position must be in PAGE_SIZE module\r\n |
LiteDB version: 5.0.7
OS: Windows 10 / Android 9 / iOS 13.5.1
.NET: Core 3.1 on windows
Describe the bug
When performing a Delete or an insert in a collection, the operation fails with an "Fail: invalid page position"
It's non systematically but it seems to happen sometimes when the application is closed and restarted like if the database file has been corrupted during the app closure.
The same operations work fine with a new database file.
Stacktrace
Insert operation
Delete operation
The text was updated successfully, but these errors were encountered: