Rethink the "database"/data structure

[johanneszab]

Currently, for each blog there are several files saved in a folder called \Index that contain all the relevant blog information and settings used by TumblThree as well as the downloaders state.

Initially I don't wanted to have the overhead of a real database and for a simple application this seem sufficient. But maybe there is a more convenient way to handle this which also offers transactionality. Currently those files (as well as the settings) are quite seldom updated/saved (i.e. settings: Only if the application is shutting down; blog: if the download is finished or canceled). Maybe there is a way to improve the download state consistency in case of failures.

[elipriaulx]

It would be great while considering this problem, to consider the format of other blogs and content sites that might be nice to cater for in the future, and also the possibility of abstracting a library to read a database out into a nuget package or something so that it can be more easily processed further. There was mention of generating a Tumblr like site offline of content in an issue somewhere; if the format was easy for others to take and act upon, then this sort of thing would become really easy.

[thomas694]

In 617b2d2 the database files of the currently crawled blogs are saved every two minutes to prevent loss of all the filenames of the files which have been downloaded in this session. After a crash these files would have been downloaded again, because their filenames are not in the database.