Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File conflicts when exporting multiple newspaper processes #6427

Open
matthias-ronge opened this issue Feb 18, 2025 · 1 comment
Open

File conflicts when exporting multiple newspaper processes #6427

matthias-ronge opened this issue Feb 18, 2025 · 1 comment
Labels
bug export Export mappings and configurations

Comments

@matthias-ronge
Copy link
Collaborator

Describe the bug
If several newspaper processes are exported from the process list (for example a whole year), the year processes are also exported for each issue. But several threads try to delete the existing parent process to replace it, but cannot find it anymore because it has already been deleted by another thread, and then abort.

To Reproduce

  1. Select a larger amount of newspaper processes and then execute: export to dms
  2. Crashes can be found in the task manager, and log file
org.apache.commons.io.IOExceptionList: /usr/local/kitodo/contents/Zeitung-1920
    at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:331)
    at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1192)
    at org.kitodo.filemanagement.FileManagement.delete(FileManagement.java:141)
    at org.kitodo.production.services.file.FileService.delete(FileService.java:483)
    at org.kitodo.export.ExportDms.prepareExportLocation(ExportDms.java:230)
    at org.kitodo.export.ExportDms.startExport(ExportDms.java:207)
    at org.kitodo.export.ExportDms.startExport(ExportDms.java:159)
    at org.kitodo.production.helper.tasks.ExportDmsTask.run(ExportDmsTask.java:78)
Caused by: java.io.FileNotFoundException: File does not exist: /usr/local/kitodo/contents/Zeitung-1920/Zeitung-1920.xml
    at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1349)
    at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:324)
    ... 7 more

Expected behavior

  • When deleting, a file that has already been deleted is ignored, or
  • exports of the same (parent) process are not done concurrently, or
  • it would also be sufficient to export the parent process once to all children.
@matthias-ronge matthias-ronge added bug export Export mappings and configurations labels Feb 18, 2025
@henning-gerhardt
Copy link
Collaborator

A workaround for this is to limit the parallel execution of tasks inside the taskmanager to one (sequentiell work). As we use this since beginning (even in 2.x) I forgot to open a bug report as this issue still exists in 3.x :-(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug export Export mappings and configurations
Projects
None yet
Development

No branches or pull requests

2 participants