Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add trailing slash to AIP recovery destination #740

Merged
merged 8 commits into from
Sep 23, 2024

Conversation

replaceafill
Copy link
Member

This also adds an integration test for AIP recovery of compressed and uncompressed AIPs stored in a local file system space.

Connected to archivematica/Issues#420

@replaceafill replaceafill requested a review from a team September 13, 2024 19:27
@Dhwaniartefact
Copy link
Contributor

I tried producing this issue and ended up with the error in storage service logs.

Steps to reproduce

  1. I gave below command which responded json values as {"message": "Recover request created successfully.", "id": 1}
curl \
    --location \
    --request POST \
    --header 'Authorization: ApiKey test:test' \
    --header 'Content-Type: application/json' \
    --data-raw '{
    "event_reason": "testing purposes",
    "pipeline": "a1aac099-d3bd-48ca-93ce-a7648c25a123",
    "user_id": "1",
    "user_email": "test@test.com"
    }' \
        'http://localhost:62081/api/v2/file/85f58e8f-e84f-4e08-8b5a-6c7dc5b14328/recover_aip/'
{"message": "Recover request created successfully.", "id": 1}
  1. I checkout in this branch, rebuild the containers and restart the services.
  2. I then approve the AIP recovery from the storage service UI for the same existing AIP.
  3. I checked the storage service logs and found the following error.

Logs

archivematica-storage-service-1  | ERROR     2024-09-17 10:22:30  django.request:log:log_response:241:  Internal Server Error: /packages/aip_recover_request
archivematica-storage-service-1  | Traceback (most recent call last):
archivematica-storage-service-1  |   File "/src/storage_service/locations/views.py", line 172, in execution_logic
archivematica-storage-service-1  |     (success, _, message) = aip.recover_aip(
archivematica-storage-service-1  |   File "/src/storage_service/locations/models/package.py", line 550, in recover_aip
archivematica-storage-service-1  |     (success, failures, message, __) = temp_aip.check_fixity(force_local=True)
archivematica-storage-service-1  |   File "/src/storage_service/locations/models/package.py", line 1983, in check_fixity
archivematica-storage-service-1  |     path = self.fetch_local_path()
archivematica-storage-service-1  |   File "/src/storage_service/locations/models/package.py", line 366, in fetch_local_path
archivematica-storage-service-1  |     self.current_location.space.move_to_storage_service(
archivematica-storage-service-1  |   File "/src/storage_service/locations/models/space.py", line 334, in move_to_storage_service
archivematica-storage-service-1  |     self.get_child_space().move_to_storage_service(
archivematica-storage-service-1  |   File "/src/storage_service/locations/models/local_filesystem.py", line 34, in move_to_storage_service
archivematica-storage-service-1  |     return self.space.move_rsync(src_path, dest_path)
archivematica-storage-service-1  |   File "/src/storage_service/locations/models/space.py", line 572, in move_rsync
archivematica-storage-service-1  |     raise StorageException(s)
archivematica-storage-service-1  | locations.models.StorageException: Rsync failed with status 23: b'sending incremental file list\nrsync: [sender] link_stat "/var/archivematica/storage_service/recover/2024-09-12_093910-85f58e8f-e84f-4e08-8b5a-6c7dc5b14328.7z" failed: No such file or directory (2)\ndelta-transmission disabled for local transfer or --whole-file\ntotal: matches=0  hash_hits=0  false_alarms=0 data=0\n\nsent 19 bytes  received 79 bytes  196.00 bytes/sec\ntotal size is 0  speedup is 0.00\nrsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.2.7]\n'
archivematica-storage-service-1  | 
archivematica-storage-service-1  | During handling of the above exception, another exception occurred:
archivematica-storage-service-1  | 
archivematica-storage-service-1  | Traceback (most recent call last):
archivematica-storage-service-1  |   File "/pyenv/data/versions/3.9.20/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
archivematica-storage-service-1  |     response = get_response(request)
archivematica-storage-service-1  |   File "/pyenv/data/versions/3.9.20/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
archivematica-storage-service-1  |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
archivematica-storage-service-1  |   File "/pyenv/data/versions/3.9.20/lib/python3.9/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapper_view
archivematica-storage-service-1  |     return view_func(request, *args, **kwargs)
archivematica-storage-service-1  |   File "/src/storage_service/locations/views.py", line 194, in aip_recover_request
archivematica-storage-service-1  |     return _handle_package_request(request, config, "locations:aip_recover_request")
archivematica-storage-service-1  |   File "/src/storage_service/locations/views.py", line 287, in _handle_package_request
archivematica-storage-service-1  |     success, err_msg = config.execution_logic(event.package)
archivematica-storage-service-1  |   File "/src/storage_service/locations/views.py", line 179, in execution_logic
archivematica-storage-service-1  |     message = _("error accessing restore files at %(path)s") % {
archivematica-storage-service-1  | UnboundLocalError: local variable '_' referenced before assignment

@replaceafill
Copy link
Member Author

replaceafill commented Sep 17, 2024

Thanks for reporting this @Dhwaniartefact. I can see how the new tests are not testing the "not happy" paths checking for errors. I'll add a new test to cover that.

However, I'm curious on how you got to that situation and I'm wondering if it's worth replicating a new scenario to catch additional problems. Could you please describe how you set up the recovery copy before requesting the recovery?

@replaceafill
Copy link
Member Author

@Dhwaniartefact I just added a test and a fix for the problem you found today. This workflow run shows the local variable '_' referenced before assignment error under the Run tests steps:

Captura desde 2024-09-17 16-28-19

Then the next commit fixes it.

@Dhwaniartefact
Copy link
Contributor

@Dhwaniartefact I just added a test and a fix for the problem you found today. This workflow run shows the local variable '_' referenced before assignment error under the Run tests steps:

Thanks for fixing this. This error produces if you create recovery request on uncorrupted AIP. The change is working as expected.

LGTM!

@replaceafill replaceafill merged commit 074d2b2 into qa/0.x Sep 23, 2024
9 checks passed
@replaceafill replaceafill deleted the dev/issue-420-aip-recovery branch September 23, 2024 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants