extraneous technicalMetadata.xml in 2 Moabs on storage root 2? #1394
Assignees
Labels
moab_remediation
online moab that may need remediation (e.g. missing files, extraneous files, corrupted content)
Milestone
Comments
jmartin-sul
added
checksum validation
moab_remediation
jmartin-sul
changed the title
|
questions for all of these remediations: have the moabs in question been replicated by preservation-service to the cloud? if so, do the archives need to be re-pushed? (yes to the latter, per Julian, but must be done by hand or with the big batch of 504 objects to be remediated as deleting the current cloud entries is tricky and requires Julian/ops) related useful query: https://github.com/sul-dlss/preservation_catalog/tree/master/db#view-the-zip-parts-for-a-given-druid input> druid = 'ab123cd4567'
input> ZipPart.joins(zipped_moab_version: [{ complete_moab: [:preserved_object] }, :zip_endpoint]).where(preserved_objects: { druid: druid }).pluck(:druid, 'current_version AS highest_version', 'zipped_moab_versions.version AS zip_version', :endpoint_name, :status) |
ndushay
changed the title
|
won't technically be in progress till next week when andrew and i pair on it, but marking it "in progress" since we have a meeting on the calendar to deal with this, and want to get the clutter out of the ready column. |
|
remediated [29] pry(main)> Audit::Checksum.validate_druid('rw416bs9212')
I, [2020-03-18T15:56:51.464871 #9849] INFO -- : 2020-03-18T22:56:51Z CV validate_druid starting for rw416bs9212
D, [2020-03-18T15:56:51.474256 #9849] DEBUG -- : Found 1 complete moabs.
I, [2020-03-18T15:56:53.944113 #9849] INFO -- : validate_checksums(rw416bs9212, services-disk02) checksum(s) match
I, [2020-03-18T15:56:53.944328 #9849] INFO -- : validate_checksums(rw416bs9212, services-disk02) CompleteMoab status changed from invalid_checksum to ok
I, [2020-03-18T15:56:54.348066 #9849] INFO -- : [{:moab_checksum_valid=>"checksum(s) match"}, {:cm_status_changed=>"CompleteMoab status changed from invalid_checksum to ok"}] for rw416bs9212
I, [2020-03-18T15:56:54.348206 #9849] INFO -- : 2020-03-18T22:56:54Z CV validate_druid ended for rw416bs9212
=> [#<AuditResults:0x000000000876e348
@actual_version=nil,
@check_name="validate_checksums",
@druid="rw416bs9212",
@log_msg_prefix="validate_checksums(rw416bs9212, services-disk02)",
@moab_storage_root=
#<MoabStorageRoot:0x00000000088499c0
id: 1,
name: "services-disk02",
created_at: Thu, 18 Jan 2018 18:55:35 UTC +00:00,
updated_at: Thu, 18 Jan 2018 18:55:35 UTC +00:00,
storage_location: "/services-disk02/sdr2objects">,
@result_array=
[{:moab_checksum_valid=>"checksum(s) match"},
{:cm_status_changed=>"CompleteMoab status changed from invalid_checksum to ok"}],
@string_prefix="validate_checksums (actual location: services-disk02; )">]
[36] pry(main)> Audit::Checksum.validate_druid('dj352gy7333')
I, [2020-03-18T15:58:44.320148 #9849] INFO -- : 2020-03-18T22:58:44Z CV validate_druid starting for dj352gy7333
D, [2020-03-18T15:58:44.323543 #9849] DEBUG -- : Found 1 complete moabs.
I, [2020-03-18T15:58:46.185690 #9849] INFO -- : validate_checksums(dj352gy7333, services-disk02) checksum(s) match
I, [2020-03-18T15:58:46.185839 #9849] INFO -- : validate_checksums(dj352gy7333, services-disk02) CompleteMoab status changed from invalid_checksum to ok
I, [2020-03-18T15:58:46.531281 #9849] INFO -- : [{:moab_checksum_valid=>"checksum(s) match"}, {:cm_status_changed=>"CompleteMoab status changed from invalid_checksum to ok"}] for dj352gy7333
I, [2020-03-18T15:58:46.531408 #9849] INFO -- : 2020-03-18T22:58:46Z CV validate_druid ended for dj352gy7333
=> [#<AuditResults:0x0000000007fbd5b8
@actual_version=nil,
@check_name="validate_checksums",
@druid="dj352gy7333",
@log_msg_prefix="validate_checksums(dj352gy7333, services-disk02)",
@moab_storage_root=
#<MoabStorageRoot:0x0000000007fbd630
id: 1,
name: "services-disk02",
created_at: Thu, 18 Jan 2018 18:55:35 UTC +00:00,
updated_at: Thu, 18 Jan 2018 18:55:35 UTC +00:00,
storage_location: "/services-disk02/sdr2objects">,
@result_array=
[{:moab_checksum_valid=>"checksum(s) match"},
{:cm_status_changed=>"CompleteMoab status changed from invalid_checksum to ok"}],
@string_prefix="validate_checksums (actual location: services-disk02; )">] |
|
remediated [94] pry(main)> Audit::Checksum.validate_druid('vf275hv8977')
I, [2020-03-18T16:04:42.934837 #9849] INFO -- : 2020-03-18T23:04:42Z CV validate_druid starting for vf275hv8977
D, [2020-03-18T16:04:42.938044 #9849] DEBUG -- : Found 1 complete moabs.
I, [2020-03-18T16:04:44.722880 #9849] INFO -- : validate_checksums(vf275hv8977, services-disk02) checksum(s) match
I, [2020-03-18T16:04:44.723062 #9849] INFO -- : validate_checksums(vf275hv8977, services-disk02) CompleteMoab status changed from invalid_checksum to ok
I, [2020-03-18T16:04:45.087758 #9849] INFO -- : [{:moab_checksum_valid=>"checksum(s) match"}, {:cm_status_changed=>"CompleteMoab status changed from invalid_checksum to ok"}] for vf275hv8977
I, [2020-03-18T16:04:45.087883 #9849] INFO -- : 2020-03-18T23:04:45Z CV validate_druid ended for vf275hv8977
=> [#<AuditResults:0x0000000008050980
@actual_version=nil,
@check_name="validate_checksums",
@druid="vf275hv8977",
@log_msg_prefix="validate_checksums(vf275hv8977, services-disk02)",
@moab_storage_root=
#<MoabStorageRoot:0x0000000008050a70
id: 1,
name: "services-disk02",
created_at: Thu, 18 Jan 2018 18:55:35 UTC +00:00,
updated_at: Thu, 18 Jan 2018 18:55:35 UTC +00:00,
storage_location: "/services-disk02/sdr2objects">,
@result_array=
[{:moab_checksum_valid=>"checksum(s) match"},
{:cm_status_changed=>"CompleteMoab status changed from invalid_checksum to ok"}],
@string_prefix="validate_checksums (actual location: services-disk02; )">] |
|
remediated [99] pry(main)> Audit::Checksum.validate_druid('rc818rv6722')
I, [2020-03-18T16:07:21.155233 #9849] INFO -- : 2020-03-18T23:07:21Z CV validate_druid starting for rc818rv6722
D, [2020-03-18T16:07:21.158995 #9849] DEBUG -- : Found 1 complete moabs.
I, [2020-03-18T16:07:22.303167 #9849] INFO -- : validate_checksums(rc818rv6722, services-disk02) checksum(s) match
I, [2020-03-18T16:07:22.303293 #9849] INFO -- : validate_checksums(rc818rv6722, services-disk02) CompleteMoab status changed from invalid_checksum to ok
I, [2020-03-18T16:07:22.553457 #9849] INFO -- : [{:moab_checksum_valid=>"checksum(s) match"}, {:cm_status_changed=>"CompleteMoab status changed from invalid_checksum to ok"}] for rc818rv6722
I, [2020-03-18T16:07:22.553583 #9849] INFO -- : 2020-03-18T23:07:22Z CV validate_druid ended for rc818rv6722
=> [#<AuditResults:0x00000000086a5010
@actual_version=nil,
@check_name="validate_checksums",
@druid="rc818rv6722",
@log_msg_prefix="validate_checksums(rc818rv6722, services-disk02)",
@moab_storage_root=
#<MoabStorageRoot:0x00000000086a5100
id: 1,
name: "services-disk02",
created_at: Thu, 18 Jan 2018 18:55:35 UTC +00:00,
updated_at: Thu, 18 Jan 2018 18:55:35 UTC +00:00,
storage_location: "/services-disk02/sdr2objects">,
@result_array=
[{:moab_checksum_valid=>"checksum(s) match"},
{:cm_status_changed=>"CompleteMoab status changed from invalid_checksum to ok"}],
@string_prefix="validate_checksums (actual location: services-disk02; )">] |
|
we believe that we've fixed the simple extraneous techMD errors that this ticket captures, and that the couple druids left with techMD files that aren't in sigCat are actually messier partially expunged objects. closing... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
at present there are 506 moabs with
invalid_checksumstatus on storage root 2. 504 seem to fit the profile described in #1393. the other two look similar, but not exactly the same: the timestamps on the files don't fit the same pattern as #1393 , but things otherwise look very similar.differences from #1393:
contentMetadata.xmlwas the late add according to the file system timestamps, but it made it intosignatureCatalog.xmltechnicalMetadata.xmlseems like it was created at the same as the other datastream files, but didn't make it into the moab'ssignatureCatalog.xml.without some context from @andrewjbtw or @blalbrit, i'd be hesitant to suggest that we treat these objects the same as the ones described in #1393, since i don't have a good explanation for how these files got there.
The text was updated successfully, but these errors were encountered: