[tvloader] got unknown checksum type SHA512

[chewong]

Unable to parse the following SPDX file with tvloader package:

...
SPDXID: SPDXRef-File-34182904f819889b086a855ee5878f2a2e0b9037
FileChecksum: SHA1: 335029255dbadc3dfeb15c8e432078b1535b90de
FileChecksum: SHA256: 0852707ebbd7e78d71c1eea3fb7ac0348aa22124a8975f787c07217058e0fe20
FileChecksum: SHA512: 2bca030aeb4aff21505267c02230bbc91bea728070fe6b46089eb7c168deeb6e0d40195a307d8e39073f322cba7abaf81a1b92011ca249eba2b8d92aedc10e38
...

Got the following error:

WARN[0000] got unknown checksum type SHA512

[swinslow]

Hi @chewong, just to confirm, were you using the SPDX-2.1 parser (tvloader/parser2v1) or the SPDX-2.2 parser (tvloader/parser2v2)?

SHA512 wasn't a valid optional checksum in SPDX v2.1 (see SPDX 2.1 spec section 4.4.4) but became valid in v2.2 (see SPDX spec section 4.4.4).

That said, looking at the parser2v2 code I can see that it isn't checking for the extended set of optional checksums:

tools-golang/tvloader/parser2v2/parse_file.go

Line 73 in 9813e3e

case spdx.SHA1, spdx.SHA256, spdx.MD5:

So I'll mark this as a bug for the v2.2 parser and will address it. Thanks for flagging!

[chewong]

I am using the SPDX-2.2 parser. Thanks!

[swinslow]

Having now taken a closer look at this (finally):

This is a bug which does need to be addressed, because it is causing valid SPDX 2.2 documents not to be parsed.

However, I am moving it to the 0.4.0 release milestone. Handling the additional checksums is likely going to require reworking the model for Packages, Files and Snippets to have checksums be handled via a algorithm-to-value map -- rather than separate fields for each possible checksum, as in the current model.

This will involve changing the API, as well as reworking the relevant parts of the tag-value, RDF and JSON parsers and savers. I don't want to further delay releasing 0.3.0 with the JSON saver, so I'm going to move this to 0.4.0 and will try to prioritize addressing that shortly after 0.3.0 goes out the door.

[ianling]

@swinslow got this one in #139

[qrdl]

I'm also asking for support for other hash types in the parser.

[lumjjb]

@chewong @qrdl this should be resolved now with #173, can you see it works now?

[qrdl]

@chewong @qrdl this should be resolved now with #173, can you see it works now?

@lumjjb I no longer have access to the project where I faced the issue so I cannot check it, sorry.

[justinabrahms]

sbom.spdx.txt

This text file is an example which has this problem, for your testing purposes. Apologies for it's 3.2mb size.

[kzantow]

@justinabrahms -- is this failing for you with the same error, that SHA512 is invalid? (It definitely should be valid according to the spec.) Which version of the library are you using?

[justinabrahms]

@kzantow For some reason, I was on v0.3.1-0.20221108182156-8a01147e6342. It seems to be fixed in 0.4.0.