SPDX tag-value parsing fails for CRLF terminated file

[surendrapathak]

The attached file fails to parse with the following error message (please remove .txt before processing)

Error: error processing file

This is likely due to its CRLF line endings. After saving with a sane line ending, parsing succeeds.

It parses successfully with online validator and spdx/tools-python

cbor-sbom.spdx.txt

[neilnaveen]

I was trying to run this, and I encountered this error

go run example_load.go cbor-sbom.spdx.txt

Error while parsing cbor-sbom.spdx.txt: unsupported SPDX version: 'Please refer to the AUTHORS file for contributor/creation details'%  

cc @kzantow

[kzantow]

A quick look makes me think bufio.NewScanner (which is used here) is not handling \r, according to a Stack Overflow question

It's possible the input could be sanitized here or the location in the first link could add a split function as the SO answer example.

[neilnaveen]

A quick look makes me think bufio.NewScanner (which is used here) is not handling \r, according to a Stack Overflow question

It's possible the input could be sanitized here or the location in the first link could add a split function as the SO answer example.

Thanks for the suggestion, I tried to make the change but when I tried to test it, I ran into this error

go run example_load.go cbor-sbom.spdx.txt

Error while parsing cbor-sbom.spdx.txt: unsupported SPDX version: 'Please refer to the AUTHORS file for contributor/creation details'%  

which I had previously ran into, this error is not related to the CRLF. How do I test this change?

cc: @kzantow

[kzantow]

@neilnaveen one way is just do delete a bunch of lines out of that file, just making sure to save it with some pertinent \r\n line ending somewhere. Alternately, you could find a working SPDX TV file and then convert it to DOS line endings

[neilnaveen]

how and where do I load a valid spdx TV file so that I can test this?

[kzantow]

@neilnaveen you can look in the tests, for one: https://github.com/spdx/tools-golang/tree/main/examples/sample-docs/tv

[surendrapathak]

Also, to compare the problematic/good sbom, you can try :

1. Problem ones: https://github.com/shamblett/cbor/releases/tag/5.1.2 (part of zip)
2. Good ones for centos:latest built with syft 0.73.0 : https://sbomlc.s3.amazonaws.com/syft-0.73.0_centos-latest.spdx.tv?AWSAccessKeyId=AKIA2ZBFUJ4NNQGYD5OF&Signature=kvkkSvDCo%2FXt6XiO4nwlDnNCyl0%3D&Expires=1709195015