oscrypto 1.3.0 does not detect openssl version on debian bookworm

[plenaerts]

On debian bookworm the installation of pyHanko to a virtualenv works fine, but running the pyhanko cli results in oscrypto.errors.LibraryNotFoundError: Error detecting the version of libcrypto.

This oscrypto bug exists wbond/oscrypto#78 and has a workaround wbond/oscrypto#78 (comment) which I've used by manually installing oscrypto first and then pip installing pyhanko.

The fix is there in wbond/oscrypto@d5f3437 but the oscrypto maintainer is not releasing the fix on pypi.

I guess pyhanko should either describe the issue and the workaround, or depend on the oscrypto commit d5f3437 according to the workaround above. I'd personally go for the former option. Depending on git versions is bad I think.

For people stumbling into this and who don't want to read up, it's just a pip install git+https://github.com/wbond/oscrypto.git@d5f3437in the environment where you will then pip install 'pyHanko[pkcs11,image-support,opentype,xmp]'.

If you want this documented I'd be happy to help with some text for https://github.com/MatthiasValvekens/pyHanko?tab=readme-ov-file#installing. In that case, don't hesitate to give me some pointers if you have specific requirements for that paragraph ;-)

[MatthiasValvekens]

Yeah, I know about this problem, it's quite annoying.

Don't really have time to deal with a permanent fix right now, but in the current version of pyHanko, oscrypto is only used for two things:

• fetching the system trust list
• one or two crusty old algorithms that are listed in the PDF spec as valid for use in file encryption, but considered too obsolete for inclusion in pyca/cryptography.

The latter is a non-issue for 99% of users, but the former is unfortunately triggered by default when running the CLI without arguments. (IIRC the import is delayed, though, so if you specify a CA file by hand, it should work)

To be honest, at this point I'd prefer forking the functionality of oscrypto that makes sense to keep around (i.e. the platform-independent API to fetch the system trust list, which likely doesn't depend on openssl at all). It's just that oscrypto's packaging process is a bit daunting, and I haven't gotten around to taking a closer look at it yet.

In the meantime, I suppose we can document the fix in the CLI readme. Can you try a test that doesn't use the system trust, just for good measure?

[plenaerts]

I signed using beid and validated. Is this "not using system trust"? My cert is not trusted obviously.

pyhanko sign validate --no-strict-syntax --pretty-print 20250116\ -\ private-stuff.signed.pdf
                                               
2025-01-16 11:54:34,928 - cli - INFO - Strict PDF syntax is disabled; this could impact validation results. Use caution.                                                                       
2025-01-16 11:54:35,065 - pyhanko.sign.validation.generic_cms - WARNING - Failed to build path                                                                                                 
============                                   
Field 1: SIG                                   
============                                   


Signer info                                    
-----------                                    
Certificate subject: "Serial Number: myRRN, Given Name: Pieter etc, Surname: Lenaerts, Common Name: Pieter Lenaerts (Signature), Country: BE"
Certificate SHA1 fingerprint: myfp
Certificate SHA256 fingerprint: again
Trust anchor: "No path to trust anchor found."
The signer's certificate is untrusted.


Integrity                                      
---------                                      
The signature is cryptographically sound.

The digest algorithm used was 'sha256'.
The signature mechanism used was 'sha256_rsa'.


Signing time                                   
------------                                   
Signing time as reported by signer: 2025-01-16T10:18:16+00:00


Modifications                                  
-------------                                  
The signature covers the entire file.


Bottom line                                    
-----------                                    
The signature is judged INVALID.


Error: Validation failed

[MatthiasValvekens]

The BeID government PKI roots are most likely not part of the certificates that are trusted by default on your machine, no... You can download them from certipost if you want, or alternatively I think they're also shipped with the BeID middleware.

https://repository.eid.belgium.be/certificates.php?cert=Root&lang=en

Pass the file(s) you need with the --trust parameter. pyHanko should be able to find/download the required intermediate certs by itself, so only the roots are needed.

[plenaerts]

Using the --trust flag to the Citizen CA chain gives a valid "bottom line".

$ pyhanko sign validate --no-strict-syntax --trust ~/tmp/citizen_ca_chain.pem --pretty-print 20250116\ -\ myfile.signed.pdf 
2025-01-17 14:20:30,109 - cli - INFO - Strict PDF syntax is disabled; this could impact validation results. Use caution.
============
Field 1: SIG
============


Signer info
-----------
Certificate subject: "Serial Number: myrrn, Given Name: Pieter names, Surname: Lenaerts, Common Name: Pieter Lenaerts (Signature), Country: BE"
Certificate SHA1 fingerprint: fp
Certificate SHA256 fingerprint: fp
Trust anchor: "Serial Number: 201627, Common Name: Citizen CA, Country: BE"
The signer's certificate is trusted.


Integrity
---------
The signature is cryptographically sound.

The digest algorithm used was 'sha256'.
The signature mechanism used was 'sha256_rsa'.


Signing time
------------
Signing time as reported by signer: 2025-01-16T10:18:16+00:00


Modifications
-------------
The signature covers the entire file.


Bottom line
-----------
The signature is judged VALID.



$

[plenaerts]

To be honest, at this point I'd prefer forking the functionality of oscrypto that makes sense to keep around (i.e. the platform-independent API to fetch the system trust list, which likely doesn't depend on openssl at all). It's just that oscrypto's packaging process is a bit daunting, and I haven't gotten around to taking a closer look at it yet.

I had a quick look. I think you're right about the openssl dependcy of those parts of the code. But ...

1. To me it looks like quite some code of very specific nature for three different platform families.
2. I think it should be an independent library packages for other packages to use...

Not what I would call a simple carve-out. I think it would be quite some work to do properly and maintain in the future.

[plenaerts]

This is embarassing. I was on an old commit of pyhanko. Current head or latest tag don't give me this error since these versions of pyhanko only import oscrypto when needed.

So ... I guess this is only an issue when using old algorithms. I didn't see where pyhanko uses oscrypto to fetch the system trust list.

[MatthiasValvekens]

PyHanko doesn't do so directly, the code that pulls in the trust list is in pyhanko-certvalidator.

Of course, by "fork" I meant releasing the stuff I need as a separate package. No point in making things more complex in this codebase :)

Anyway, glad to hear that your problem seems to be solved!