Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Add policy about patching #3951

Merged
merged 2 commits into from
Jul 11, 2022

Conversation

jgsogo
Copy link
Contributor

@jgsogo jgsogo commented Dec 18, 2020

We need a policy about patching that is sustainable in time regarding libraries and reviewers (thanks again). We read feedback from issues here and also from comments in Slack and this is the proposal we think that can be most adecuate for regular consumers of Conan packages from ConanCenter.

close #3903


This is a live and thriving community full of passionate people. I'm personally very thankful to have you all around. Thanks! 😊

@conan-center-bot

This comment has been minimized.

should raise a `ConanInvalidConfiguration` from the `validate()` method.

**Vulnerability patches.-** Patches published to CVE databases or declared as
vulnerabilities by the authors in non-mainstream libraries will be applied
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
vulnerabilities by the authors in non-mainstream libraries will be applied
vulnerabilities by the authors will be applied

These seems like an unnecessary addition

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Active projects will apply them (in a timely manner 🤞 ) so we should not need to handle that case. I am okay with the original text

@Croydon
Copy link
Contributor

Croydon commented Dec 18, 2020

This policy would probably be stricter in some cases as I would have personally suggested, but I understand the arguments and I'm okay with it, as long as we are less strict about these things when it comes to cci.<date> versions. If we can agree on that, then this needs to be added to the policy text as well.


Another exemplary edge case to consider:

Assuming an application does depend on OpenSSL. However, the package does only support an OpenSSL version which is end-of-life and has known vulnerabilities/CVEs.

Someone proposes a patch that makes the application compile and work with still supported OpenSSL versions.

The application itself has no newer releases which could be added and used instead.

Would this patch count as a vulnerability fixing patch and be therefore accepted or would it count as a supporting-newer-dependencies patch and would therefore be denied?


Personally, I would count this as a vulnerability fixing patch which is important and shouldn't be generally denied. However, as with all patches it should proportional. Meaning if the patch would have a few thousand lines of code or something like that then we can't reasonable maintain this.

considered feature patches and they are not allowed either. They can
introduce new behaviors or bugs not considered when delivering the
library by maintainers. If a requirement is known not to work, the recipe
should raise a `ConanInvalidConfiguration` from the `validate()` method.
Copy link
Contributor

@SpaceIm SpaceIm Dec 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe wait for conan 1.32.0 in CI?

Also, why raise only in validate(), when we can raise before dependency graph resolution (saves from potential download of useless recipes)?
For me, the new method validate() is only for verifications based on informations unknown before dependency graph resolution (options values and versions of dependencies).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! The final version of the requirements is only known after the graph is fully resolved. There can be overrides and diamonds, the final version is not known in the configure() method while Conan is building the graph. I agree this would only happen with version ranges and with a few modifications in the graph resolution algorithm (a challenge for Conan 2). Now it will raise a conflict or not depending on the order of the requirements listed in app:

image

I agree this is not a scenario in conan-center-index where the version of the requirements is hardcoded.

Comment on lines +29 to +30
Patches to sources to add support to newer versions of dependencies are
considered feature patches and they are not allowed either. They can
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙊 broken by websocketpp to allow recent version of boost while maintainer was MIA

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replies to Croydon #3951 (comment)

prince-chrismc
prince-chrismc previously approved these changes Dec 19, 2020
@jgsogo jgsogo marked this pull request as draft December 19, 2020 11:53
@jgsogo
Copy link
Contributor Author

jgsogo commented Dec 19, 2020

I'm converting the PR to a draft so it doesn't get merged accidentally.

@theirix
Copy link
Contributor

theirix commented Dec 22, 2020

It is hard to make a clear distinction between security patches, bug/feature patches and even build patches.

Packaging and patching OSS in well studied and practised in Debian, for example. Description and origin of a Debian patch are tracked by DEP-3 format metadata. Upstream and forward status (whether the patch was applied or sent in upstream) allows maintainers to track PRs. There is even lint tooling around DEP-3. Forcing this complex format for recipes can be overkill. Probably we should use a similar description (structured or free-formed) for our patches to understand why a given patch is here and when it can be removed with a new release without searching in CCI or upstream issues.

By introducing quite large fixes for newer C++ standards and rare compilers we already change the code and behaviour. It can be changed explicitly by adding backports or security fixes. Since a package version 1.2.3 does not change with patching, it is already hard to tell if a package can still be called of version 1.2.3. If patches are properly described and categorized, I see no problem with security fixes, bug fixes and backports too. As we can see, a new release can be awaited for a long time or upstream may be dead.

Of course, a delta with upstream should be kept as minimal as we can for all kind of patches.

Proposed changes to the policy:

  • Add a requirement for source patches and vulnerability patches to have a clear description, version attribution and maybe an upstream bug reference.
  • Allow packages to have source patches.

@prince-chrismc
Copy link
Contributor

It is hard to make a clear distinction between security patches, bug/feature patches and even build patches.

It's intended to fix a CVE?

Certainly in favor for leaving a trail in the code 👍

@ericLemanissier
Copy link
Contributor

Quick suggestion: if a patch makes the binary deviate from upstream's version, Could it be ok to make it opt-in by using an option (with_patches or something else) which is False by default ?

@gocarlos
Copy link
Contributor

would be interesting to see how npm and cargo handle this...
as far as I know, when I do npm install -> npm checks for dependencies that have CVE, but does not apply patches to recipes

@prince-chrismc
Copy link
Contributor

would be interesting to see how npm and cargo handle this...
as far as I know, when I do npm install -> npm checks for dependencies that have CVE, but does not apply patches to recipes

To my knowledge, NPM just tracks the fix versions based on the CVE database when a fixed version gets published. example


I really like your idea Eric!

Could it be ok to make it opt-in

Perhaps a virtual release? piggy back of the existing system? This way we can maintain the consumer expectation of 1.3.6 === 1.3.6 but still allow cci.[<version>|<date>] to offer extra patches?

@prince-chrismc
Copy link
Contributor

So I understand where you are coming from with the new guidelines for patches. It sounds great to have the packages to be as vanilla and close to the developer intentions as possible. This is great to make conan and its packages a drop-in replacement for whatever you where using before to handle your project's dependencies.
These new policies are great to prevent a snow ball effect of having more and more patches retro fitting features and fixes and ending up with a package that is very different from what you get by building it from sources yourself.
Regardless, I think we could have some exceptions in place for cases like this one where it is a small change to fix a package which is otherwise non-functional in some scenarios and doesn't change the libraries behavior in any way. In this case we're just adding some headers here and there.
It makes even more sense with boost, which is a monster of a package and it is hard to migrate big projects that depend on it from one version to another.

Originally posted by @ericriff in #3918 (comment)

@stale
Copy link

stale bot commented Jan 22, 2021

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jan 22, 2021
@jgsogo
Copy link
Contributor Author

jgsogo commented Jan 25, 2021

This PR is not to close, but we are waiting for some _technical details (conan-io/hooks#268) before merging it.

@jgsogo jgsogo removed the stale label Jan 25, 2021
@stale
Copy link

stale bot commented Feb 24, 2021

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Feb 24, 2021
@stale
Copy link

stale bot commented Mar 26, 2021

This pull request has been automatically closed because it has not had recent activity. Thank you for your contributions.

@stale
Copy link

stale bot commented Jun 24, 2021

This pull request has been automatically closed because it has not had recent activity. Thank you for your contributions.

@stale
Copy link

stale bot commented Jul 28, 2021

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jul 28, 2021
@stale
Copy link

stale bot commented Aug 27, 2021

This pull request has been automatically closed because it has not had recent activity. Thank you for your contributions.

@stale stale bot closed this Aug 27, 2021
@prince-chrismc prince-chrismc mentioned this pull request Aug 30, 2021
4 tasks
@danimtb
Copy link
Member

danimtb commented Jul 4, 2022

I think we should have the policy in place to handle issues that may arise in patches contributed via PRs. So I am reopening this PR to continue its approval process

@danimtb danimtb reopened this Jul 4, 2022
@stale stale bot removed the stale label Jul 4, 2022
@SSE4
Copy link
Contributor

SSE4 commented Jul 7, 2022

needs a rebase

@conan-center-bot
Copy link
Collaborator

Updating docs!

@danimtb
Copy link
Member

danimtb commented Jul 11, 2022

let's move this forward and polish the technical details at conan-io/hooks#268 in a a new PR

@conan-center-bot conan-center-bot merged commit 59d62fe into conan-io:master Jul 11, 2022
AndreyMlashkin pushed a commit to AndreyMlashkin/conan-center-index that referenced this pull request Jul 29, 2022
Co-authored-by: SSE4 <tomskside@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[docs] Add a policy about patching
10 participants