Skip to content

Commit

Permalink
Run cargo update on the runtime lockfiles and the SDK lockfile (#3856)
Browse files Browse the repository at this point in the history
If CI fails, commit the necessary fixes to this PR until all checks
pass. If required, update entries in
[crateNameToLastKnownWorkingVersions](https://github.com/smithy-lang/smithy-rs/blob/6b42eb5ca00a2dc9c46562452e495a2ec2e43d0f/aws/sdk/build.gradle.kts#L503-L504).

---------

Co-authored-by: ysaito1001 <awsaito@amazon.com>
  • Loading branch information
aws-sdk-rust-ci and ysaito1001 authored Oct 4, 2024
1 parent 2c0274c commit e7f1031
Show file tree
Hide file tree
Showing 9 changed files with 1,001 additions and 744 deletions.
189 changes: 99 additions & 90 deletions aws/rust-runtime/Cargo.lock

Large diffs are not rendered by default.

158 changes: 88 additions & 70 deletions aws/rust-runtime/aws-config/Cargo.lock

Large diffs are not rendered by default.

549 changes: 248 additions & 301 deletions aws/sdk/Cargo.lock

Large diffs are not rendered by default.

432 changes: 202 additions & 230 deletions rust-runtime/Cargo.lock

Large diffs are not rendered by default.

25 changes: 24 additions & 1 deletion tools/ci-build/sdk-lockfiles/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion tools/ci-build/sdk-lockfiles/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "sdk-lockfiles"
version = "0.1.1"
version = "0.1.2"
authors = ["AWS Rust SDK Team <aws-sdk-rust@amazon.com>"]
description = """
A CLI tool to audit lockfiles for Smithy runtime crates, AWS runtime crates, `aws-config`, and the workspace containing
Expand All @@ -14,10 +14,12 @@ publish = false
anyhow = "1.0.87"
cargo-lock = { version = "9.0.0", features = ["dependency-tree"] }
clap = { version = "4.4.11", features = ["derive", "env"] }
once_cell = "1.15.0"
petgraph = "0.6.5"
smithy-rs-tool-common = { path = "../smithy-rs-tool-common" }
tracing = "0.1.40"
tracing-subscriber = { version = "0.3.18", features = ["env-filter"] }

[dev-dependencies]
itertools = "0.13.0"
tracing-test = "0.2.4"
29 changes: 26 additions & 3 deletions tools/ci-build/sdk-lockfiles/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,35 @@ Error: there are lockfile audit failures

This tool is intended for automated use.

## Limitation
## Limitations
### Unable to audit dependencies in `CargoDependency.kt`
The `sdk-lockfiles` tool does not verify whether new dependencies introduced in [CargoDependency.kt](https://github.com/smithy-lang/smithy-rs/blob/main/codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/rustlang/CargoDependency.kt)
are included in the SDK lockfile. This is because dependencies in `CargoDependency.kt` are represented as a Kotlin data
class. Consequently, dependencies added via the code generator, `inlineable`, or `aws-inlineable` are not considered by
`sdk-lockfiles`.

This limitation is acceptable for our operational purposes. Our release script always executes
`./gradlew aws:sdk:syncAwsSdkLockfile`, which ensures that any dependencies added in `CargoDependency.kt` are properly
This limitation is acceptable for our operational purposes. Our release script [always executes
`./gradlew aws:sdk:syncAwsSdkLockfile`](https://github.com/smithy-lang/smithy-rs/blob/b62000e4d733ca06bc98fd9b57c91468718b8f9f/tools/ci-scripts/generate-smithy-rs-release#L36), which ensures that any dependencies added in `CargoDependency.kt` are properly
reflected in the SDK lockfile.

### False positives
The `sdk-lockfiles` tool may report false positives based on the contents of a lockfile. For example, if a section of
the lockfile appears as follows
```
pin-project v1.1.5
├── tower v0.4.13
│ ├── aws-smithy-experimental v0.1.4
│ ├── aws-smithy-http-server v0.63.3
│ │ └── aws-smithy-http-server-python v0.63.2
│ ├── aws-smithy-http-server-python v0.63.2
...
```
the tool cannot identify which dependent crate of `tower` enables `tower`'s Cargo feature to include `pin-project`.
In the case above, `aws-smithy-experimental` does not enable this feature, while `aws-smithy-http-server` does.
Among the Smithy runtime crates above, only `aws-smithy-experimental` is used by SDKs. When `aws-smithy-experimental`
is compiled for a generated SDK without server-related Smithy runtime crates, `pin-project` will not appear in the
SDK lockfile. Therefore, while it may appear that `aws-smithy-experimental` depends on `pin-project`, it is a false
positive for the audit.

To address this limitation, we maintain a list of known false positives in `false-positives.txt`. Any dependency
included in this file will not be flagged as an audit error.
2 changes: 2 additions & 0 deletions tools/ci-build/sdk-lockfiles/false-positives.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
aws-smithy-experimental -> pin-project
aws-smithy-experimental -> pin-project-internal
Loading

0 comments on commit e7f1031

Please sign in to comment.