Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a publish-build-info command for reporting build results to crates.io #3627

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/bin/cargo.rs
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ macro_rules! each_subcommand{
$mac!(package);
$mac!(pkgid);
$mac!(publish);
$mac!(publish_build_info);
$mac!(read_manifest);
$mac!(run);
$mac!(rustc);
Expand Down
74 changes: 74 additions & 0 deletions src/bin/publish_build_info.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
use cargo::core::Workspace;
use cargo::ops;
use cargo::util::{CliResult, Config};
use cargo::util::important_paths::find_root_manifest_for_wd;

#[derive(Deserialize)]
pub struct Options {
flag_target: Option<String>,
flag_host: Option<String>,
flag_token: Option<String>,
flag_manifest_path: Option<String>,
flag_verbose: u32,
flag_quiet: Option<bool>,
flag_color: Option<String>,
flag_dry_run: bool,
flag_frozen: bool,
flag_locked: bool,
cmd_pass: bool,
#[allow(dead_code)] // Pass and fail are mutually exclusive
cmd_fail: bool,
}

pub const USAGE: &'static str = "
Upload a package's build info to the registry: whether the crate built
successfully on a particular target with a particular version of Rust.

Usage:
cargo publish-build-info [options] (pass|fail)

Options:
-h, --help Print this message
--target TRIPLE Build for the target triple
--host HOST Host to upload the package to
--token TOKEN Token to use when uploading
--manifest-path PATH Path to the manifest of the package to publish
--dry-run Perform all checks without uploading
-v, --verbose ... Use verbose output (-vv very verbose/build.rs output)
-q, --quiet No output printed to stdout
--color WHEN Coloring: auto, always, never
--frozen Require Cargo.lock and cache are up to date
--locked Require Cargo.lock is up to date

";

pub fn execute(options: Options, config: &Config) -> CliResult {
config.configure(options.flag_verbose,
options.flag_quiet,
&options.flag_color,
options.flag_frozen,
options.flag_locked)?;

let Options {
flag_token: token,
flag_host: host,
flag_manifest_path,
flag_dry_run: dry_run,
flag_target: target,
cmd_pass,
..
} = options;

let root = find_root_manifest_for_wd(flag_manifest_path.clone(), config.cwd())?;
let ws = Workspace::new(&root, config)?;
ops::publish_build_info(&ws, ops::PublishBuildInfoOpts {
config: config,
token: token,
index: host,
dry_run: dry_run,
rust_version: config.rustc()?.version_channel_date()?.to_string(),
target: target,
passed: cmd_pass,
})?;
Ok(())
}
2 changes: 1 addition & 1 deletion src/cargo/ops/cargo_compile.rs
Original file line number Diff line number Diff line change
Expand Up @@ -659,7 +659,7 @@ fn scrape_build_config(config: &Config,
None => None,
};
let jobs = jobs.or(cfg_jobs).unwrap_or(::num_cpus::get() as u32);
let cfg_target = config.get_string("build.target")?.map(|s| s.val);
let cfg_target = config.build_target_triple()?;
let target = target.or(cfg_target);
let mut base = ops::BuildConfig {
host_triple: config.rustc()?.host.clone(),
Expand Down
1 change: 1 addition & 0 deletions src/cargo/ops/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ pub use self::cargo_package::{package, PackageOpts};
pub use self::registry::{publish, registry_configuration, RegistryConfig};
pub use self::registry::{registry_login, search, http_proxy_exists, http_handle};
pub use self::registry::{modify_owners, yank, OwnersOptions, PublishOpts};
pub use self::registry::{publish_build_info, PublishBuildInfoOpts};
pub use self::cargo_fetch::fetch;
pub use self::cargo_pkgid::pkgid;
pub use self::resolve::{resolve_ws, resolve_ws_precisely, resolve_with_previous};
Expand Down
57 changes: 56 additions & 1 deletion src/cargo/ops/registry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use std::time::Duration;

use curl::easy::{Easy, SslOpt};
use git2;
use registry::{Registry, NewCrate, NewCrateDependency};
use registry::{Registry, NewCrate, NewCrateDependency, NewVersionBuildInfo};

use url::percent_encoding::{percent_encode, QUERY_ENCODE_SET};

Expand Down Expand Up @@ -179,6 +179,61 @@ fn transmit(config: &Config,
}
}

pub struct PublishBuildInfoOpts<'cfg> {
pub config: &'cfg Config,
pub token: Option<String>,
pub index: Option<String>,
pub dry_run: bool,
pub rust_version: String,
pub target: Option<String>,
pub passed: bool,
}

pub fn publish_build_info(ws: &Workspace, opts: PublishBuildInfoOpts) -> CargoResult<()> {
let pkg = ws.current()?;

let (mut registry, _reg_id) = registry(opts.config, opts.token, opts.index)?;

// Upload build info to the specified destination
opts.config.shell().status("Uploading build info", pkg.package_id().to_string())?;

let target = opts.target;
let cfg_target = opts.config.build_target_triple()?;
let target = target.or(cfg_target);
let target = target.unwrap_or(opts.config.rustc()?.host.clone());

transmit_build_info(
opts.config, &pkg, &mut registry, opts.dry_run,
&opts.rust_version, &target, opts.passed)?;

Ok(())
}

fn transmit_build_info(config: &Config,
pkg: &Package,
registry: &mut Registry,
dry_run: bool,
rust_version: &str,
target: &str,
passed: bool) -> CargoResult<()> {

// Do not upload if performing a dry run
if dry_run {
config.shell().warn("aborting upload due to dry run")?;
return Ok(());
}

registry.publish_build_info(&NewVersionBuildInfo {
name: pkg.name().to_string(),
vers: pkg.version().to_string(),
rust_version: rust_version.to_string(),
target: target.to_string(),
passed: passed,
}).map_err(|e| {
CargoError::from(e.to_string())
})
}

pub fn registry_configuration(config: &Config) -> CargoResult<RegistryConfig> {
let index = config.get_string("registry.index")?.map(|p| p.val);
let token = config.get_string("registry.token")?.map(|p| p.val);
Expand Down
4 changes: 4 additions & 0 deletions src/cargo/util/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,10 @@ impl Config {

pub fn cwd(&self) -> &Path { &self.cwd }

pub fn build_target_triple(&self) -> CargoResult<Option<String>> {
self.get_string("build.target").map(|r| r.map(|s| s.val))
}

pub fn target_dir(&self) -> CargoResult<Option<Filesystem>> {
if let Some(dir) = env::var_os("CARGO_TARGET_DIR") {
Ok(Some(Filesystem::new(self.cwd.join(dir))))
Expand Down
4 changes: 4 additions & 0 deletions src/cargo/util/rustc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,8 @@ impl Rustc {
util::process(&self.path)
}
}

pub fn version_channel_date(&self) -> CargoResult<&str> {
self.verbose_version.lines().next().ok_or(internal("rustc -v didn't have any lines"))
}
}
21 changes: 21 additions & 0 deletions src/crates-io/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,15 @@ pub struct NewCrateDependency {
pub kind: String,
}

#[derive(Serialize)]
pub struct NewVersionBuildInfo {
pub name: String,
pub vers: String,
pub rust_version: String,
pub target: String,
pub passed: bool,
}

#[derive(Deserialize)]
pub struct User {
pub id: u32,
Expand Down Expand Up @@ -228,6 +237,18 @@ impl Registry {
})
}

pub fn publish_build_info(&mut self, build_info: &NewVersionBuildInfo)
-> Result<()> {
let body = serde_json::to_string(build_info)?;
let url = format!("/crates/{}/{}/build_info", build_info.name, build_info.vers);

let body = self.put(url, body.as_bytes())?;

assert!(serde_json::from_str::<R>(&body)?.ok);

Ok(())
}

pub fn search(&mut self, query: &str, limit: u8) -> Result<(Vec<Crate>, u32)> {
let formated_query = percent_encode(query.as_bytes(), QUERY_ENCODE_SET);
let body = self.req(
Expand Down
75 changes: 75 additions & 0 deletions src/doc/crates-io.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,81 @@ The syntax for teams is currently `github:org:team` (see examples above).
In order to add a team as an owner one must be a member of that team. No
such restriction applies to removing a team as an owner.

## `cargo publish-build-info`

The `cargo publish-build-info` command is intended to help automate reporting
on which versions of Rust your crate's released versions work successfully
with. It is meant to work with the results of continuous integration runs. It
will work with any CI service; below are instructions for Travis CI, but the
idea should be generalizable to any setup.

`cargo publish-build-info` will report the version of rustc, the version of
your crate, and the target that you run the command with. The target may
optionally be specified as something other than the operating system the
command is running on by specifying the `--target` flag.

When CI runs on a tagged (released) version of your crate, run this command
with the value `pass` or `fail` depending on the results of your CI script.

Results with a particular crate version, rustc version, and target can only be
reported once. A possible enhancement is to allow overwriting in the future.
Until then, only report on your final tagged release version.

Crates.io must already know about a crate and version in order for you to
publish build information about them, so intended workflow is:

1. Run regular CI to verify your crate compiles and passes tests
2. Bump to the version you want to release in `Cargo.toml` and commit
3. Tag that commit since the CI setup recommended below will only run on tagged
versions
4. Publish to crates.io
5. Push the tag in order to run CI on the tagged version, which will then run
`cargo publish-build-info` with the results.

Yes, you can report possibly-incorrect results manually, but your users
will probably report a bug if they can't reproduce your reported results.

On crate list pages such as search results, your crate will have a badge if you
have reported `pass` results for the max version of your crate. If you have
reported that it passes on stable, the version of stable will be displayed in a
green badge. If no stable versions have a reported pass result, but a beta
version of Rust has, the date of the latest beta that passed will be displayed
in a yellow badge. If there have been no pass results on stable or beta but
there have been for nightly, the date of the latest nightly that passed will be
displayed in an orange badge. If there have been no pass results reported for
any Rust version, no badge will be shown for that crate.

If there have been any results reported for the Tier 1 targets on 64 bit
architectures for a version of a crate, there will be a section on that
version's page titled "Build info" that will display more detailed results for
the latest version of each of the stable, beta, and nightly channels for those
targets.

### Travis configuration to automatically report build info

First, make an [encrypted environment variable][travis-env] named TOKEN with
your crates.io API key.

Then add this to your `.travis.yml`, substituting in your secure environment
variable value where indicated:

```yml
env:
- secure: [your secure env var value here]

after_script: >
if [ -n "$TRAVIS_TAG" ] ; then
result=$([[ $TRAVIS_TEST_RESULT = 0 ]] && echo pass || echo fail)
cargo publish-build-info $result --token TOKEN
fi
```

The code in `after_script` checks to see if you're building a tagged commit,
and if so, checks to see if the build passed or failed, then runs the `cargo
publish-build-info` command to send the build result to crates.io.

[travis-env]: https://docs.travis-ci.com/user/environment-variables/#Defining-encrypted-variables-in-.travis.yml

## GitHub permissions

Team membership is not something GitHub provides simple public access to, and it
Expand Down