Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Use absolute_path instead of fs::canonicalize #3324

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,9 @@ structopt = "0.2.15"

config_proc_macro = { path = "config_proc_macro" }

[target.'cfg(windows)'.dependencies]
winapi = { version = "0.3", features = ["errhandlingapi", "fileapi"] }

# A noop dependency that changes in the Rust repository, it's a bit of a hack.
# See the `src/tools/rustc-workspace-hack/README.md` file in `rust-lang/rust`
# for more information.
Expand Down
10 changes: 5 additions & 5 deletions src/bin/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ use failure::err_msg;
use getopts::{Matches, Options};

use crate::rustfmt::{
load_config, CliOptions, Color, Config, Edition, EmitMode, ErrorKind, FileLines, FileName,
FormatReportFormatterBuilder, Input, Session, Verbosity,
absolute_path, load_config, CliOptions, Color, Config, Edition, EmitMode, ErrorKind, FileLines,
FileName, FormatReportFormatterBuilder, Input, Session, Verbosity,
};

fn main() {
Expand Down Expand Up @@ -432,9 +432,9 @@ fn determine_operation(matches: &Matches) -> Result<Operation, ErrorKind> {
.iter()
.map(|s| {
let p = PathBuf::from(s);
// we will do comparison later, so here tries to canonicalize first
// to get the expected behavior.
p.canonicalize().unwrap_or(p)
// we will do comparison later, so here tries to get the absolute
// path first to get the expected behavior.
absolute_path(&p).unwrap_or(p)
})
.collect();

Expand Down
12 changes: 7 additions & 5 deletions src/cargo-fmt/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
#![deny(warnings)]

use cargo_metadata;
use rustfmt_nightly as rustfmt;

use std::cmp::Ordering;
use std::collections::{BTreeMap, BTreeSet};
use std::env;
use std::fs;
use std::hash::{Hash, Hasher};
use std::io::{self, Write};
use std::iter::FromIterator;
Expand Down Expand Up @@ -51,6 +51,8 @@ pub struct Opts {
format_all: bool,
}

use crate::rustfmt::absolute_path;

fn main() {
let exit_status = execute();
std::io::stdout().flush().unwrap();
Expand Down Expand Up @@ -169,10 +171,10 @@ pub struct Target {
impl Target {
pub fn from_target(target: &cargo_metadata::Target) -> Self {
let path = PathBuf::from(&target.src_path);
let canonicalized = fs::canonicalize(&path).unwrap_or(path);
let path = absolute_path(&path).unwrap_or(path);

Target {
path: canonicalized,
path,
kind: target.kind[0].clone(),
edition: target.edition.clone(),
}
Expand Down Expand Up @@ -247,9 +249,9 @@ fn get_targets(strategy: &CargoFmtStrategy) -> Result<BTreeSet<Target>, io::Erro

fn get_targets_root_only(targets: &mut BTreeSet<Target>) -> Result<(), io::Error> {
let metadata = get_cargo_metadata(None)?;
let current_dir = env::current_dir()?.canonicalize()?;
let current_dir = absolute_path(env::current_dir()?)?;
let current_dir_manifest = current_dir.join("Cargo.toml");
let workspace_root_path = PathBuf::from(&metadata.workspace_root).canonicalize()?;
let workspace_root_path = absolute_path(PathBuf::from(&metadata.workspace_root))?;
let in_workspace_root = workspace_root_path == current_dir;

for package in metadata.packages {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based on the findings in https://github.com/rust-lang/rustfmt/pull/3309/files#diff-0f04fceb10cfdeda3716c7824a8dc2deR246, maybe you need to get the absolute_path of manifest_path:

absolute_path(PathBuf::from(&package.manifest_path)) == current_dir_manifest

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@topecongiro can we merge this PR with #3309 ? this would fix also @rivy 's comment #3324 (comment)

Copy link

@rivy rivy Mar 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be a bug against PathBuf.canonicalize(). Any conversion to UNC must use the absolute path. PathBuf is taking that responsibility and should do it itself.

Here, as a workaround, I think you should use both; canonicalize the absolute path...

let workspace_root_path = absolute_path(PathBuf::from(&metadata.workspace_root))?.canonicalize()?;

Copy link

@rivy rivy Mar 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect a wrapper function (maybe just a passthru for non-windows OS) which executes this work-around would be a good way to proceed. And, I'd replace all uses of canonicalize() in the code base. A quick repo search shows only about about eight uses.

Not meaning to add more work, but any path comparisons should probably be reviewed for possibly needing canonicalizing of both sides. And I'm not sure where those might be scattered throughout the code base.

$0.02

Thanks for taking the time to review and repair this issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review this in #3309 once this PR is merged.

Expand Down
13 changes: 7 additions & 6 deletions src/config/file_lines.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ use std::{cmp, fmt, iter, str};

use serde::{ser, Deserialize, Deserializer, Serialize, Serializer};
use serde_json as json;

use syntax::source_map::{self, SourceFile};

use crate::utils::absolute_path;

/// A range of lines in a file, inclusive of both ends.
pub struct LineRange {
pub file: Rc<SourceFile>,
Expand Down Expand Up @@ -220,7 +221,7 @@ impl FileLines {
Some(ref map) => map,
};

match canonicalize_path_string(file_name).and_then(|file| map.get(&file)) {
match absolute_path_string(file_name).and_then(|file| map.get(&file)) {
Some(ranges) => ranges.iter().any(f),
None => false,
}
Expand Down Expand Up @@ -259,9 +260,9 @@ impl<'a> iter::Iterator for Files<'a> {
}
}

fn canonicalize_path_string(file: &FileName) -> Option<FileName> {
fn absolute_path_string(file: &FileName) -> Option<FileName> {
match *file {
FileName::Real(ref path) => path.canonicalize().ok().map(FileName::Real),
FileName::Real(ref path) => absolute_path(path).ok().map(FileName::Real),
_ => Some(file.clone()),
}
}
Expand Down Expand Up @@ -291,8 +292,8 @@ pub struct JsonSpan {
impl JsonSpan {
fn into_tuple(self) -> Result<(FileName, Range), String> {
let (lo, hi) = self.range;
let canonical = canonicalize_path_string(&self.file)
.ok_or_else(|| format!("Can't canonicalize {}", &self.file))?;
let canonical = absolute_path_string(&self.file)
.ok_or_else(|| format!("Can't get absolute path {}", &self.file))?;
Ok((canonical, Range::new(lo, hi)))
}
}
Expand Down
4 changes: 4 additions & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ extern crate lazy_static;
#[macro_use]
extern crate log;

#[cfg(windows)]
extern crate winapi;

use std::cell::RefCell;
use std::collections::HashMap;
use std::fmt;
Expand All @@ -32,6 +35,7 @@ pub use crate::config::{
load_config, CliOptions, Color, Config, Edition, EmitMode, FileLines, FileName, NewlineStyle,
Range, Verbosity,
};
pub use crate::utils::absolute_path;

pub use crate::format_report_formatter::{FormatReportFormatter, FormatReportFormatterBuilder};

Expand Down
44 changes: 43 additions & 1 deletion src/utils.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
use std::borrow::Cow;
use std::io;
use std::path;

use bytecount;

use rustc_target::spec::abi;
use syntax::ast::{
self, Attribute, CrateSugar, MetaItem, MetaItemKind, NestedMetaItem, NestedMetaItemKind,
Expand Down Expand Up @@ -615,6 +616,47 @@ pub(crate) fn unicode_str_width(s: &str) -> usize {
s.width()
}

#[cfg(windows)]
pub fn absolute_path<P: AsRef<path::Path>>(p: P) -> io::Result<path::PathBuf> {
use std::ffi::OsString;
use std::iter::once;
use std::os::windows::ffi::{OsStrExt, OsStringExt};
use std::ptr::null_mut;
use winapi::um::errhandlingapi::GetLastError;
use winapi::um::fileapi::GetFullPathNameW;

// FIXME: This `MAX_PATH` may be valid only from Windows 10, version 1607.
// https://docs.microsoft.com/ja-jp/windows/desktop/FileIO/naming-a-file#paths
const MAX_PATH: usize = 32767;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is correct.

From my reading of the documentation noted above, it looks like a MAX_PATH of 32000+ can only be depended on for Windows 10, for certain later versions, and only if the system/user has opted-in, by changing a registry entry. This, to me, means that it can never be depended on and should likely not be used.

MAX_PATH should be ~260 characters for normal paths (absolute or relative). For longer paths (up to 32000+ characters), the path would need to be in "extended" format which means absolute form prefixed with "\\?\".

Copy link
Contributor

@scampi scampi May 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like you cannot use \\?\ with relative path according to https://docs.microsoft.com/en-gb/windows/desktop/FileIO/naming-a-file#maximum-path-length-limitation

Because you cannot use the \\?\ prefix with a relative path, relative paths are always limited to a total of MAX_PATH characters.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to prefix the path with \\?\ and I get an error:

The specified path is invalid. (os error 161)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must be in absolute path form.

let wide: Vec<u16> = p
.as_ref()
.as_os_str()
.encode_wide()
.chain(once(0))
.collect();
let mut buffer: Vec<u16> = vec![0; MAX_PATH];
unsafe {
let result = GetFullPathNameW(
wide.as_ptr(),
MAX_PATH as u32,
buffer.as_mut_ptr(),
null_mut(),
);
if result == 0 {
Err(std::io::Error::from_raw_os_error(GetLastError() as i32))
} else {
Ok(path::PathBuf::from(OsString::from_wide(
&buffer[..result as usize],
)))
}
}
}

#[cfg(not(windows))]
pub fn absolute_path<P: AsRef<path::Path>>(p: P) -> io::Result<path::PathBuf> {
std::fs::canonicalize(p)
}

pub(crate) fn get_skip_macro_names(attrs: &[ast::Attribute]) -> Vec<String> {
let mut skip_macro_names = vec![];
for attr in attrs {
Expand Down