Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trying to read directory with BufReader::lines while filtering errors consumes 100% CPU #43504

Closed
mkpankov opened this issue Jul 27, 2017 · 2 comments

Comments

@mkpankov
Copy link
Contributor

If we File::open() a directory, then wrap it in BufReader, call lines() and then filter_map() only successful reads, the programs never finishes:

use std::fs::File;
use std::io::{BufRead, BufReader};

fn main() {
    let file = File::open(".").unwrap();
    let reader = BufReader::new(file);
    let lines = reader.lines();
    // Comment to make it output infinite errors
    let lines = lines.filter_map(|s| s.ok());
    for line in lines {
        println!("Line: {:?}", line);
    }
    println!("Finished");
}

"Finished" is never printed. No "Line: ..." lines are printed. Instead 100% CPU is consumed.

Commenting the filter_map() line makes it output infinite Line: Err(Error { repr: Os { code: 21, message: "Is a directory" } }).

https://play.rust-lang.org/?gist=f6f0fdbcc83b9630b493c025e3432d31&version=stable

Looks similar to #34703, but not sure it's because of same reasons.

Not sure what would a proper fix would be, but I think I'd prefer lines() to output a single item with error instead of spamming it.

@mkpankov mkpankov changed the title Trying to read directory with BufReader::lines while filtering errors hangs Trying to read directory with BufReader::lines while filtering errors consumes 100% CPU Jul 27, 2017
@TimNN
Copy link
Contributor

TimNN commented Jul 27, 2017

That behaviour is expected, as far as I can tell:

  • Opening the directory is fine
  • Whenever you attempt to read from the opened directory (here: whenever you attempt to read a line), the OS returns an error (as expected)
  • You ignore the produced errors (.filter_map, .ok())

Thus the only thing this program does, is ignoring errors as fast as the OS can produce them, leading to 100% CPU consumption.

@arielb1
Copy link
Contributor

arielb1 commented Jul 27, 2017

This is the expected behavior of filter_map. It keeps on trying (to read from the directory) until it would succeed (aka never). You might want to use .take_while(|x| x.is_ok()).map(|x| x.unwrap()), or just a for-loop, to stop after the first error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants