Version 0.3

[bovee]

Preliminary change list:

• better error reporting (i.e. parse failure now gives the record it failed on)
• simplification of fastx_bytes, fastx_stream into parse_sequence_reader and fastx_cli into parse_sequence_path
• SeqRecord is now SequenceRecord and many of its methods have been spun out into a Sequence trait that allows working on e.g. byte slices
• the .kmers method has been simplified and a new .canonical_kmers method has been introduced with much of the originals functionality (and an takes an explicit reverse_complement to allow its reference to be & instead of &mut)
• FASTQ parser tracks second ID field
• automatic decompression now takes Read instead of Read + Seek so we can handle e.g. gzip files piped in through stdin
• removal of single-file zip handling to support above (zip requires Seek) 😞
• lots of cleanup (cargo clippyd)
• addition of fuzzing targets

[bovee]

Benchmarks

Current benchmarks are within range of error, although perhaps slightly slower (this may be expected since we're now doing more tracking of parse state to allow better error messages):

New

test bench_bitkmer_speed ... bench:  89,666,605 ns/iter (+/- 1,059,245)
test bench_fasta_bytes   ... bench:  16,579,247 ns/iter (+/- 1,685,079)
test bench_fasta_file    ... bench:  16,507,717 ns/iter (+/- 1,163,518)
test bench_fastq_bytes   ... bench:   4,438,460 ns/iter (+/- 866,800)
test bench_fastq_file    ... bench:   4,408,823 ns/iter (+/- 316,858)
test bench_kmer_speed    ... bench: 200,107,045 ns/iter (+/- 7,185,221)

Old

test bench_bitkmer_speed ... bench:  90,731,881 ns/iter (+/- 9,133,680)
test bench_fasta_bytes   ... bench:  16,563,511 ns/iter (+/- 1,104,243)
test bench_fasta_file    ... bench:  16,537,416 ns/iter (+/- 646,898)
test bench_fastq_bytes   ... bench:   4,274,139 ns/iter (+/- 159,441)
test bench_fastq_file    ... bench:   4,298,571 ns/iter (+/- 171,011)
test bench_kmer_speed    ... bench: 196,687,450 ns/iter (+/- 9,193,177)

Fuzzing

I set up fuzzing and created test cases for parsing FASTA and FASTQ files. These fairly rapidly caught a small bug with FASTQ files having different lines endings (\r\n vs \n) between their ID and sequence lines causing panics. I then ran cargo fuzz run parse_fasta out past 300,000 iterations and cargo fuzz run parse_fastq out past 500,000 iterations without finding any more issues.

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@a192a17). Click here to learn what that means.
The diff coverage is 90%.

@@          Coverage Diff           @@
##             master   #26   +/-   ##
======================================
  Coverage          ?   90%           
======================================
  Files             ?     5           
  Lines             ?   590           
  Branches          ?     0           
======================================
  Hits              ?   531           
  Misses            ?    59           
  Partials          ?     0

Impacted Files	Coverage Δ
src/formats/mod.rs	86.27% <86.27%> (ø)
src/formats/fasta.rs	88.7% <88.7%> (ø)
src/formats/fastq.rs	90.12% <90.12%> (ø)
src/formats/buffer.rs	94.73% <94.73%> (ø)
tests/format_specimens.rs	97.91% <97.91%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a192a17...b4bf635. Read the comment docs.

[bovee]

This is a breaking update. As a rough updating guide based off porting both finch and some of our internal pipelines to v0.3, the following changes (at a minimum) are going to be necessary to keep code working:

Change

for (ix, kmer, is_canonical) in read.kmers(k, true) {
    ...
}

to

let rc = read.reverse_complement();
for (ix, kmer, is_canonical) in read.canonical_kmers(k, &rc) {
    ...
}

Change

use needletail::seq::SeqRecord;

to

use needletail::SequenceRecord;

Change

use needletail::fastx::fastx_cli;

to

use needletail::parse_sequence_path;

Also, if anyone's using needletail v0.2 and needs help/advice on porting to v0.3, I'm happy to look over code and answer questions.

[boydgreenfield]

👏