Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.8.0 #17

Merged
merged 369 commits into from
Apr 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
369 commits
Select commit Hold shift + click to select a range
ff17bed
added simulate command
farchaab Dec 19, 2023
e4e7492
removed simulate reads
farchaab Dec 19, 2023
ab8e3c0
moved to simulate rule dir
farchaab Dec 19, 2023
64565c8
renamed
farchaab Dec 19, 2023
e6dd441
added simulate targets
farchaab Dec 19, 2023
3c3f6d4
moved list_reads to targets
farchaab Dec 19, 2023
485c749
moved to simulate rule dir
farchaab Dec 19, 2023
e5c7cb0
deleted example
farchaab Dec 20, 2023
524c953
moved rules
farchaab Dec 20, 2023
03df528
moved dir
farchaab Dec 20, 2023
1271637
added simulate snakefile
farchaab Dec 20, 2023
625c964
added samtools env
farchaab Dec 20, 2023
fa6e2af
added seqkit env
farchaab Dec 20, 2023
194bd85
moved calculate_coverage
farchaab Dec 20, 2023
3dfdcfc
added log file copy
farchaab Dec 20, 2023
2595904
added dirs
farchaab Dec 20, 2023
de404e2
update simulate command
farchaab Dec 20, 2023
33a59d9
update config
farchaab Dec 20, 2023
2bb2f3c
update simulate
farchaab Dec 20, 2023
1f1f21d
added art env
farchaab Dec 20, 2023
7b83467
added parse_samples function, updated dirs
farchaab Dec 20, 2023
a70beb3
removed rules
farchaab Dec 20, 2023
183fb9d
added samples table rule
farchaab Dec 20, 2023
daa2296
update dirs and added cov table in targets
farchaab Dec 20, 2023
e84c92b
update rule
farchaab Dec 20, 2023
8d8f5ea
corrected fastq outdir
farchaab Dec 20, 2023
c0b408f
added conda env and benchmark
farchaab Dec 20, 2023
161ed21
removed logging, added entry table
farchaab Dec 20, 2023
a1b097d
added samples script
farchaab Dec 20, 2023
362acb3
renamed samples_tables to replicates_tables
farchaab Jan 9, 2024
65bda5e
create new aggregate_samples rule
farchaab Jan 9, 2024
7df5602
added new directories
farchaab Jan 9, 2024
8dc5b97
corrected samples rules path path
farchaab Jan 9, 2024
7c07402
added long reads params
farchaab Jan 9, 2024
aac38ce
use samples.tsv as input for unique entries
farchaab Jan 9, 2024
dd3b938
removed chunks, added parse_fasta, added split contigs
farchaab Jan 9, 2024
fd78b7d
added contigs split in targets
farchaab Jan 9, 2024
ff02326
added contigs split for pbsim3
farchaab Jan 9, 2024
5d8f4c5
cat fastqs in cat directory
farchaab Jan 9, 2024
c557385
updated paths, removed chunks, added contigs splits
farchaab Jan 9, 2024
fb3f1a8
updated output path
farchaab Jan 9, 2024
ddbe85d
removed chunks
farchaab Jan 9, 2024
ec59a96
added replicates script
farchaab Jan 9, 2024
e00e59c
removed replicates from samples script
farchaab Jan 9, 2024
4f25560
added split contigs script
farchaab Jan 9, 2024
59e7a20
fixed err_profile and removed chunks
farchaab Jan 9, 2024
d82ec55
added command epilogs, added missing logs and renamed some args
farchaab Jan 11, 2024
43e7de2
renamed err_profile to error
farchaab Jan 11, 2024
12793e6
removed old snakefile
farchaab Jan 11, 2024
f4316c1
added snakefile for run command
farchaab Jan 11, 2024
2e909ce
added util functions
farchaab Jan 11, 2024
078340c
added run command and sim_options
farchaab Jan 11, 2024
80d79f9
removed parse fastas
farchaab Jan 11, 2024
ba21d59
fixed targets and removed parse fastas
farchaab Jan 11, 2024
fd55d6e
added list_fastas function
farchaab Jan 11, 2024
be2603e
fixed short reads dir
farchaab Jan 11, 2024
fb66bd2
removed shebang
farchaab Jan 11, 2024
1a27e98
removed examples
farchaab Jan 11, 2024
b5f4a34
added minimal test example
farchaab Jan 12, 2024
429e5e6
fixed when input is one file
farchaab Jan 12, 2024
7c88524
added test command
farchaab Jan 12, 2024
bbd3aa3
began adding hmp_template command
farchaab Jan 12, 2024
a8e1373
added hmp templates
farchaab Jan 12, 2024
6b57e80
added site and sample options for hmp-template command
farchaab Jan 12, 2024
3e28262
added download options for assembly_finder
farchaab Jan 15, 2024
5b79249
renamed total bases to bases
farchaab Jan 19, 2024
5b260c4
added abundances in accepted cols
farchaab Jan 19, 2024
da38db1
added frag _len and frag_sd
farchaab Jan 19, 2024
4a518b1
changed total bases to bases
farchaab Jan 19, 2024
d220faf
added readthedocs
farchaab Jan 19, 2024
b33b7e2
added custom_error and compressed args
farchaab Jan 24, 2024
cefcd66
bumped python and snakemake version
farchaab Jan 24, 2024
d6b5e7c
added custom_err and compressed args
farchaab Jan 24, 2024
6feba58
improved merge and dtypes, added path
farchaab Jan 24, 2024
76b04ea
fixed single sample bug, fasta paths are now from cov table
farchaab Jan 24, 2024
8f66a8a
removed examples
farchaab Jan 24, 2024
f6ad339
fixed sim targets with bam
farchaab Jan 24, 2024
ad873fd
added uncompressed fasta, changed fasta path inference
farchaab Jan 24, 2024
e813bf5
fixed merge bams
farchaab Jan 24, 2024
da708b7
added art custom profiles
farchaab Jan 24, 2024
3b764db
improved doc link
farchaab Jan 24, 2024
6716b74
bumped down python and snakemake versions
farchaab Jan 25, 2024
4b90b83
added missing args
farchaab Jan 25, 2024
ecbe926
renamed options
farchaab Jan 25, 2024
892d79b
removed nan fasta
farchaab Jan 25, 2024
7af3dc8
added skip fasta processing and read shuffling
farchaab Jan 31, 2024
6870bfb
added skip shuffling arg in simulate
farchaab Jan 31, 2024
82f5341
added skip fasta processing and read shuffling args
farchaab Jan 31, 2024
46a3860
adapted fasta input for when skipping fasta processing
farchaab Jan 31, 2024
038a20a
added option to skip reads shuffling
farchaab Jan 31, 2024
bafcbdc
adapt fasta input when pre-processing or not
farchaab Jan 31, 2024
10b0196
added shuffle directory
farchaab Jan 31, 2024
c1c88db
added anonamyze reads rule
farchaab Jan 31, 2024
bec4fb8
added contig name in read header
farchaab Jan 31, 2024
32c5a42
removed space
farchaab Jan 31, 2024
6b50988
add nb column in unique entries table
farchaab Feb 1, 2024
726a9e0
dont add replicate suffix when only one replicate
farchaab Feb 1, 2024
84c2cbd
removed count column
farchaab Feb 1, 2024
1da1612
updated resources
farchaab Feb 5, 2024
9259cfd
fixed art conda name
farchaab Feb 5, 2024
b9471ba
added bioconvert env
farchaab Feb 5, 2024
f7d47cd
added fasta processing env
farchaab Feb 5, 2024
90d4e88
added pbccs env
farchaab Feb 5, 2024
57fa0e0
added pbsim3 env
farchaab Feb 5, 2024
b8d8828
removed samtools env
farchaab Feb 5, 2024
23a406b
added resources and threads
farchaab Feb 5, 2024
808a2d4
added resources
farchaab Feb 5, 2024
72cff62
added conda and resources
farchaab Feb 5, 2024
9d0d519
added example snakemake-slurm profile
farchaab Feb 6, 2024
daacd83
added ref category param
farchaab Feb 19, 2024
a41c49c
added sum bases column
farchaab Feb 19, 2024
708703f
merged branch
farchaab Feb 19, 2024
f80f21d
added replicates table in logs
farchaab Feb 19, 2024
9b27120
removed double backslashes
farchaab Feb 19, 2024
9318f72
fixed dtypes
farchaab Feb 19, 2024
0c70c17
pull changes
farchaab Feb 19, 2024
3b26a50
fixed snakemake log
farchaab Feb 19, 2024
3bdcb4e
removed contig merge
farchaab Feb 29, 2024
d39a357
added pbsim3 model and ratio params
farchaab Feb 29, 2024
fa0553a
updated config
farchaab Feb 29, 2024
98562d4
added model and ratio, added long reads default error profiles
farchaab Feb 29, 2024
6986696
added long reads params and wildcards constraints
farchaab Feb 29, 2024
c454f47
added and updated functions
farchaab Feb 29, 2024
c464c85
moved function to functions.smk
farchaab Feb 29, 2024
42dd180
removed split contigs
farchaab Feb 29, 2024
5761585
added same reads post-rules steps for long and short reads
farchaab Feb 29, 2024
6969b54
fixed paths, moved read-post processing to reads.smk
farchaab Feb 29, 2024
e4b8ef2
simulate for contigs instead
farchaab Feb 29, 2024
92f2045
save split contigs in subdir
farchaab Feb 29, 2024
edd8662
added split temp directory
farchaab Mar 4, 2024
1e5c0f1
renamed anonymized reads log
farchaab Mar 4, 2024
b325260
changed split contig directory
farchaab Mar 4, 2024
e0a55fa
added mkdocs material theme
farchaab Mar 5, 2024
88065bd
added taxonkit env
farchaab Mar 5, 2024
d220404
added tax profile directory
farchaab Mar 5, 2024
f3243f9
added one aggregate function, added tax profile in output
farchaab Mar 5, 2024
c0099a3
added tax profile rules
farchaab Mar 5, 2024
e378cd7
added table cash to fix pandas memory error
farchaab Mar 15, 2024
0facd20
added splash screen to help
farchaab Mar 15, 2024
9890425
customized material docs theme
farchaab Mar 15, 2024
e296fa1
updated docs requirements
farchaab Mar 15, 2024
d18e8f0
updated docs
farchaab Mar 15, 2024
9231463
bumped version
farchaab Mar 28, 2024
3be27d9
updated to work with assembly_finder 0.6.0
farchaab Mar 28, 2024
524a2e1
updated download options
farchaab Mar 28, 2024
790f965
updated install requirements to include rich-click
farchaab Mar 28, 2024
c46ed99
Added rich-click option and command groups
farchaab Mar 28, 2024
5c3ef0c
updated env
farchaab Apr 2, 2024
cef22e5
fixed rank and nrank options
farchaab Apr 2, 2024
cf401e0
changed to updated assembly_finder options
farchaab Apr 2, 2024
b67bbcd
bumped version
farchaab Apr 2, 2024
6b9c2d3
updated assembly finder rule to newest version
farchaab Apr 2, 2024
6f303e6
updated download targets
farchaab Apr 2, 2024
99ea3b9
bumped assembly_finder version
farchaab Apr 2, 2024
aaf0173
removed skip fasta processing
farchaab Apr 2, 2024
b6de90b
changed error and mean_len defaults
farchaab Apr 2, 2024
e48a9a1
reduced minimal test
farchaab Apr 2, 2024
9964e5f
removed wildcards constraints and skip fasta processing
farchaab Apr 2, 2024
c13ea73
updated assembly_finder options and removed constraints and skip fast…
farchaab Apr 2, 2024
b94f955
bumped version
farchaab Apr 2, 2024
9f7aca5
changed fasta_input
farchaab Apr 2, 2024
bb33b58
updated target downloads
farchaab Apr 2, 2024
03627a7
updated targets simulate
farchaab Apr 2, 2024
95ca9fd
changed compressed to gzip
farchaab Apr 2, 2024
b90f86e
changed taxid to tax_id
farchaab Apr 2, 2024
02b5e60
removed skip_fa_proc
farchaab Apr 2, 2024
dc40d82
updated with new assembly_summary headers
farchaab Apr 2, 2024
9b0c725
updated headers
farchaab Apr 2, 2024
0b86793
updated gitignore
farchaab Apr 5, 2024
175927a
simplified merge_config
farchaab Apr 5, 2024
5b38c21
added conda-forge and defaults
farchaab Apr 5, 2024
aac4b85
updated LICENSE
farchaab Apr 8, 2024
7c1c579
updated env
farchaab Apr 8, 2024
884f0cd
changed fasta to processing
farchaab Apr 8, 2024
3ebbcc5
updated functions
farchaab Apr 8, 2024
db13b3e
removed env
farchaab Apr 8, 2024
4787c04
removed split in targets
farchaab Apr 8, 2024
5ef8fe6
checkpoint output no longer temp
farchaab Apr 8, 2024
1db2399
added rename_fasta rule
farchaab Apr 8, 2024
2f5703e
added checkpoint output in function
farchaab Apr 8, 2024
f1663d6
updated input fasta and table
farchaab Apr 8, 2024
2a49e0d
added tax_id, seed, cov_sim and samplename in table
farchaab Apr 8, 2024
411595d
updated install requirements and license
farchaab Apr 8, 2024
5be331f
fasta column defined by user or accession column
farchaab Apr 8, 2024
597b04d
removed gzip option
farchaab Apr 8, 2024
596a514
removed global env
farchaab Apr 8, 2024
380db3a
fixed channel order
farchaab Apr 8, 2024
ced426d
added contig in get_value
farchaab Apr 8, 2024
139439d
changed default conda prefix
farchaab Apr 8, 2024
b6085aa
added pacbio and nanopore in test command
farchaab Apr 8, 2024
43e8341
fixed simulators options
farchaab Apr 8, 2024
211cde1
added hifi accuracy and passes in test command
farchaab Apr 8, 2024
b51fe13
fixed env
farchaab Apr 8, 2024
0c40aad
added temp output
farchaab Apr 8, 2024
4663162
added split dir as input in cat_contig_fastqs
farchaab Apr 8, 2024
10925b5
updated model path
farchaab Apr 8, 2024
eeed28b
removed subdirs in split directory
farchaab Apr 8, 2024
14ac1ce
removed versions
farchaab Apr 8, 2024
2272bd0
fixed long reads error profiles in test command
farchaab Apr 8, 2024
0ee0bac
added temp output
farchaab Apr 8, 2024
538f603
removed tmpdir
farchaab Apr 8, 2024
9c170c0
added gh-pages workflow
farchaab Apr 9, 2024
578d2d7
removed readthedocs
farchaab Apr 9, 2024
a12cc21
updated mess workflow
farchaab Apr 9, 2024
06dc085
removed requirements
farchaab Apr 9, 2024
8c00040
updated citation
farchaab Apr 9, 2024
9dcb8fc
updated docs
farchaab Apr 9, 2024
66e3bf7
Generate new screengrabs with rich-codex
github-actions[bot] Apr 9, 2024
97cdb70
revert to old conda-prefix
farchaab Apr 9, 2024
f51f5a0
Generate new screengrabs with rich-codex
github-actions[bot] Apr 9, 2024
f8544fa
renamed abundance and proportion columns
farchaab Apr 12, 2024
d61ce84
build docs when push on main
farchaab Apr 12, 2024
49b2c25
renamed workflow image
farchaab Apr 12, 2024
8a16e68
added local fasta path option
farchaab Apr 12, 2024
23de46d
added split dir in targets
farchaab Apr 12, 2024
b2f8635
updated config
farchaab Apr 12, 2024
8bcb570
added seqkit stats rule
farchaab Apr 12, 2024
7cf011a
removed dir in cat_contig_fastqs
farchaab Apr 12, 2024
7c4013b
updated gitignore
farchaab Apr 15, 2024
b6a4bed
added coverage and pytest worflow
farchaab Apr 15, 2024
e032a79
fixed when seq-tech but no error profile in long reads
farchaab Apr 15, 2024
2e6d369
changed bam flag to --bam/--no-bam
farchaab Apr 15, 2024
afecd9c
updated resources
farchaab Apr 15, 2024
e64687f
added local fasta minimal example
farchaab Apr 15, 2024
d0f7a13
added copy_log_file
farchaab Apr 15, 2024
46270c2
removed temp checkpoint
farchaab Apr 15, 2024
e3618d3
adapted resources
farchaab Apr 15, 2024
43005d2
removed logs
farchaab Apr 15, 2024
ac66f29
updated config
farchaab Apr 16, 2024
ef6ebd2
reduced coverage in examples
farchaab Apr 16, 2024
0b2e955
fixed resources
farchaab Apr 16, 2024
66d8690
added unit tests
farchaab Apr 16, 2024
50d2788
removed snakemake version restriction
farchaab Apr 16, 2024
211fb4f
added taxonkit data-dir path
farchaab Apr 16, 2024
aa7064c
added pbsim3 models
farchaab Apr 16, 2024
46d1d6f
added pbsim3 model path as option
farchaab Apr 16, 2024
35f8ae4
added conda envs
farchaab Apr 16, 2024
7f4d797
moved long reads processing to reads.smk
farchaab Apr 16, 2024
d2b8bb3
added file renaming in pbsim3 rule
farchaab Apr 16, 2024
befd653
moved hmp_templates
farchaab Apr 16, 2024
3d0797d
updated minimum python version, constrained pulp
farchaab Apr 16, 2024
11676a5
fixed variable name
farchaab Apr 16, 2024
aa51cc9
test on python 3.12 only
farchaab Apr 16, 2024
2b7adce
decreased coverage for tests
farchaab Apr 16, 2024
360d9c6
fixed python versions tests
farchaab Apr 16, 2024
f976cc2
fixed sed version for macos
farchaab Apr 17, 2024
6ebc6c4
fixed taxdump url
farchaab Apr 17, 2024
1919cdb
improved envs formatting
farchaab Apr 17, 2024
41d6dfb
removed zcat in anonymize reads
farchaab Apr 17, 2024
c811b49
removed hifi test
farchaab Apr 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/workflows/build-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Build images / docs
on:
push:
branches:
- master
- main
# Cancel if a newer run is started
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
rich_codex:
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: 3.x

- uses: actions/cache@v2
with:
key: ${{ github.ref }}
path: .cache

- name: Install requirements for docs
run: pip install mkdocs-material .

- name: Generate terminal images with rich-codex
uses: ewels/rich-codex@v1
with:
commit_changes: "true"
clean_img_paths: docs/imgages/*.svg
skip_git_checks: "true"
terminal_width: 85

- name: Build and deploy docs
run: mkdocs gh-deploy --force
60 changes: 60 additions & 0 deletions .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: Unit tests

on:
push:
branches: ["main"]
paths:
- ".github/workflows/unit-tests.yaml"
- "tests/**"
- "mess/**"
- "setup.py"
pull_request:
branches: ["main"]
paths:
- ".github/workflows/unit-tests.yaml"
- "tests/**"
- "mess/**"
- "setup.py"

permissions:
contents: read

jobs:
tests:
name: "Python ${{ matrix.python-version }} on ${{ matrix.os }}"
runs-on: ${{ matrix.os }}

defaults:
run:
shell: bash -el {0}

strategy:
fail-fast: false
matrix:
os: [macos-latest, ubuntu-latest]
python-version: ["3.10", "3.12"]

steps:
- uses: "actions/checkout@v3"
with:
fetch-depth: 0

- uses: conda-incubator/setup-miniconda@v3
with:
auto-update-conda: true
miniforge-version: "latest"
miniforge-variant: Mambaforge
use-mamba: true
mamba-version: "*"
channels: conda-forge,bioconda,defaults
channel-priority: strict
activate-environment: mess
python-version: ${{ matrix.python-version }}
auto-activate-base: false

- name: "Test and generate coverage report on ${{ matrix.os }} for Python ${{ matrix.python-version }}"
run: |
python -m pip install --upgrade pip
python -m pip install pytest coverage
python -m pip install .
coverage run -m pytest
20 changes: 18 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,18 @@
.idea/
.snakemake/
mess.egg-info/
mess/__pycache__
build/

htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
tests/__pycache__
.pytest_cache

.snakemake
mess/workflow/conda
mess/workflow/taxonkit
Loading