Added the profile2cami tool, a component of the TaxonKit suite. #6085

Alby-Git · 2024-06-14T08:00:35Z

Tool details for profile2cami

Name: profile2cami
Description: A utility from the taxonkit suite for converting taxonomic profiles into the CAMI format.
Homepage: profile2cami
License: License

Checklist for submission

FOR CONTRIBUTOR:

- I have read the CONTRIBUTING.md document and this tool is appropriate for the tools-iuc repo.
- License permits unrestricted use (educational + commercial)
- This PR adds a new tool or tool collection
- This PR updates an existing tool or tool collection
- This PR does something else (explain below)

Thank you for your consideration.

bgruening · 2024-06-14T20:56:20Z

tools/taxonkit/profile2cami/profile2cami.xml

+        <param name="show_rank" type="text" value="superkingdom,phylum,class,order,family,genus,species,strain" label="Show Ranks" help="Specify the ranks to show in the result file." />
+    </inputs>
+    <outputs>
+        <data name="cami_output" format="tsv" label="Converted CAMI Output" />


Should we add a proper CAMI format to Galaxy?

Taxonkit serves as an intermediary step in my project to prepare data for the upcoming CAMI Opal Tool, which requires the CAMI Profiling Bioboxes format as input. Within the scope of my project, implementing the CAMI format isn't necessary, but it could be beneficial for broader applications.

It is well-defined and could become a standard (https://github.com/CAMI-challenge/contest_information/blob/master/file_formats/CAMI_TP_specification.mkd) probably a good start for me to learn how to add a new format. But as table it works just fine ...

bgruening · 2024-06-14T20:57:26Z

tools/taxonkit/profile2cami/test-data/ncbi_taxonomy.loc

can we call this file .loc.test?

I have tested renaming the file; unfortunately, I think Galaxy needs a .loc file to find the content.

bgruening · 2024-06-14T20:59:25Z

tools/taxonkit/profile2cami/test-data/test-db/delnodes.dmp

Is it possible to shrink this file?

Totally! Reduced the size a bit; I hope that is sufficient.

bgruening

Awesome quality! Just a few tiny questions. Thanks

paulzierep

Looks great. In generally, I am just a bit confused about the recomputation of the abundance. Basic: 99.999999999999986; with percentage: 1.000000000000000; that could just be NumPy rounding issues, but why does recompute not consider the deleted taxon ?

paulzierep · 2024-06-20T11:46:57Z

tools/taxonkit/profile2cami/profile2cami.xml

+        <param name="show_rank" type="text" value="superkingdom,phylum,class,order,family,genus,species,strain" label="Show Ranks" help="Specify the ranks to show in the result file." />
+    </inputs>
+    <outputs>
+        <data name="cami_output" format="tsv" label="Converted CAMI Output" />


It is well-defined and could become a standard (https://github.com/CAMI-challenge/contest_information/blob/master/file_formats/CAMI_TP_specification.mkd) probably a good start for me to learn how to add a new format. But as table it works just fine ...

paulzierep

I think with these small changes it is ready. But when I tested it, I realised, that the results seem not to comply with the CAMI satetment: The percentages given for all taxa from the same rank should sum up to <= 100%
https://github.com/CAMI-challenge/contest_information/blob/master/file_formats/CAMI_TP_specification.mkd
However, the tools seems to sum up percentages when lower ranks are reported, this is not correct. I need to inverstigate, but if the tools is indeed wrong, I think we should not merge it ...

paulzierep · 2024-07-02T07:01:40Z

tools/taxonkit/.shed.yml

@@ -0,0 +1,18 @@
+name: taxonkit
+owner: iuc
+description: Tools for metagenomic profiling and analysis


Its not used for profiling and specific to NCBI TaxonKit - A Practical and Efficient NCBI Taxonomy Toolkit something like that I guess

paulzierep · 2024-07-02T07:02:56Z

tools/taxonkit/macros.xml

@@ -0,0 +1,17 @@
+ <macros>


Please add bio.tool xml: https://bio.tools/taxonkit

paulzierep · 2024-07-02T10:25:56Z

tools/taxonkit/taxonkit_profile2cami.xml

+        --data-dir '${taxonomy.fields.path}'
+        --abundance-field '${abundance_field}'
+        --taxid-field '${taxid_field}'
+        #if $percentage:


See @bernt-matthias comment https://github.com/galaxyproject/tools-iuc/pull/6096/files/0f22502b798ad0481fd372e66c1cb26e31734ee8#r1647385458
Can you please use truevalue and remove the if for all such cases ?

paulzierep · 2024-07-02T10:26:37Z

tools/taxonkit/taxonkit_profile2cami.xml

+    ]]>
+    </command>
+    <inputs>
+        <param name="input_file" type="data" format="txt" label="Input Profile File" help="Input the tab-delimited profile file with TaxId and abundance columns." />


Suggested change

<param name="input_file" type="data" format="txt" label="Input Profile File" help="Input the tab-delimited profile file with TaxId and abundance columns." />

<param name="input_file" type="data" format="txt" label="Input Profile File" help="A tab-delimited profile file with TaxId and abundance columns." />

paulzierep · 2024-07-02T10:27:42Z

tools/taxonkit/taxonkit_profile2cami.xml

+    </command>
+    <inputs>
+        <param name="input_file" type="data" format="txt" label="Input Profile File" help="Input the tab-delimited profile file with TaxId and abundance columns." />
+        <param argument="--taxonomy" type="select" label="NCBI taxonomy" help="To have actual human-readable taxon names in the standardised output">


Suggested change

<param argument="--taxonomy" type="select" label="NCBI taxonomy" help="To have actual human-readable taxon names in the standardised output">

<param argument="--taxonomy" type="select" label="NCBI taxonomy" help="This NCBI database is used to map human-readable taxon names to TaxId's.">

paulzierep · 2024-07-02T10:29:57Z

tools/taxonkit/taxonkit_profile2cami.xml

+
+        Profile2CAMI is a tool for converting metagenomic profile tables to CAMI format.
+
+        **Inputs**


All info that is given in the input parameters can be removed from the help

paulzierep · 2024-07-02T11:55:36Z

tools/taxonkit/taxonkit_profile2cami.xml

+        <param name="keep_zero" type="boolean" value="false" label="Keep Zero Abundances" help="Check to keep taxons with abundance of zero." />
+        <param name="sample_id" type="text" value="" label="Sample ID" help="Optional sample ID to include in the result file." />
+        <param name="taxonomy_id" type="text" value="" label="Taxonomy ID" help="Optional taxonomy ID to include in the result file." />
+        <param name="show_rank" type="text" value="superkingdom,phylum,class,order,family,genus,species,strain" label="Show Ranks" help="Specify the ranks to show in the result file." />


Can you use a type="select" and multiple="true" param for this ´, like this one:

<param name="show_rank" type="select" multiple="true" label="Show Ranks"> <option value="superkingdom">Superkingdom</option> <option value="phylum">Phylum</option> <option value="class">Class</option> <option value="order">Order</option> <option value="family">Family</option> <option value="genus">Genus</option> <option value="species">Species</option> <option value="strain">Strain</option> </param>

Make all options

paulzierep · 2024-07-02T13:36:24Z

When investigating further, I think this tool does currently not work as expected: shenwei356/taxonkit#99
Before they look into the issue, I would wait with merging it ...

shenwei356 · 2024-07-03T14:29:12Z

Hi guys, a new TaxonKit version is ready for this: bioconda/bioconda-recipes#48917

bgruening · 2024-07-06T16:57:06Z

tools/taxonkit/tool_data_table_conf.xml.test

+        <!-- Locations of taxonomy data downloaded from NCBI -->
+    <table name="ncbi_taxonomy" comment_char="#">
+        <columns>value, name, path</columns>
+        <file path="${__HERE__}/test-data/ncbi_taxonomy.loc" />


Suggested change

<file path="${__HERE__}/test-data/ncbi_taxonomy.loc" />

<file path="${__HERE__}/test-data/ncbi_taxonomy.loc.test" />

bgruening · 2024-07-06T16:57:32Z

tools/taxonkit/tool-data/ncbi_taxonomy.loc.sample

@@ -0,0 +1,2 @@
+#value	name	path


and this file should be renamed to .loc.test

@bgruening thank you for your review. I hope the changes now align with your suggestions.

bgruening

Thanks! LGTM!

paulzierep

with this changes (only cosmetics) its good for merge from my side

paulzierep · 2024-07-08T07:31:57Z

tools/taxonkit/tool_data_table_conf.xml.sample

+        <!-- Locations of taxonomy data downloaded from NCBI -->
+    <table name="ncbi_taxonomy" comment_char="#">
+        <columns>value, name, path</columns>
+        <file path="${__HERE__}/test-data/ncbi_taxonomy.loc.sample" />


Suggested change

<file path="${__HERE__}/test-data/ncbi_taxonomy.loc.sample" />

<file path="tool-data/ncbi_taxonomy.loc" />

afaik, the ${__HERE__} variable is only used in tests and the real data will be stored in the tool-data folder, @bgruening can you confirm that ?
Also the path should point to the .loc file not the sample.

paulzierep · 2024-07-08T07:33:11Z

tools/taxonkit/tool-data/ncbi_taxonomy.loc.sample

@@ -0,0 +1,2 @@
+#value	name	path
+test-db-tox	"Test Database"  ${__HERE__}/test-db


Suggested change

test-db-tox "Test Database" ${__HERE__}/test-db

test-db-tox "Test Database" tool-deta/test-db

Same as the other comment, although the real path does not really matter, since the DM sets it

bernt-matthias · 2024-07-08T07:44:19Z

tools/taxonkit/tool-data/ncbi_taxonomy.loc.sample

@@ -0,0 +1,2 @@
+#value	name	path
+test-db-tox	"Test Database"  ${__HERE__}/test-db


Suggested change

test-db-tox "Test Database" ${__HERE__}/test-db

Line should be removed. The .sample files and their content would be used.

removed or commented ?, in most other tools there is a comment showing how it looks like

commented is also fine .. an example is a good thing to have

paulzierep

small change required

paulzierep · 2024-07-08T07:56:08Z

tools/taxonkit/taxonkit_profile2cami.xml

+        #if $taxonomy_id:
+            -t '${taxonomy_id}'
+        #end if
+        --show-rank '${ranks}'


can you put this in an if condition, so that if nothing is selected, all ranks are used as is the default by the tool ? also, please adjust the test for it, that's more convenient for the user

paulzierep · 2024-07-09T12:22:54Z

Good to merge from my side @bgruening @bernt-matthias ?

…xyproject#6085) * Added the profile2cami tool, a component of the TaxonKit suite. * Renamed shed.yml -> .shed.yml * Shrinked delnodes.dmp 684kb -> 150kb * Renamed test.loc -> ncbi_taxonomy.loc * Adjusted taxonkit, removed folder profile2cami, added suite in .shed.yml * Updated Taxonkit to version 0.17.0 - worked on issues. * Renamed taxonomy.loc -> taxonomy.loc.test * Adjusted test, adjusted loc.sample * Try to fix linting error * Resolving linting error * Adjusted the output label.

Alby-Git added 2 commits June 14, 2024 09:57

Added the profile2cami tool, a component of the TaxonKit suite.

33a53eb

Renamed shed.yml -> .shed.yml

461a8a7

bgruening reviewed Jun 14, 2024

View reviewed changes

Alby-Git added 2 commits June 17, 2024 11:20

Shrinked delnodes.dmp 684kb -> 150kb

3db4745

Renamed test.loc -> ncbi_taxonomy.loc

d15df70

bernt-matthias mentioned this pull request Jun 20, 2024

Added Cami Opal - A tool for evaluating taxonomic metagenome profilers #6096

Open

5 tasks

paulzierep reviewed Jun 20, 2024

View reviewed changes

Adjusted taxonkit, removed folder profile2cami, added suite in .shed.yml

49cc76e

paulzierep reviewed Jul 2, 2024

View reviewed changes

paulzierep mentioned this pull request Jul 3, 2024

profile2cami not working as expected shenwei356/taxonkit#99

Open

Updated Taxonkit to version 0.17.0 - worked on issues.

bb7d131

bgruening reviewed Jul 6, 2024

View reviewed changes

Renamed taxonomy.loc -> taxonomy.loc.test

ef148fa

bgruening approved these changes Jul 7, 2024

View reviewed changes

paulzierep reviewed Jul 8, 2024

View reviewed changes

bernt-matthias reviewed Jul 8, 2024

View reviewed changes

paulzierep reviewed Jul 8, 2024

View reviewed changes

Alby-Git added 3 commits July 8, 2024 10:46

Adjusted test, adjusted loc.sample

4a88efd

Try to fix linting error

c66c0ed

Resolving linting error

71fc99f

bgruening approved these changes Jul 9, 2024

View reviewed changes

SantaMcCloud mentioned this pull request Jul 13, 2024

Add name2taxid function from taxonkit #6146

Merged

5 tasks

Adjusted the output label.

079f9b4

bgruening merged commit 695ea58 into galaxyproject:main Jul 26, 2024
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added the profile2cami tool, a component of the TaxonKit suite. #6085

Added the profile2cami tool, a component of the TaxonKit suite. #6085

Alby-Git commented Jun 14, 2024

bgruening Jun 14, 2024

Alby-Git Jun 17, 2024

paulzierep Jun 20, 2024

bgruening Jun 14, 2024

Alby-Git Jun 17, 2024 •

edited

Loading

bgruening Jun 14, 2024

Alby-Git Jun 17, 2024

bgruening left a comment

paulzierep left a comment

paulzierep Jun 20, 2024

paulzierep left a comment

paulzierep Jul 2, 2024

paulzierep Jul 2, 2024

paulzierep Jul 2, 2024

paulzierep Jul 2, 2024

paulzierep Jul 2, 2024

paulzierep Jul 2, 2024

paulzierep Jul 2, 2024

paulzierep commented Jul 2, 2024

shenwei356 commented Jul 3, 2024

bgruening Jul 6, 2024

bgruening Jul 6, 2024

Alby-Git Jul 7, 2024

bgruening left a comment

paulzierep left a comment

paulzierep Jul 8, 2024 •

edited

Loading

paulzierep Jul 8, 2024

bernt-matthias Jul 8, 2024

paulzierep Jul 8, 2024 •

edited

Loading

bernt-matthias Jul 8, 2024

paulzierep left a comment

paulzierep Jul 8, 2024 •

edited

Loading

paulzierep commented Jul 9, 2024

	<param name="input_file" type="data" format="txt" label="Input Profile File" help="Input the tab-delimited profile file with TaxId and abundance columns." />
	<param name="input_file" type="data" format="txt" label="Input Profile File" help="A tab-delimited profile file with TaxId and abundance columns." />

	<param argument="--taxonomy" type="select" label="NCBI taxonomy" help="To have actual human-readable taxon names in the standardised output">
	<param argument="--taxonomy" type="select" label="NCBI taxonomy" help="This NCBI database is used to map human-readable taxon names to TaxId's.">


		Profile2CAMI is a tool for converting metagenomic profile tables to CAMI format.

		Inputs

	<file path="${__HERE__}/test-data/ncbi_taxonomy.loc" />
	<file path="${__HERE__}/test-data/ncbi_taxonomy.loc.test" />

	<file path="${__HERE__}/test-data/ncbi_taxonomy.loc.sample" />
	<file path="tool-data/ncbi_taxonomy.loc" />

		@@ -0,0 +1,2 @@
		#value name path
		test-db-tox "Test Database" ${__HERE__}/test-db

	test-db-tox "Test Database" ${__HERE__}/test-db
	test-db-tox "Test Database" tool-deta/test-db

Added the profile2cami tool, a component of the TaxonKit suite. #6085

Added the profile2cami tool, a component of the TaxonKit suite. #6085

Conversation

Alby-Git commented Jun 14, 2024

Tool details for profile2cami

Checklist for submission

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Alby-Git Jun 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bgruening left a comment

Choose a reason for hiding this comment

paulzierep left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulzierep left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulzierep commented Jul 2, 2024

shenwei356 commented Jul 3, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bgruening left a comment

Choose a reason for hiding this comment

paulzierep left a comment

Choose a reason for hiding this comment

paulzierep Jul 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulzierep Jul 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulzierep left a comment

Choose a reason for hiding this comment

paulzierep Jul 8, 2024 • edited Loading

Choose a reason for hiding this comment

paulzierep commented Jul 9, 2024

Alby-Git Jun 17, 2024 •

edited

Loading

paulzierep Jul 8, 2024 •

edited

Loading

paulzierep Jul 8, 2024 •

edited

Loading

paulzierep Jul 8, 2024 •

edited

Loading