-
Notifications
You must be signed in to change notification settings - Fork 63
Tutorial
- Intro
- Getting help
- Preparing the alignment
- Tree inference
- Bootstrapping
- Tree likelihood evaluation
- Other commands
RAxML-NG replaces standard RAxML as well as the corresponding supercomputer version ExaML. So RAxML-NG is one single code base that scales from the laptop to the supercomputer. RAxML-NG does not (yet) support all options of standard RAxML, only the most important and frequently used ones. Some options are now implemented as stand-alone tools, e.g. phylogenetic placement (EPA).
This tutorial is based on RAxML-NG practical taught by Alexandros Stamatakis at COME 2018. It will cover most common use cases, for information on advanced usage see next section.
Before you start:
- Download and install RAxML-NG (instructions)
- Download a toy dataset:
If you run RAxML-NG executable without parameters, it will show quick usage help:
RAxML-NG v. 0.7.0git BETA released on 21.11.2018 by The Exelixis Lab.
Authors: Alexey Kozlov, Alexandros Stamatakis, Diego Darriba, Tomas Flouri, Benoit Morel.
Latest version: https://github.com/amkozlov/raxml-ng
Questions/problems/suggestions? Please visit: https://groups.google.com/forum/#!forum/raxml
WARNING: This is a BETA release, please use at your own risk!
Usage: raxml-ng [OPTIONS]
Commands (mutually exclusive):
--help display help information
--version display version information
--evaluate evaluate the likelihood of a tree (with model+brlen optimization)
--search ML tree search.
--bootstrap bootstrapping
--all all-in-one (ML search + bootstrapping).
--support compute bipartition support for a given reference tree (e.g., best ML tree)
and a set of replicate trees (e.g., from a bootstrap analysis)
--bsconverge test for bootstrapping convergence using autoMRE criterion
--terrace check whether a tree lies on a phylogenetic terrace
--check check alignment correctness and remove empty columns/rows
--parse parse alignment, compress patterns and create binary MSA file
--start generate parsimony/random starting trees and exit
--loglh compute the likelihood of a fixed tree (no model/brlen optimization)
Input and output options:
[...]
More comprehensive documentation is available in GitHub wiki. Further information and benchmarks can be found in biorxiv preprint and in Chapter 4 of Alexey's PhD thesis.
If you cannot find an answer to your question in the above sources, or if you think you found a bug, please contact us via RAxML google group.
- Please use search function before posting, since many questions have been answered before.
- Please use google group and not personal e-mail for asking questions about RAxML-NG. This will save everybody's time: you might get help sooner from other Exelixis lab members or your fellow users. And whoever might encounter the same problem in the future will benefit from the answer.
Before we get started, let's first check that the MSA can actually be read and doesn't contain sites with only undetermined characters or sequences with undetermined characters or duplicate taxon names, etc. etc.
raxml-ng --check --msa prim.phy --model GTR+G --prefix T1
Doing this check before getting started is super-important as more than 50% of all failed RAxML runs are due to tree or MSA format errors!
We will always also use --prefix to avoid over-writing previous output files.
For large alignments, we also recommend using --parse command after (or instead of) --check:
raxml-ng --parse --msa prim.phy --model GTR+G --prefix T2
In addition to MSA sanity check, this command will perform two useful operations:
- Compress alignment patterns and store MSA in the binary format (RAxML Binary Alignment, RBA):
NOTE: Binary MSA file created: T2.raxml.rba
Since pattern compression could take quite some time for large MSAs, loading RBA file is (much) faster compared to FASTA or PHYLIP.
- Estimate memory requirements and optimal number of CPUs/threads (see Parallelization section for details)
* Estimated memory requirements : 2 MB
* Recommended number of threads / MPI processes: 2
Now let's infer a tree under GTR+GAMMA with default parameters. We will use 2 threads as suggested above, and set a fixed random seed to ensure reproducibility:
raxml-ng --msa prim.phy --model GTR+G --prefix T3 --threads 2 --seed 2
This command will perform 20 tree searches using 10 random and 10 parsimony-based starting trees, and pick the best-scoring topology:
Analysis options:
run mode: ML tree search
start tree(s): random (10) + parsimony (10)
This default settings is a reasonable choice for many practical cases. However, computational resources permitting, we might want to increase the number of starting trees to explore the tree space more thoroughly:
raxml-ng --msa prim.phy --model GTR+G --prefix T4 --threads 2 --seed 2 --tree pars{25},rand{25}
Conversely, we can perform a quick-and-dirty search from a single random starting tree using the --search1 command:
raxml-ng --search1 --msa prim.phy --model GTR+G --prefix T5 --threads 2 --seed 2
Let's compare the results of all three runs:
grep "Final LogLikelihood:" T{3,4,5}.raxml.logT3.raxml.log:Final LogLikelihood: -5708.923977
T4.raxml.log:Final LogLikelihood: -5708.923977
T5.raxml.log:Final LogLikelihood: -5708.979717
This looks quite good: likelihood surface seems to have a clear peak, which we can find regardless of the search parameters. However, T5 has a slightly worse likelihood, does it mean it has a distinct topology? Let's check it using the old RAxML for now:
$ cat T{3,4}.raxml.mlTrees T5.raxml.bestTree > mltrees
$ raxmlHPC-AVX -m GTRGAMMA -f r -z mltrees -n RF[...]
Found 71 trees in File mltrees
Average relative RF in this set: 0.000000
Number of unique trees in this tree set: 1
No, in fact all 71 resulting topologies (one per starting tree) are identical, so we can be pretty sure that we found the globally optimal ML tree.
However, not all datasets are so well-behaved:
$ raxml-ng --msa fusob.phy --model GTR+G --prefix T6 --seed 2 --threads 2
$ grep "ML tree search #" T6.raxml.log[00:00:03] ML tree search #1, logLikelihood: -9974.668088
[00:00:07] ML tree search #2, logLikelihood: -9974.666644
[00:00:11] ML tree search #3, logLikelihood: -9974.669417
[00:00:15] ML tree search #4, logLikelihood: -9974.664855
[00:00:19] ML tree search #5, logLikelihood: -9974.663779
[00:00:22] ML tree search #6, logLikelihood: -9974.666906
[00:00:26] ML tree search #7, logLikelihood: -9974.668155
[00:00:30] ML tree search #8, logLikelihood: -9974.664340
[00:00:33] ML tree search #9, logLikelihood: -9974.666937
[00:00:37] ML tree search #10, logLikelihood: -9974.666388
[00:00:40] ML tree search #11, logLikelihood: -9980.601114
[00:00:43] ML tree search #12, logLikelihood: -9974.675123
[00:00:46] ML tree search #13, logLikelihood: -9980.602470
[00:00:49] ML tree search #14, logLikelihood: -9974.671637
[00:00:52] ML tree search #15, logLikelihood: -9980.602668
[00:00:54] ML tree search #16, logLikelihood: -9980.601182
[00:00:57] ML tree search #17, logLikelihood: -9974.672801
[00:01:00] ML tree search #18, logLikelihood: -9974.668668
[00:01:03] ML tree search #19, logLikelihood: -9974.669997
[00:01:06] ML tree search #20, logLikelihood: -9980.607281
Here, we see why it is so important to use multiple starting trees: some searches converged to a local optimum with a much lower likelihood (-9980.607281 vs. -9974.669997).
NOTE: As of v. 0.7.0b, RAxML-NG supports only standard/slow bootstrap algorithm (-b option in RAxML8). It is much slower than rapid bootstrapping implemented in RAxML8 (-x or -f a options), but gives more accurate support values estimates.
RAxML-NG can perform standard non-parametric bootstrap by re-sampling alignment columns and re-inferring tree for each bootstrap replicate MSA:
raxml-ng --bootstrap --msa prim.phy --model GTR+G --prefix T7 --seed 2 --threads 2
By default, RAxML-NG will perform MRE-based bootstopping test after every 50 replicates, and terminate once the diagnostic statistics drops below the cutoff value:
bootstrap replicates: max: 1000 + bootstopping (autoMRE, cutoff: 0.030000)
[...]
[00:00:15] Bootstrap tree #50, logLikelihood: -5762.777409
[00:00:15] Bootstrapping converged after 50 replicates.
This was fast! Let's manually increase the number of replicates to be on the safe side:
raxml-ng --bootstrap --msa prim.phy --model GTR+G --prefix T8 --seed 2 --threads 2 --bs-trees 200
Bootstrap convergence can also be checked post-hoc using the --bsconverge command, we can also change the cutoff value to make the test more or less stringent:
raxml-ng --bsconverge --bs-trees T7.raxml.bootstraps --prefix T9 --seed 2 --threads 2 --bs-cutoff 0.01
# trees avg WRF avg WRF in % # perms: wrf <= 1.00 % converged?
50 7.400 1.644 0 NO
Bootstopping test did not converge after 50 trees
As we can see, with 1% cutoff 50 replicates are not enough. What about 200?
raxml-ng --bsconverge --bs-trees T8.raxml.bootstraps --prefix T10 --seed 2 --threads 2 --bs-cutoff 0.01
# trees avg WRF avg WRF in % # perms: wrf <= 1.00 % converged?
50 7.400 1.644 0 NO
100 11.702 1.300 245 NO
150 13.960 1.034 457 NO
200 16.484 0.916 648 NO
Bootstopping test did not converge after 200 trees
Still no convergence, but weighted RF distance (avg WRF in %) is steadily decreasing as we add more replicates, and now lies below the 1% cutoff for 648 out of 1000 permutations (convergence requirement: >990/1000). This looks promising, and we can expect convergence after few hundreds replicates. Luckily, bootstraps are independent, thus we can reuse 200 bootstrap trees we have already inferred. So let's add 400 more replicate trees. It is however important to specify a distinct random seed for the second run, otherwise first 200 trees will be identical to the first batch!
raxml-ng --bootstrap --msa prim.phy --model GTR+G --prefix T11 --seed 333 --threads 2 --bs-trees 400
Now, we simply concatenate replicate trees from both runs, and re-assess the convergence:
$ cat T8.raxml.bootstraps T11.raxml.bootstraps > allbootstraps
$ raxml-ng --bsconverge --bs-trees allbootstraps --prefix T12 --seed 2 --threads 1 --bs-cutoff 0.01
# trees avg WRF avg WRF in % # perms: wrf <= 1.00 % converged?
50 7.400 1.644 0 NO
100 11.702 1.300 245 NO
150 13.960 1.034 457 NO
200 16.484 0.916 648 NO
250 17.410 0.774 841 NO
300 18.900 0.700 927 NO
350 20.060 0.637 942 NO
400 22.076 0.613 969 NO
450 23.856 0.589 973 NO
500 26.164 0.581 985 NO
550 27.844 0.563 985 NO
600 28.462 0.527 991 YES
Bootstopping test converged after 600 trees
Perfect, now we have convergence even with a more stringent cutoff, although we had to conduct 600 replicate searches instead of just 50. On large datasets, this quickly gets very expensive computationally. Hence in practice, the default cutoff value of 3% should be sufficient in most cases (see Pattengale et al., 2010).
So now, what do we do with the bootstrap trees? We can either summarize them via some sort of consensus tree (strict, majority, majority rule extended, e.g. using standard RAxML or some other tool) or we can map them onto the best-scoring ML tree on the original MSA. It's debatable what might be the best way of doing it, but there seems to be a trend toward mapping the BS support values onto the best-scoring/best-known ML tree, so let's do that.
We will use the ML tree obtained in the run T3:
raxml-ng --support --tree T3.raxml.bestTree --bs-trees allbootstraps --prefix T13 --threads 2
Now we can actually look at the tree with supports,
T13.raxml.support
using some tree viewer.
Beware of the branch support visualization issues [https://academic.oup.com/mbe/article/34/6/1535/3077051] that can arise with the NEWICK format!!!
Alternatively, we can compute so-called Transfer Bootstrap Expectation support metric (Lemoine et al. 2018), which is supposedly more appropriate for very large trees:
raxml-ng --support --tree T3.raxml.bestTree --bs-trees allbootstraps --prefix T14 --threads 2 --bs-metric tbe
But wait, we could have been lazier:
raxml-ng --all --msa prim.phy --model GTR+G --prefix T15 --seed 2 --threads 2 --bs-metric fbp,tbe
This will do all of the above steps (20 ML inferences on the original MSA, inferring bootstrap replicate trees, and drawing support values on the best-scoring tree) with just one command :-)
$ ls T15.*
T15.raxml.bestModel T15.raxml.bestTree T15.raxml.bootstraps T15.raxml.log T15.raxml.mlTrees
T15.raxml.rba T15.raxml.startTree T15.raxml.supportFBP T15.raxml.supportTBE
Another standard task is to evaluate trees, i.e., compute the likelihood of a given fixed tree topology by just optimizing model and/or branch length parameters on that fixed tree. This is frequently needed in model and hypothesis testing :-)
The basic option is --evaluate
With --opt-model on/off you can enable/disable model parameter optimization.
With --opt-branches on/off you can enable/disable branch length optimization.
Let's do some small tests that also show how the likelihood improves as we add more and more free parameters to our model. We will use the best-scoring ML tree again:
Let's first evaluate it under the most simple model, Jukes-Cantor (JC):
raxml-ng --evaluate --msa test.fa --threads 1 --model JC --tree T9.raxml.bestTree --prefix E1
Now, let's add rate heterogeneity to this:
raxml-ng --evaluate --msa test.fa --threads 1 --model JC+G -tree T9.raxml.bestTree --prefix E2
Now let's take a simple GTR model (without rate heterogeneity):
raxml-ng --evaluate --msa test.fa --threads 1 --model GTR --tree T9.raxml.bestTree --prefix E3
GTR with the Gamma model of rate heterogeneity, but empirical base frequencies:
raxml-ng --evaluate --msa test.fa --threads 1 --model GTR+G+FC --tree T9.raxml.bestTree --prefix E4
And now also doing a ML estimate of the base frequencies. How many more free parameters do we get?.
raxml-ng --evaluate --msa test.fa --threads 1 --model GTR+G+FO --tree T9.raxml.bestTree --prefix E5
Let's check the results:
grep logLikelihood E*.raxml.log
E1.raxml.log:[00:00:00] Tree #1, final logLikelihood: -4444.084375 <- JC
E2.raxml.log:[00:00:00] Tree #1, final logLikelihood: -4270.170317 <- JC+GAMMA
E3.raxml.log:[00:00:00] Tree #1, final logLikelihood: -4280.099457 <- GTR
E4.raxml.log:[00:00:00] Tree #1, final logLikelihood: -4075.205410 <- GTR + GAMMA + empirical base freqs
E5.raxml.log:[00:00:00] Tree #1, final logLikelihood: -4069.207897 <- GTR + GAMMA + estimated base freqs