Project currently is stuck on revision xxx when using -latest flag

[marcadella]

Bug report

I get the error message: Project xxx currently is sticked on revision: yyy -- you need to specify explicitly a revision with the option -r to use it, even though I am using the -latest flag when running nextflow run.

There is no reason why Nextflow would not pull the latest commit of the master branch and use it when -r has been used previously.

Environment

• Nextflow version: 20.04.1.5335
• Java version: 11.0.7
• Operating system: Linux

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[thorellk]

I get this error message as well when running NEXTFLOW version 20.10.0 on our Linux HPC.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[esrice]

I am having this issue as well. Should the default behavior in the absence of any flags not be to use the latest version of the default branch on github? Pulling a repository for the first time, these two commands:

nextflow run WarrenLab/longread-polish-nf [pipeline-specific parameters]
nextflow run WarrenLab/longread-polish-nf -latest [pipeline-specific parameters]

both crash with the error:

N E X T F L O W  ~  version 20.10.0
Pulling WarrenLab/longread-polish-nf ...
downloaded from https://github.com/WarrenLab/longread-polish-nf.git
Project `WarrenLab/longread-polish-nf` currently is sticked on revision: main -- you need to specify explicitly a revision with the option `-r` to use it

Whereas adding the flag -r main works. Should it look for a 'main' or 'master' branch by default? Is this perhaps an issue with github switching from using 'master' to 'main' as the default branch name?

[wbazant]

I've had this problem when pulling my own pipeline, which only has one branch and I didn't mind what I named it. Could 'main' be an okay default for pulling?

Being "sticked on revision" isn't even correct English but amazingly it helped me find this issue!

[krokicki]

I'm running into this issue on tower.nf, with the "pull latest" option set. I suspect that @esrice is correct that it has to do with the new 'main' branch, since I only have this issue with newer repos and not any that use 'master'.

[esrice]

@krokicki I looked into this more after writing that comment, and IIRC it is indeed an issue with 'master' being hard-coded into some code somewhere. I don't know enough groovy to write an elegant solution for this myself. However, I get around this in my own projects by either doing '-latest -r main' on the command line, or by adding this line to nextflow.config:

manifest.defaultBranch = 'main'

Hope this helps!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tjdurant]

I am also having an issue with this.

We use an enterprise instance of GitLab where the default branch is called 'main'.

If make a brand-new, empty repo, only initialized with a ReadMe. That is 'commit 1'.

Then I commit my 'hello world' main.nf, and this is 'commit 2'.

main.nf:

#!/usr/bin/env nextflow

nextflow.enable.dsl=1

params.greeting  = 'Hello world!' 
greeting_ch = Channel.from(params.greeting) 

process splitLetters { 

    container = "<URL to image in private gitlab registry>"

    input: 
    val x from greeting_ch 

    output: 
    file 'chunk_*' into letters_ch 

    """ 
    printf '$x' | split -b 6 - chunk_  
    """ 
} 

process convertToUpper { 

    container = "<URL to image in private gitlab registry>"

    input:  
    file y from letters_ch.flatten() 

    output: 
    stdout into result_ch 

    """ 
    cat $y | tr '[a-z]' '[A-Z]'  
    """ 
} 

result_ch.view{ it }  

If I run without specifying the revision:

❯ nextflow run <URL to code repo>
N E X T F L O W  ~  version 22.04.0
Project `<project name>` currently is sticked on revision: main -- you need to specify explicitly a revision with the option `-r` to use it

Then I specify the revision with 'main' and it works:

❯ nextflow run <URL to code repo> -r main
N E X T F L O W  ~  version 22.04.0
Launching `<URL to private gitlab repo>` [extravagant_turing] DSL1 - revision: <commit 2 SHA> [main]
executor >  local (3)
[1d/682ad8] process > splitLetters (1)   [100%] 1 of 1 ✔
[8d/554e41] process > convertToUpper (2) [100%] 2 of 2 ✔
HELLO
WORLD!

Then I specify the revision with the commit 2 SHA and it works:

❯ nextflow run <URL to code repo> -r <commit 2 SHA>
N E X T F L O W  ~  version 22.04.0
Launching `<URL to private gitlab repo>` [extravagant_turing] DSL1 - revision: <commit 2 SHA>
executor >  local (3)
[1d/682ad8] process > splitLetters (1)   [100%] 1 of 1 ✔
[8d/554e41] process > convertToUpper (2) [100%] 2 of 2 ✔
HELLO
WORLD!

But, if I make a change to the code, take the exclamation point out, re-commit that, which is now 'commit 3' and run that with -r main:

❯ nextflow run <URL to code repo> -r main
N E X T F L O W  ~  version 22.04.0
NOTE: Your local project version looks outdated - a different revision is available in the remote repository [<commit 3 SHA>]
Launching `<URL to code repo>` [chaotic_meucci] DSL1 - revision: <commit 2 SHA> [main]
executor >  local (3)
[c7/a4d444] process > splitLetters (1)   [100%] 1 of 1 ✔
[54/e7ea47] process > convertToUpper (2) [100%] 2 of 2 ✔
HELLO
WORLD!

I see that nextflow is running commit 2 and the exclamation point is still there.

So, I try it with the commit 3 SHA:

❯ nextflow run <URL to code repo> -r <commit 3 SHA>
N E X T F L O W  ~  version 22.04.0
Unknown error accessing project `<project name>` -- Repository may be corrupted: <user path>/.nextflow/assets/<gitlab group>/<gitlab project name>

I then go and delete this folder on my local machine:

<user path>/.nextflow/assets/<gitlab group>/<gitlab project name>

re-run this:

nextflow run <URL to code repo> -r <commit 3 SHA>

And it works again:

❯ nextflow run <URL to code repo> -r <commit 3 SHA>
N E X T F L O W  ~  version 22.04.0
Pulling <project name> ...
 downloaded from <URL to code repo>
Launching `<URL to code repo>` [spontaneous_cuvier] DSL1 - revision: <commit 3 SHA>
executor >  local (3)
[26/096929] process > splitLetters (1)   [100%] 1 of 1 ✔
[fc/d1180f] process > convertToUpper (2) [100%] 2 of 2 ✔
HELLO
WORLD

And if i run again with -r main:

❯ nextflow run <URL to code repo> -r main
N E X T F L O W  ~  version 22.04.0
Launching `<URL to code repo>` [furious_bohr] DSL1 - revision: <commit 3 SHA> [main]
executor >  local (3)
[b8/1bd952] process > splitLetters (1)   [100%] 1 of 1 ✔
[3f/fc8dbb] process > convertToUpper (1) [100%] 2 of 2 ✔
WORLD
HELLO

So, it seems there are a few things here:

1. The -r main can point to previous commits if the commit SHA isn't specified.
2. The commit SHA can be used but can run into conflicts with local copies of the repo.

Just wanted to share this troubleshooting in case it is helpful for anyone who is experiencing something similar and to stimulate any additional discussion on this topic/issue if needed.

Happy to provide more information if needed.

Thanks,
Tom

Environment

• Nextflow version: 22.04.0

• Java version:
  openjdk version "11.0.13" 2021-10-19
  OpenJDK Runtime Environment JBR-11.0.13.7-1751.21-jcef (build 11.0.13+7-b1751.21)
  OpenJDK 64-Bit Server VM JBR-11.0.13.7-1751.21-jcef (build 11.0.13+7-b1751.21, mixed mode)

• Local Operating system: macOS

• Local Bash version: zsh 5.8 (x86_64-apple-darwin20.0)

[alanhoyle]

I'm still seeing this on version 22.10.6 build 5843

[pditommaso]

I believe so! has not been fixed yet!

[robsyme]

I can try and help improve the error message and perhaps also the documentation. Before I do so, I'd like to clarify the reasons for the behaviour.

My understanding is that:

1. If you running with the revision argument -r 3.0 is a very deliberate action
2. If you run the same workflow without that argument in the future, Nextflow is preventing you from running because it suspects that always want to run with this version and Nextflow supsects that you've simply forgotten the -r.

Is that correct? Is the "stickied" descriptor a Nextflow-specific term or a little-known git term?

[pditommaso]

We are planning to solve this limitation #2870 (comment)