Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding parentheses to directory structure when using bin #1549

Open
avsastry opened this issue Mar 31, 2020 · 7 comments · May be fixed by #5193
Open

Adding parentheses to directory structure when using bin #1549

avsastry opened this issue Mar 31, 2020 · 7 comments · May be fixed by #5193
Labels
pinned

Comments

@avsastry
Copy link

Bug report

Expected behavior and actual behavior

I'm currently working within a Dropbox folder that is auto-named as "Dropbox (UCSD)". The presence of these parentheses seem to be properly accounted for in all cases, except when I'm trying to run a script from the "bin" folder. In the .command.run file, I see the following code:

nxf_container_env() {
cat << EOF 
export PATH="/home/anand/Dropbox (UCSD)/Projects/nextflow_test/bin:\$PATH"
EOF
}

It seems that the export call to use the "bin" directory doesn't backslash the parentheses and the spaces, resulting in a run error:

  /bin/bash: eval: line 0: syntax error near unexpected token `('
  /bin/bash: eval: line 0: `export PATH=/home/anand/Dropbox (UCSD)/Projects/nextflow_test/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'

Steps to reproduce the problem

I can attach a working version if necessary

Program output

Mar-31 13:26:47.561 [main] DEBUG nextflow.cli.Launcher - $> nextflow run main.nf
Mar-31 13:26:47.710 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 20.01.0
Mar-31 13:26:47.794 [main] INFO  nextflow.cli.CmdRun - Launching `main.nf` [romantic_kirch] - revision: 4a8194bef6
Mar-31 13:26:48.151 [main] DEBUG nextflow.Session - Session uuid: f9bd066a-4c03-468f-b137-0a79fcc07474
Mar-31 13:26:48.151 [main] DEBUG nextflow.Session - Run name: romantic_kirch
Mar-31 13:26:48.154 [main] DEBUG nextflow.Session - Executor pool size: 8
Mar-31 13:26:48.290 [main] DEBUG nextflow.cli.CmdRun -
  Version: 20.01.0 build 5264
  Created: 12-02-2020 10:14 UTC (02:14 PDT)
  System: Linux 4.15.0-91-generic
  Runtime: Groovy 2.5.8 on OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08
  Encoding: UTF-8 (UTF-8)
  Process: 12645@sbrg-sastry [127.0.1.1]
  CPUs: 8 - Mem: 31.3 GB (23.2 GB) - Swap: 31.9 GB (31.9 GB)
Mar-31 13:26:48.470 [main] DEBUG nextflow.file.FileHelper - Can't check if specified path is NFS (1): /home/anand/Dropbox (UCSD SBRG)/Projects/dev_testing/test (folder)/work
  ext2/ext3

Mar-31 13:26:48.470 [main] DEBUG nextflow.Session - Work-dir: /home/anand/Dropbox (UCSD SBRG)/Projects/dev_testing/test (folder)/work [null]
Mar-31 13:26:48.529 [main] DEBUG nextflow.Session - Observer factory: TowerFactory
Mar-31 13:26:48.532 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Mar-31 13:26:48.981 [main] DEBUG nextflow.Session - Session start invoked
Mar-31 13:26:49.653 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Mar-31 13:26:49.679 [main] DEBUG nextflow.Session - Workflow process names [dsl1]: use_bin_folder
Mar-31 13:26:49.741 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Mar-31 13:26:49.741 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Mar-31 13:26:49.747 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Mar-31 13:26:49.754 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=8; memory=31.3 GB; capacity=8; pollInterval=100ms; dumpInterval=5m
Mar-31 13:26:49.904 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > use_bin_folder -- maxForks: 8; blocking: false
Mar-31 13:26:49.955 [main] DEBUG nextflow.script.ScriptRunner - > Await termination 
Mar-31 13:26:49.957 [main] DEBUG nextflow.Session - Session await
Mar-31 13:26:50.235 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Mar-31 13:26:50.242 [Task submitter] INFO  nextflow.Session - [cf/dd5d60] Submitted process > use_bin_folder
Mar-31 13:26:50.332 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: use_bin_folder; status: COMPLETED; exit: 127; error: -; workDir: /home/anand/Dropbox (UCSD SBRG)/Projects/dev_testing/test (folder)/work/cf/dd5d60f054d580a87bd60f9081f050]
Mar-31 13:26:50.380 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'use_bin_folder'

Caused by:
  Process `use_bin_folder` terminated with an error exit status (127)

Command executed:

  ./run_test.sh

Command exit status:
  127

Command output:
  (empty)

Command error:
  .command.sh: line 2: ./run_test.sh: No such file or directory

Work dir:
  /home/anand/Dropbox (UCSD SBRG)/Projects/dev_testing/test (folder)/work/cf/dd5d60f054d580a87bd60f9081f050

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
Mar-31 13:26:50.391 [main] DEBUG nextflow.Session - Session await > all process finished
Mar-31 13:26:50.391 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `use_bin_folder` terminated with an error exit status (127)
Mar-31 13:26:50.486 [main] DEBUG nextflow.Session - Session await > all barriers passed
Mar-31 13:26:50.497 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=0ms; failedDuration=54ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=1; peakCpus=1; peakMemory=0; ]
Mar-31 13:26:50.607 [main] DEBUG nextflow.CacheDB - Closing CacheDB done
Mar-31 13:26:50.620 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

### Environment 

* Nextflow version: 20.01.0.5264
* Java version: 1.8.0_242
* Operating system: Linux
* Bash version: 4.4.20

### Additional context
@stale
Copy link

stale bot commented Aug 28, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Aug 28, 2020
@pditommaso
Copy link
Member

keep open

@stale stale bot removed the stale label Oct 12, 2020
@stale
Copy link

stale bot commented Mar 11, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Mar 11, 2021
@stale stale bot closed this as completed May 12, 2021
@pditommaso pditommaso reopened this May 12, 2021
@stale stale bot removed the stale label May 12, 2021
@stale stale bot removed the stale label May 12, 2021
@hukai916
Copy link

hukai916 commented Aug 1, 2024

Same bug for me. Any update on it?

@bentsherman
Copy link
Member

Maybe the bin path should be escaped with Escape.path() in addition to Escape.variable():

nextflow/modules/nextflow/src/main/groovy/nextflow/processor/TaskProcessor.groovy

Lines 2090 to 2091 in 2fb5bc0

// escape both wrapping double quotes and the dollar var placeholder
script << /export $name="${Escape.variable(value)}"/

@hukai916
Copy link

hukai916 commented Aug 1, 2024

Thanks @bentsherman, these headaches are all caused by the stupid decision made by the Dropbox team to include spaces in folder names. The solution (link below) provided by rwmorris solves the headache. Hope it helps someone else.

https://www.dropboxforum.com/t5/Plans-and-Subscriptions/How-to-remove-spaces-from-Dropbox-folder-names-spaces-causing/td-p/191195

@pditommaso
Copy link
Member

@bentsherman worth to give a try to it

@bentsherman bentsherman linked a pull request Aug 1, 2024 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
pinned
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix unescaped spaces and parentheses in container env script nextflow-io/nextflow
4 participants
@pditommaso @bentsherman @avsastry @hukai916