Deploy: Remove untracked files from project folder

[tangrufus]

No description provided.

[swalkinshaw]

🤔 don't deploys fail right now if there's untracked files?

If so, this would silently delete files and allow them to succeed?

[tangrufus]

don't deploys fail right now if there's untracked files?

No, untracked files just being ignored by git archive and git clone / git reset tasks.

don't deploys fail right now if there's tracked files?

No longer failling after #999 (comment)

If so, this would silently delete files and allow them to succeed?

Untracked files are actually okay to exist. They don't make a difference.
Example: DEPLOY_UNFINISHED before #1145

Adding git clean ensures we always have the same source (idempotence).

[swalkinshaw]

Ah yeah forgot about that force change.

Makes sense 👍

[tjedrak]

Guys, I got an error on that task:
(I'm sorry if it's not the right place for pointing bugs)
Env: Mac os Catalina 10.15.2
Vagrant 2.2.7
ansible 2.8.4
VirtualBox [>= 4.3.10]: 5.2.20r125813
Bitbucket (instead of github)

TASK [deploy : Remove untracked files from project folder] *********************************************************************************************************************
System info:
  Ansible 2.8.4; Darwin
  Trellis Head
---------------------------------------------------
The conditional check 'not not(git_clean.stdout)' failed. The error was:
error while evaluating conditional (not not(git_clean.stdout)): 'dict object'
has no attribute 'stdout'
fatal: [167.172.111.171]: FAILED! => {}

[tangrufus]

What is the output when you run ansible in verbose mode?

ansible-playbook deploy.yml -e env=production -e site=example.com -vvv
deploy production example.com -vvv

[tjedrak]

TASK [deploy : Remove untracked files from project folder] **************************************************************************************************
task path: /Users/tomaszjedrak/Wordpress/kryptokasyna/trellis/roles/deploy/tasks/update.yml:27
Using module file /Library/Python/2.7/site-packages/ansible/modules/commands/command.py
Pipelining is enabled.
<staging.kryptokasyna.com> ESTABLISH SSH CONNECTION FOR USER: web
<staging.kryptokasyna.com> SSH: EXEC ssh -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="web"' -o ConnectTimeout=10 -o ControlPath=/Users/tomaszjedrak/.ansible/cp/9e79ed61d4 staging.kryptokasyna.com '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
<staging.kryptokasyna.com> (1, '', 'Traceback (most recent call last):\n  File "<stdin>", line 114, in <module>\n  File "<stdin>", line 106, in _ansiballz_main\n  File "<stdin>", line 49, in invoke_module\n  File "/tmp/ansible_command_payload_fsFCB_/__main__.py", line 327, in <module>\n  File "/tmp/ansible_command_payload_fsFCB_/__main__.py", line 263, in main\nOSError: [Errno 2] No such file or directory: \'/srv/www/staging.kryptokasyna.com/shared/source\'\n')
<staging.kryptokasyna.com> Failed to connect to the host via ssh: Traceback (most recent call last):
  File "<stdin>", line 114, in <module>
  File "<stdin>", line 106, in _ansiballz_main
  File "<stdin>", line 49, in invoke_module
  File "/tmp/ansible_command_payload_fsFCB_/__main__.py", line 327, in <module>
  File "/tmp/ansible_command_payload_fsFCB_/__main__.py", line 263, in main
OSError: [Errno 2] No such file or directory: '/srv/www/staging.kryptokasyna.com/shared/source'
System info:
  Ansible 2.8.4; Darwin
  Trellis Head
---------------------------------------------------
The conditional check 'not not(git_clean.stdout)' failed. The error was:
error while evaluating conditional (not not(git_clean.stdout)): 'dict object'
has no attribute 'stdout'
fatal: [staging.kryptokasyna.com]: FAILED! => {}

[tangrufus]

What if you reorder these 2 tasks in https://github.com/roots/trellis/blob/ca56ae5f0b2b262af173d4a0b025e2f69d883621/roles/deploy/tasks/update.yml:

- - name: Remove untracked files from project folder
-  command: git clean -fdx
-   args:
-     chdir: "{{ project_source_path }}"
-   register: git_clean
-   changed_when: not not(git_clean.stdout)

  - name: Failed connection to remote repo
    fail:
      msg: |
        Git repo {{ project.repo }} cannot be accessed. Please verify the repository exists and you have SSH forwarding set up correctly.
        More info:
        > https://roots.io/trellis/docs/deploys/#ssh-keys
        > https://roots.io/trellis/docs/ssh-keys/#cloning-remote-repo-using-ssh-agent-forwarding
    when: git_clone is failed

+ - name: Remove untracked files from project folder
+  command: git clean -fdx
+   args:
+     chdir: "{{ project_source_path }}"
+   register: git_clean
+   changed_when: not not(git_clean.stdout)

I expect deployment now fails at Failed connection to remote repo.
If thats the case, enable logging in https://github.com/roots/trellis/blob/ca56ae5f0b2b262af173d4a0b025e2f69d883621/roles/deploy/tasks/update.yml, it should give a better error message about why git clone was unsuccessful.

  - name: Clone project files
    git:
      repo: "{{ project_git_repo }}"
      dest: "{{ project_source_path }}"
      version: "{{ project_version }}"
      accept_hostkey: "{{ project.repo_accept_hostkey | default(repo_accept_hostkey | default(true)) }}"
      force: yes
    ignore_errors: true
-   no_log: true
+   no_log: false
    register: git_clone

[tjedrak]

@tangrufus you were right,
it failed at Failed connection to remote repo

TASK [deploy : Clone project files] ***************************************************************************************************************************************************************************
task path: /Users/tomaszjedrak/Wordpress/kryptokasyna/trellis/roles/deploy/tasks/update.yml:16
Using module file /Library/Python/2.7/site-packages/ansible/modules/source_control/git.py
Pipelining is enabled.
<staging.kryptokasyna.com> ESTABLISH SSH CONNECTION FOR USER: web
<staging.kryptokasyna.com> SSH: EXEC ssh -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="web"' -o ConnectTimeout=10 -o ControlPath=/Users/tomaszjedrak/.ansible/cp/9e79ed61d4 staging.kryptokasyna.com '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
<staging.kryptokasyna.com> rc=1, stdout and stderr censored due to no log
<staging.kryptokasyna.com> Failed to connect to the host via ssh: <error censored due to no log>
System info:
  Ansible 2.8.4; Darwin
  Trellis Head
---------------------------------------------------
fatal: [staging.kryptokasyna.com]: FAILED! => {
    "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result",
    "changed": false
}
...ignoring

TASK [deploy : Failed connection to remote repo] **************************************************************************************************************************************************************
task path: /Users/tomaszjedrak/Wordpress/kryptokasyna/trellis/roles/deploy/tasks/update.yml:27
System info:
  Ansible 2.8.4; Darwin
  Trellis Head
---------------------------------------------------
Git repo git@bitbucket.org:one7six/kryptokasyna.git cannot be accessed.
Please verify the repository exists and you have SSH forwarding set up
correctly.
More info:
> https://roots.io/trellis/docs/deploys/#ssh-keys
> https://roots.io/trellis/docs/ssh-keys/#cloning-remote-repo-using-ssh-
agent-forwarding

fatal: [staging.kryptokasyna.com]: FAILED! => {
    "changed": false
}

and here is what I got after switching no_log

TASK [deploy : Clone project files] ***************************************************************************************************************************************************************************
System info:
  Ansible 2.8.4; Darwin
  Trellis Head
---------------------------------------------------
Cloning into '/srv/www/staging.kryptokasyna.com/shared/source'...
Received disconnect from 18.205.93.0 port 22:2: too many authentication
failures
Disconnected from 18.205.93.0 port 22
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Cloning into '/srv/www/staging.kryptokasyna.com/shared/source'...
Received disconnect from 18.205.93.0 port 22:2: too many authentication
failures
Disconnected from 18.205.93.0 port 22
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

fatal: [staging.kryptokasyna.com]: FAILED! => {"changed": false, "cmd": "/usr/bin/git clone --origin origin '' /srv/www/staging.kryptokasyna.com/shared/source", "rc": 128, "stderr_lines": ["Cloning into '/srv/www/staging.kryptokasyna.com/shared/source'...", "Received disconnect from 18.205.93.0 port 22:2: too many authentication failures", "Disconnected from 18.205.93.0 port 22", "fatal: Could not read from remote repository.", "", "Please make sure you have the correct access rights", "and the repository exists."], "stdout": "", "stdout_lines": []}
...ignoring

TASK [deploy : Failed connection to remote repo] **************************************************************************************************************************************************************
System info:
  Ansible 2.8.4; Darwin
  Trellis Head
---------------------------------------------------
Git repo git@bitbucket.org:one7six/kryptokasyna.git cannot be accessed.
Please verify the repository exists and you have SSH forwarding set up
correctly.
More info:
> https://roots.io/trellis/docs/deploys/#ssh-keys
> https://roots.io/trellis/docs/ssh-keys/#cloning-remote-repo-using-ssh-
agent-forwarding

fatal: [staging.kryptokasyna.com]: FAILED! => {"changed": false}

So question is, why is that? Why too many authentication does it check all ssh keys? and server is set to block after many failed try outs? That IP 18.205.93.0 is bitbucket.org one of IPs.
I checked locally

 $ ssh -T bitbucket.org -i ~/.ssh/maker
authenticated via a deploy key.

You can use git or hg to connect to Bitbucket. Shell access is disabled.

This deploy key has read access to the following repositories:
one7six/kryptokasyna

also in config:

wordpress_sites:
  staging.kryptokasyna.com:
    site_hosts:
      - canonical: staging.kryptokasyna.com
        # redirects:
        #   - otherdomain.com
    local_path: ../site # path targeting local Bedrock site directory (relative to Ansible root)
    repo: git@bitbucket.org:one7six/kryptokasyna.git # replace with your Git repo URL
    repo_subtree_path: site # relative path to your Bedrock/WP directory in your repo
    branch: staging
    multisite:
      enabled: false
    ssl:
      enabled: true
      provider: self-signed
    cache:
      enabled: false

and yes, repo exists.

[tjedrak]

@tangrufus ok I revised all config files, deployed new droplet and appears it was my mistake. I used the non-existing branch on bitbucket for staging.