Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

separate repository machine [#1640] #1687

Merged

Conversation

ar3ndt
Copy link
Contributor

@ar3ndt ar3ndt commented Sep 21, 2020

No description provided.

@ar3ndt ar3ndt force-pushed the feature/separate-repository-machine branch from 0e8b8ac to 9ddaba5 Compare September 21, 2020 11:17
@seriva seriva self-requested a review September 23, 2020 12:28
seriva
seriva previously approved these changes Sep 23, 2020
Copy link
Contributor

@sk4zuzu sk4zuzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ar3ndt I believe minimal-cluster-config.yml for the "any" provider is missing 🤔

Copy link
Contributor

@sk4zuzu sk4zuzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also running it on this branch with some old config gives:

09:22:47 INFO cli.engine.ansible.AnsibleCommand - task path: /workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml:124
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1a1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1a2]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1b1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1b2]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1b3]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1a3]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}

Are we aiming for enabling this by default and break existing clusters?
If so why there is no documentation about it or any kind of a warning? 🤔

Copy link
Contributor

@sk4zuzu sk4zuzu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please take a look here:
https://github.com/epiphany-platform/epiphany/blob/cc944fd1a5ca9f8649bf4446a4470a32cf0e6ad5/core/src/epicli/data/common/ansible/playbooks/roles/helm/tasks/main.yml#L12-L15
Target host changed to localhost but this code didn't. That means that we are trying to do some stuff on ansible controller that we were previously doing on the remote. I don't think that is correct. 🤔

The full path here would be /var/www/html/epirepo/helm-charts/system here on the localhost, that hardly makes any sense IMO 👎 :( Please prove me wrong if I'm mistaken 👍 :)

core/src/epicli/data/common/ansible/playbooks/helm.yml Outdated Show resolved Hide resolved
@ar3ndt
Copy link
Contributor Author

ar3ndt commented Sep 24, 2020

@ar3ndt I believe minimal-cluster-config.yml for the "any" provider is missing thinking

added

@ar3ndt
Copy link
Contributor Author

ar3ndt commented Sep 24, 2020

Also running it on this branch with some old config gives:

09:22:47 INFO cli.engine.ansible.AnsibleCommand - task path: /workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml:124
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1a1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1a2]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1b1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1b2]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1b3]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1a3]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}

Are we aiming for enabling this by default and break existing clusters?
If so why there is no documentation about it or any kind of a warning? thinking

working on it

@ar3ndt
Copy link
Contributor Author

ar3ndt commented Sep 24, 2020

Please take a look here:

https://github.com/epiphany-platform/epiphany/blob/cc944fd1a5ca9f8649bf4446a4470a32cf0e6ad5/core/src/epicli/data/common/ansible/playbooks/roles/helm/tasks/main.yml#L12-L15

Target host changed to localhost but this code didn't. That means that we are trying to do some stuff on ansible controller that we were previously doing on the remote. I don't think that is correct. thinking

The full path here would be /var/www/html/epirepo/helm-charts/system here on the localhost, that hardly makes any sense IMO -1 :( Please prove me wrong if I'm mistaken +1 :)

this code actually is not neccessary here. Deleted

@ar3ndt
Copy link
Contributor Author

ar3ndt commented Sep 25, 2020

Also running it on this branch with some old config gives:

09:22:47 INFO cli.engine.ansible.AnsibleCommand - task path: /workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml:124
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1a1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1a2]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1b1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1b2]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1b3]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}
09:22:47 ERROR cli.engine.ansible.AnsibleCommand - fatal: [x1a3]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'repository'\n\nThe error appears to be in '/workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/preflight_facts/tasks/kubernetes.yml': line 124, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PREFLIGHT_FACTS | Produce default repository and image registry urls\n  ^ here\n"}

Are we aiming for enabling this by default and break existing clusters?
If so why there is no documentation about it or any kind of a warning? thinking

working on it

added changelog note

@ar3ndt ar3ndt force-pushed the feature/separate-repository-machine branch 2 times, most recently from c67687f to 46c8dbe Compare September 25, 2020 14:36
CHANGELOG-0.8.md Outdated Show resolved Hide resolved
@sk4zuzu
Copy link
Contributor

sk4zuzu commented Sep 28, 2020

@ar3ndt I executed one test in vscode using "any" provider from your branch. I got this error:

09:04:05 INFO cli.engine.ansible.AnsibleCommand - TASK [helm : Add system-charts Helm repository from url http://localhost/epirepo] ************************************************************
09:04:05 INFO cli.engine.ansible.AnsibleCommand - task path: /workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/helm/tasks/main.yml:17
09:04:05 ERROR cli.engine.ansible.AnsibleCommand - fatal: [127.0.0.1]: FAILED! => {"changed": true, "cmd": "helm repo add system-charts http://localhost/epirepo/helm-charts/system", "delta": "0:00:00.055212", "end": "2020-09-28 09:04:05.277737", "msg": "non-zero return code", "rc": 1, "start": "2020-09-28 09:04:05.222525", "stderr": "Error: looks like \"http://localhost/epirepo/helm-charts/system\" is not a valid chart repository or cannot be reached: failed to fetch http://localhost/epirepo/helm-charts/system/index.yaml : 404 Not Found", "stderr_lines": ["Error: looks like \"http://localhost/epirepo/helm-charts/system\" is not a valid chart repository or cannot be reached: failed to fetch http://localhost/epirepo/helm-charts/system/index.yaml : 404 Not Found"], "stdout": "", "stdout_lines": []}
09:04:05 INFO cli.engine.ansible.AnsibleCommand -  

Cluster config I used:

kind: epiphany-cluster
title: "Epiphany cluster Config"
provider: any
name: any1
specification:
  name: any1
  admin_user:
    name: ubuntu
    key_path: /workspaces/epiphany/core/src/epicli/clusters/id_rsa
  components:
    kubernetes_master:
      count: 1
      machines:
        - default-k8s-master1
    kubernetes_node:
      count: 3
      machines:
        - default-k8s-node1
        - default-k8s-node2
        - default-k8s-node3
    repository:
      count: 1
      machines:
        - default-k8s-master2
    postgresql:
      count: 1
      machines:
        - default-k8s-master3
---
kind: configuration/shared-config
title: Shared configuration that will be visible to all roles
name: default
specification:
  use_ha_control_plane: true
  promote_to_ha: false
provider: any
---
kind: infrastructure/machine
provider: any
name: default-k8s-master1
specification:
  hostname: x1a1
  ip: 10.20.2.10
---
kind: infrastructure/machine
provider: any
name: default-k8s-master2
specification:
  hostname: x1a2
  ip: 10.20.2.11
---
kind: infrastructure/machine
provider: any
name: default-k8s-master3
specification:
  hostname: x1a3
  ip: 10.20.2.12
---
kind: infrastructure/machine
provider: any
name: default-k8s-node1
specification:
  hostname: x1b1
  ip: 10.20.2.20
---
kind: infrastructure/machine
provider: any
name: default-k8s-node2
specification:
  hostname: x1b2
  ip: 10.20.2.21
---
kind: infrastructure/machine
provider: any
name: default-k8s-node3
specification:
  hostname: x1b3
  ip: 10.20.2.22

What is wrong with my setup? 🤔

@ar3ndt
Copy link
Contributor Author

ar3ndt commented Sep 28, 2020

checking

@ar3ndt
Copy link
Contributor Author

ar3ndt commented Sep 28, 2020

@ar3ndt I executed one test in vscode using "any" provider from your branch. I got this error:

09:04:05 INFO cli.engine.ansible.AnsibleCommand - TASK [helm : Add system-charts Helm repository from url http://localhost/epirepo] ************************************************************
09:04:05 INFO cli.engine.ansible.AnsibleCommand - task path: /workspaces/epiphany/core/src/epicli/clusters/build/any1/ansible/roles/helm/tasks/main.yml:17
09:04:05 ERROR cli.engine.ansible.AnsibleCommand - fatal: [127.0.0.1]: FAILED! => {"changed": true, "cmd": "helm repo add system-charts http://localhost/epirepo/helm-charts/system", "delta": "0:00:00.055212", "end": "2020-09-28 09:04:05.277737", "msg": "non-zero return code", "rc": 1, "start": "2020-09-28 09:04:05.222525", "stderr": "Error: looks like \"http://localhost/epirepo/helm-charts/system\" is not a valid chart repository or cannot be reached: failed to fetch http://localhost/epirepo/helm-charts/system/index.yaml : 404 Not Found", "stderr_lines": ["Error: looks like \"http://localhost/epirepo/helm-charts/system\" is not a valid chart repository or cannot be reached: failed to fetch http://localhost/epirepo/helm-charts/system/index.yaml : 404 Not Found"], "stdout": "", "stdout_lines": []}
09:04:05 INFO cli.engine.ansible.AnsibleCommand -  

Cluster config I used:

kind: epiphany-cluster
title: "Epiphany cluster Config"
provider: any
name: any1
specification:
  name: any1
  admin_user:
    name: ubuntu
    key_path: /workspaces/epiphany/core/src/epicli/clusters/id_rsa
  components:
    kubernetes_master:
      count: 1
      machines:
        - default-k8s-master1
    kubernetes_node:
      count: 3
      machines:
        - default-k8s-node1
        - default-k8s-node2
        - default-k8s-node3
    repository:
      count: 1
      machines:
        - default-k8s-master2
    postgresql:
      count: 1
      machines:
        - default-k8s-master3
---
kind: configuration/shared-config
title: Shared configuration that will be visible to all roles
name: default
specification:
  use_ha_control_plane: true
  promote_to_ha: false
provider: any
---
kind: infrastructure/machine
provider: any
name: default-k8s-master1
specification:
  hostname: x1a1
  ip: 10.20.2.10
---
kind: infrastructure/machine
provider: any
name: default-k8s-master2
specification:
  hostname: x1a2
  ip: 10.20.2.11
---
kind: infrastructure/machine
provider: any
name: default-k8s-master3
specification:
  hostname: x1a3
  ip: 10.20.2.12
---
kind: infrastructure/machine
provider: any
name: default-k8s-node1
specification:
  hostname: x1b1
  ip: 10.20.2.20
---
kind: infrastructure/machine
provider: any
name: default-k8s-node2
specification:
  hostname: x1b2
  ip: 10.20.2.21
---
kind: infrastructure/machine
provider: any
name: default-k8s-node3
specification:
  hostname: x1b3
  ip: 10.20.2.22

What is wrong with my setup? thinking

fixed

@ar3ndt ar3ndt force-pushed the feature/separate-repository-machine branch 2 times, most recently from 462fd18 to d95f1af Compare September 28, 2020 13:42
Copy link
Contributor

@rpudlowski93 rpudlowski93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only one change needed from my side in this PR (comment left). Moreover, it is necessary to change in core/src/epicli/data/common/ansible/playbooks/roles/repository/tasks/create-helm-repo.yml
{{ repository_url }} to localhost. Except of these two changes, everything works fine, tested. Good job :)

@ar3ndt ar3ndt force-pushed the feature/separate-repository-machine branch 2 times, most recently from ef49c8b to d1d8f56 Compare September 30, 2020 13:35
@ar3ndt ar3ndt force-pushed the feature/separate-repository-machine branch from 2ad74aa to 95cc078 Compare September 30, 2020 15:06
@ar3ndt ar3ndt force-pushed the feature/separate-repository-machine branch from 95cc078 to 2e36027 Compare September 30, 2020 15:39
Copy link
Contributor

@mkyc mkyc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ar3ndt ar3ndt dismissed stale reviews from seriva, rpudlowski93, and sk4zuzu September 30, 2020 15:47

done

@ar3ndt ar3ndt merged commit c8ee97f into hitachienergy:develop Sep 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants