Unable to retrieve source image from internet when CPD_AIRGAP=true

[bjdzliu]

Describe the bug

I run the install cp4d command in airgap vm, but it throw a error that said it retrieves source image from cp.icr.io/cp/cpd/

Follow the below guide
https://ibm.github.io/cloud-pak-deployer/50-advanced/private-registry-and-air-gapped/#use-case-3---mirror-images-using-a-portable-image-registry

Already download the image, move the cpd-config to a airgap vm

when executed 'cp-deploy.sh', got fail message in cpd-mirror-images.log

TASK [utils : fail] ************************************************************
Tuesday 29 August 2023  05:35:40 +0000 (0:44:18.833)       0:50:11.142 ********
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Playbook fail while running 'case_mirror_with_dep.yml' for component 'cpfs'"}

check log /data/cpd-config/cpd-status/log/mirror_ibm-cp-common-services.log
`
error: unable to retrieve source image cp.icr.io/cp/cpd/postgresql manifest sha256:ec7970104d7e73c3e06ced54c5c7d3e0df18af6721e794ca8213ea446e3d93ff: Get "https://cp.icr.io/v2/cp/cpd/postgresql/manifests/sha256:ec7970104d7e73c3e06ced54c5c7d3e0df18af6721e794ca8213ea446e3d93ff": unsupported: The requested authentication method is not supported. Try again using appropriate login credentials for the resource you are attempting to access.

error: unable to retrieve source image cp.icr.io/cp/cpd/edb-postgres-license-provider manifest sha256:fdebb413179f7e08ffe0726edfcb1ad382d4ec5f9a8c71de956a6dc8b1744661: Get "https://cp.icr.io/v2/cp/cpd/edb-postgres-license-provider/manifests/sha256:fdebb413179f7e08ffe0726edfcb1ad382d4ec5f9a8c71de956a6dc8b1744661": unsupported: The requested authentication method is not supported. Try again using appropriate login credentials for the resource you are attempting to access.
`

To Reproduce
Steps to reproduce the behavior:

1. set env
   export CPD_OC_LOGIN="oc login xxxxx:6443 -u kubeadmin -p 79Txxxxxxx --insecure-skip-tls-verify"

export STATUS_DIR=/data/cpd-config/cpd-status

export CONFIG_DIR=/data/cpd-config

export CPD_AIRGAP=true

3. run

./cp-deploy.sh vault set \
 -vs image-registry-cpd453 \
 -vsv "admin:Harbor12345"

4. run
   ./cp-deploy.sh env apply

5. See error

TASK [cp4d-mirror-images : Copy case files to the /data/cpd-config/cpd-status/cp4d/case directory] ***
Tuesday 29 August 2023  05:35:49 +0000 (0:00:09.171)       0:55:42.371 ********

TASK [cp4d-mirror-images : fail] ***********************************************
Tuesday 29 August 2023  05:35:50 +0000 (0:00:00.686)       0:55:43.057 ********
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Mirroring of images failed, check mirror*.log files in /data/cpd-config/cpd-status/log for details"}

PLAY RECAP *********************************************************************
localhost                  : ok=619  changed=82   unreachable=0    failed=1    skipped=359  rescued=0    ignored=0

Expected behavior
why the process still download image from internet?

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [RHEL8.8]
• Install [CP4D 4.7.1]

[fketelaars]

@bjdzliu I have not been able to reproduce the issue. Can you please share the full log?

[fketelaars]

@bjdzliu I have reproduced the issue after all. The images are copied from the portable registry to the specified private registry, but for some reason there is another attempt to mirror from the entitled registry to the private registry as well:

error: unable to retrieve source image cp.icr.io/cp/cpd/edb-postgres-license-provider manifest sha256:16158bf65001641b22db0308fd8697fed59441af5e2fd6c2e482b27d07a1b817: Get "https://cp.icr.io/v2/cp/cpd/edb-postgres-license-provider/manifests/sha256:16158bf65001641b22db0308fd8697fed59441af5e2fd6c2e482b27d07a1b817": unsupported: The requested authentication method is not supported. Try again using appropriate login credentials for the resource you are attempting to access.

I will dig deeper into this to understand if this is a deployer issue or a CP4D product issue. Deployer uses the cp4d mirror-images command to mirror the images.

[fketelaars]

@bjdzliu I found the issue in the deployer code and have fixed it; currently running through some tests. I will push the code in a hotfix. If you have already sent the (large) status directory to the air-gapped infrastructure, these are the steps you can follow to implement the hotfix without having to resend the entire directory:

1. Update the automation-roles/50-install-cloud-pak/cp4d/cp4d-mirror-images/templates/mirror-images.j2 with the following content:

#!/bin/bash

set -o pipefail
mirror-images \
    --release={{ _p_current_cp4d_cluster.cp4d_version }} \
    --components={%- if _p_current_cp4d_cluster.cp4d_version >= '4.7.0' %}ibm-cert-manager,ibm-licensing,{%- endif %}{{ _cp4d_components }} \
{% if _p_source_registry_hostname == _v_portable_registry -%}
    --source_registry={{ _v_portable_registry }}:{{ _v_portable_registry_port }} \
{% endif -%}    
    --target_registry={{ _target_registry_url }} \
    --arch={{ _openshift_processor_arch }} \
    --preview={%- if _p_preview_script -%}true{%- else -%}false{%- endif %} \
    --case-download={%- if cpd_airgap | default(False) -%}false{%- else -%}true{%- endif %} 2>&1 | tee {{ status_dir }}/log/{{ _p_current_cp4d_cluster.project }}-mirror-images.log

2. Re-run the deployer, adding the --cpd-develop argument.

./cp-deploy.sh env apply --cpd-develop