podman generates RepoDigest for image even before pushing to repository

[maratsal]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

When you run podman build it generates RepoDigest value (even before pushing to the repo). After pushing to the repository and pulling image from the repo we get 2 different RepoDigests for the same repositories - one is valid from remote repository and one is wrong - generated locally on build step.

Steps to reproduce the issue:

1. podman build

podman build -t mynicerepo.com/mynicecontainer:0.0.42 .
STEP 1/6: FROM quay.io/operator-framework/helm-operator:v1.23.0
Trying to pull quay.io/operator-framework/helm-operator:v1.23.0...
Getting image source signatures
Copying blob sha256:00330fdc59e2764c1c62881617b535f4f325f92e5a518fb31b182f279fd196a9
Copying blob sha256:d6e3788a121c267b721a23e030dd339c41f70721e09bdf4b21051cfb22421025
Copying blob sha256:b336a633f9e2c8ec18c45427d7e957df6d20ec09e71eee92c2f3247bb63c2d66
Copying blob sha256:8b9d535b7a0c7e28a320d7993a65b4cd76226f07cadc52ea4fa2c0e283b8641a
Copying blob sha256:793728829df1765e573ef484e5c1bc618820d2c8d80b3e3ec2da94be3e78d97a
Copying config sha256:ad9afde5417fb5427cadc7292a84008407b2d74c1fa146497fdcf1cc2706ef15
Writing manifest to image destination
Storing signatures
STEP 2/6: ENV HOME=/opt/helm
--> d5e0ef83f73
STEP 3/6: COPY watches.yaml ${HOME}/watches.yaml
--> a37f0afa482
STEP 4/6: COPY helm-charts  ${HOME}/helm-charts
--> 73a7b92c77b
STEP 5/6: COPY terms-and-conditions.pdf /licenses/
--> 035e5bb1d98
STEP 6/6: WORKDIR ${HOME}
COMMIT mynicerepo.com/mynicecontainer:0.0.42
--> 4f1edd07a92
Successfully tagged mynicerepo.com/mynicecontainer:0.0.42
4f1edd07a929bb97c2f985f87f509de7a77684468cb8bf2ab83799e5cad33439

...

podman inspect --format='{{.RepoDigests}}' mynicerepo.com/mynicecontainer:0.0.42
[mynicerepo.com/mynicecontainer@sha256:a68a0413a154950b5490cc66cbb1b92cdea7cfb7edc3a5cc471bdc83bb03504b]

podman push mynicerepo.com/mynicecontainer:0.0.42

podman inspect --format='{{.RepoDigests}}' mynicerepo.com/mynicecontainer:0.0.42
[mynicerepo.com/mynicecontainer@sha256:a68a0413a154950b5490cc66cbb1b92cdea7cfb7edc3a5cc471bdc83bb03504b]

2. podman pull

$ podman pull mynicerepo.com/mynicecontainer:0.0.42
Trying to pull mynicerepo.com/mynicecontainer:0.0.42...
Getting image source signatures
Copying blob sha256:dab6b7de158ba6c1423241d8af62b2a83b2f6f2fca6344cc7d6946fb1cd2c94b
Copying blob sha256:d6e3788a121c267b721a23e030dd339c41f70721e09bdf4b21051cfb22421025
Copying blob sha256:b336a633f9e2c8ec18c45427d7e957df6d20ec09e71eee92c2f3247bb63c2d66
Copying blob sha256:82eebca6734c56e1a9cc6864ecd12eb488afb593f76b7dc58cd17e93ae09baab
Copying blob sha256:b40ce520043e1b0cb1b5d4a10551002994ec5c0d49fc2d5e7fdc1b840d4e65da
Copying blob sha256:3451f35d2494effa3f39a43e0426b7d5a3b2a141331f38334a19067c6c05c28c
Copying blob sha256:720a3863ec487ece18940dcb127f3985c4ec2d2561bfcd1dbedeeb6a220fc172
Copying blob sha256:158cbf681325b342d86131d6f43ffc68c8653f0c434bbf06f77ebd287381f936
Copying config sha256:4f1edd07a929bb97c2f985f87f509de7a77684468cb8bf2ab83799e5cad33439
Writing manifest to image destination
Storing signatures
4f1edd07a929bb97c2f985f87f509de7a77684468cb8bf2ab83799e5cad33439

podman inspect --format='{{.RepoDigests}}' mynicerepo.com/mynicecontainer:0.0.42
[mynicerepo.com/mynicecontainer@sha256:a68a0413a154950b5490cc66cbb1b92cdea7cfb7edc3a5cc471bdc83bb03504b mynicerepo.com/mynicecontainer@sha256:eed7a413a236a7fa8f1b5cb1a38764bffd496890a962d064eefaf8ae2755ee87]

Describe the results you received:

RepoDigest field is having 2 digests: fake and real one.

Describe the results you expected:

RepoDigest should be having only one real RepoDigest

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Client:       Podman Engine
Version:      4.1.1
API Version:  4.1.1
Go Version:   go1.18.3
Built:        Tue Jun 14 16:12:46 2022
OS/Arch:      darwin/amd64

Server:       Podman Engine
Version:      4.2.0
API Version:  4.2.0
Go Version:   go1.18.4
Built:        Thu Aug 11 10:42:17 2022
OS/Arch:      linux/amd64

Output of podman info:

host:
  arch: amd64
  buildahVersion: 1.27.0
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-2.fc36.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: '
  cpuUtilization:
    idlePercent: 97.37
    systemPercent: 1.21
    userPercent: 1.41
  cpus: 1
  distribution:
    distribution: fedora
    variant: coreos
    version: "36"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 501
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 5.19.6-200.fc36.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 1169104896
  memTotal: 2066894848
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.5-1.fc36.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.5
      commit: 54ebb8ca8bf7e6ddae2eb919f5b82d1d96863dea
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/501/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-0.2.beta.0.fc36.x86_64
    version: |-
      slirp4netns version 1.2.0-beta.0
      commit: 477db14a24ff1a3de3a705e51ca2c4c1fe3dda64
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 0h 60m 58.00s
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphRootAllocated: 106825756672
  graphRootUsed: 2425380864
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 6
  runRoot: /run/user/501/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 4.2.0
  Built: 1660228937
  BuiltTime: Thu Aug 11 10:42:17 2022
  GitCommit: ""
  GoVersion: go1.18.4
  Os: linux
  OsArch: linux/amd64
  Version: 4.2.0

Package info (e.g. output of rpm -q podman or apt list podman):

brew info podman
==> podman: stable 4.2.0 (bottled), HEAD
Tool for managing OCI containers and pods
https://podman.io/
/usr/local/Cellar/podman/4.1.1 (174 files, 47.7MB) *
  Poured from bottle on 2022-07-01 at 16:58:52
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb
License: Apache-2.0
==> Dependencies
Build: go-md2man ✘, go@1.18 ✘
Required: qemu ✘
==> Options
--HEAD
        Install HEAD version
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d
==> Analytics
install: 26,346 (30 days), 65,171 (90 days), 203,423 (365 days)
install-on-request: 25,563 (30 days), 63,954 (90 days), 202,065 (365 days)
build-error: 3 (30 days)

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

No

Additional environment details (AWS, VirtualBox, physical, etc.):

N/A

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[mtrmac]

To link issues together, this is very close to #14779 ; with the difference that this focuses on the RepoDigests field in Rodman image inspect, while the other one on the Digest field in podman image list.

[rhatdan]

@mtrmac is there something we should do about this?

[mtrmac]

@rhatdan Actually tracking RepoDigests, instead of trying to manufacture it, would help.

I don’t immediately know whether it is still tracked elsewhere in Podman or whether that was closed for inactivity.