[build][reproducible]Debian package versions pinning and downloading strategy

[baxia-lan]

Description

Open the issue to track the discussion in the thread

We are seeing warnings reported when resolve pinned debian package versions
https://sonic-build.azurewebsites.net/api/sonic/artifacts?branchName=master&platform=broadcom&buildId=514080&target=target%2Fsonic-broadcom.bin.log

Warning: the version of the package libcairo2-dev is not specified.
Warning: the version of the package libdbus-1-dev is not specified.
Warning: the version of the package libgirepository1.0-dev is not specified.
Warning: the version of the package libsystemd-dev is not specified.
Warning: the version of the package pkg-config is not specified.

These packages downloaded from mirrored/public debian registry do not have versions pinned thus make the build not reproducible.

Proposals

• Pinning ALL debian packages versions including existing and purged packages, not only the incremental ones. And report failure if packages versions are not pinned during reproducible build.

  • Pros: As all packages versions are pinned, no confusion for apt-get when resolving the packages version to download, this makes the build always reproducible; also this allow mirrored repositories to host multiple versions of packages shared between branches.

• The snapshot of debian repository per release branch should only have one available version per package and all debian packages should be downloaded only from mirrored repositories.

  • Pros: TBA

[k-v1]

@xumia
Could you please clarify about MIRROR_SNAPSHOT usage?
In #13097 you added this feature and in #17113 you enabled this option by default.
Should we still support reproducible builds for SONiC with disabled MIRROR_SNAPSHOT?
If not then we probably should remove apt-get pinning from SONiC. Otherwise need to fix all those places where we get 'Warning: the version of the package is not specified'

[judyjoseph]

@xumia could you check this issue from buildteam.