--noreplace false for portage doesn't reinstall packages with changed use flags

[desultory]

Summary

If you change the package use flags for a package and call tell the portage module to make the package present with changed_use set as true, ansible will detect the package is present and skip it, I'd expect this flag to literally just change how the parameters are passed to portage, especially since it doesn't seem to be checking dependencies itself

Issue Type

Bug Report

Component Name

portage

Ansible Version

$ ansible --version
ansible [core 2.13.7]
  config file = None
  configured module search path = ['/home/desu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.10/site-packages/ansible
  ansible collection location = /home/desu/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/lib/python-exec/python3.10/ansible
  python version = 3.10.9 (main, Feb 16 2023, 08:44:07) [GCC 11.3.1 20221209]
  jinja version = 3.1.2
  libyaml = True

Community.general Version

$ ansible-galaxy collection list community.general

Configuration

$ ansible-config dump --only-changed
# /usr/lib/python3.10/site-packages/ansible_collections
Collection        Version
----------------- -------
community.general 5.8.3

# /home/desu/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
community.general 6.2.0

OS / Environment

Linux boxxy 5.15.74.2-microsoft-custom-WSL2+ #20 SMP Tue Jan 10 11:55:17 CST 2023 x86_64 AMD Ryzen 9 7950X 16-Core Processor AuthenticAMD GNU/Linux

Linux nas 6.1.11-gentoo-ansible #11 SMP Mon Feb 13 07:39:24 CST 2023 x86_64 AMD EPYC 7R32 48-Core Processor AuthenticAMD GNU/Linux

Steps to Reproduce

- name: Configure and install isc dhcp server
  become: yes
  block:
  - name: Set dhcp use flags
    template:
      src: dhcp
      dest: /etc/portage/package.use/dhcp
      owner: root
      group: root
      mode: u=rw,g=r,o=r
  - name: Install dhcpd
    portage:
      name: net-misc/dhcp
      state: present
      changed_use: true
      #noreplace: false
  - name: Deploying /etc/dhcp/dhcp.conf
    template:
      backup: yes
      src: dhcpd.conf
      dest: /etc/dhcp/dhcpd.conf
      owner: root
      group: root
      mode: u=rw,g=r,o=r

Expected Results

I would expect the package to reinstall when the use flags are changed and "changed_use" is passed, even if --noreplace is passed:

nas /etc/portage/package.use # emerge --noreplace -U dhcp

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] net-misc/dhcp-4.4.3_p1::gentoo  USE="(selinux) server ssl -client* -ipv6 -ldap -vim-syntax" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 1) net-misc/dhcp-4.4.3_p1::gentoo
>>> Jobs: 0 of 1 complete, 1 running                Load avg: 0.06, 0.19, 0.20^C

Exiting on signal 2

Actual Results

TASK [service_dhcp : Set dhcp use flags] **************************************************************************************************************************************************
task path: /home/desu/projects/config/roles/service_dhcp/tasks/configure.yaml:4
changed: [nas] => {"changed": true, "checksum": "442848e8a82bfe340d4834b161d8894f357e937b", "dest": "/etc/portage/package.use/dhcp", "gid": 0, "group": "root", "md5sum": "8207ce82331edfa1a18226177a54ac21", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:portage_conf_t", "size": 29, "src": "/home/desu/.ansible/tmp/ansible-tmp-1676657536.4394581-2059-252009263040972/source", "state": "file", "uid": 0}

TASK [service_dhcp : Install dhcpd] *******************************************************************************************************************************************************
task path: /home/desu/projects/config/roles/service_dhcp/tasks/configure.yaml:11
redirecting (type: modules) ansible.builtin.portage to community.general.portage
redirecting (type: modules) ansible.builtin.portage to community.general.portage
ok: [nas] => {"changed": false, "msg": "Packages already present."}

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• plugins/modules/portage.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @Tatsh @wltjr
click here for bot help

[wltjr]

Hi @desultory,
I think you might be confusing --newuse, -N with --changed-use, -U, the behavior you are describing is portage specific, not anything with the ansible portage module.

Note where it says it does not trigger reinstallation.

   --changed-use, -U
          Tells emerge to include installed packages where USE flags  have
          changed  since  installation. This option also implies the --se-
          lective option. Unlike --newuse, the --changed-use  option  does
          not  trigger reinstallation when flags that the user has not en-
          abled are added or removed.

   --newuse, -N
          Tells emerge to include installed packages where USE flags  have
          changed since compilation. This option also implies the --selec-
          tive option.  USE flag changes include:

          A USE flag was added to a package.  A USE flag was removed  from
          a  package.  A USE flag was turned on for a package.  A USE flag
          was turned off for a package.

          USE flags may be toggled by your profile as well as your USE and
          package.use  settings.  If  you  would like to skip rebuilds for
          which disabled flags have been added to or  removed  from  IUSE,
          see  the related --changed-use option. If you would like to skip
          rebuilds for specific packages, see the --exclude option.

However, you may still have the same issue because of the conditional, I am not seeing where newuse: true would change the output you got, it would not change that conditional just as it did not for changed_use: true.

It does seem like a valid bug, but with the wrong portage option/flag. I am not able to confirm this at the moment, but if you switch your config from changed_use: to newuse:, that will confirm either way for sure; if you are able to change and re-test to confirm. Per the code, it should still produce the same error and fail to re-emerge.

[desultory]

Hi @desultory, I think you might be confusing --newuse, -N with --changed-use, -U, the behavior you are describing is portage specific, not anything with the ansible portage module.

Note where it says it does not trigger reinstallation.

   --changed-use, -U
          Tells emerge to include installed packages where USE flags  have
          changed  since  installation. This option also implies the --se-
          lective option. Unlike --newuse, the --changed-use  option  does
          not  trigger reinstallation when flags that the user has not en-
          abled are added or removed.

   --newuse, -N
          Tells emerge to include installed packages where USE flags  have
          changed since compilation. This option also implies the --selec-
          tive option.  USE flag changes include:

          A USE flag was added to a package.  A USE flag was removed  from
          a  package.  A USE flag was turned on for a package.  A USE flag
          was turned off for a package.

          USE flags may be toggled by your profile as well as your USE and
          package.use  settings.  If  you  would like to skip rebuilds for
          which disabled flags have been added to or  removed  from  IUSE,
          see  the related --changed-use option. If you would like to skip
          rebuilds for specific packages, see the --exclude option.

However, you may still have the same issue because of the conditional, I am not seeing where newuse: true would change the output you got, it would not change that conditional just as it did not for changed_use: true.

It does seem like a valid bug, but with the wrong portage option/flag. I am not able to confirm this at the moment, but if you switch your config from changed_use: to newuse:, that will confirm either way for sure; if you are able to change and re-test to confirm. Per the code, it should still produce the same error and fail to re-emerge.

In this case I'm forcing -client on net-misc/dhcp, so that should trigger a rebuild. Relevant files:
https://github.com/desultory/config/blob/main/roles/service_dhcp/templates/dhcp
https://github.com/desultory/config/blob/main/roles/service_dhcp/tasks/configure.yaml

I was testing by running emerge -U dhcp after deploying the file, and running that manually forces a rebuild only after I change the package use flags. I think the issue is that the ansible portage module doesn't even make portage run a command if the package is installed and noreplace is true - even if there are use flag changes. I'd run the script without my noreplace: false addition, and it would just say "Ok" for that section, like there were no changes to be made.

[wltjr]

I was testing by running emerge -U dhcp after deploying the file, and running that manually forces a rebuild only after I change the package use flags.

Yes, but that happens because you are specifically emerging that package, it has nothing to do with the -U flag.

$ USE="-client" emerge -pv1O dhcp
These are the packages that would be merged, in order:
[ebuild     U  ] net-misc/dhcp-4.4.3_p1-r1::gentoo [4.4.3-r1::gentoo] USE="ipv6 ssl -client* -ldap (-selinux) -server -vim-syntax" 9,845 KiB
Total: 1 package (1 upgrade), Size of downloads: 9,845 KiB

$ USE="-client" emerge -pv1OU dhcp
These are the packages that would be merged, in order:
[ebuild     U  ] net-misc/dhcp-4.4.3_p1-r1::gentoo [4.4.3-r1::gentoo] USE="ipv6 ssl -client* -ldap (-selinux) -server -vim-syntax" 9,845 KiB
Total: 1 package (1 upgrade), Size of downloads: 9,845 KiB

I think the issue is that the ansible portage module doesn't even make portage run a command if the package is installed and noreplace is true - even if there are use flag changes. I'd run the script without my noreplace: false addition, and it would just say "Ok" for that section, like there were no changes to be made.

Yes, regardless of the flag, that conditional I provided a link to I believe needs to be modified to correct the issue. Unless someone else is able to resolve it sooner, I will try to get to it ASAP.

[desultory]

I did testing by running:
emerge -U --noreplace dhcp

/etc/portage/package.use/dhcp file which may contain either net-misc/dhcp server client or net-misc/dhcp server -client

When it contains -client and is built/installed with that, I get this output:

nas /etc/portage/package.use # emerge -U --noreplace dhcp

These are the packages that would be merged, in order:

Calculating dependencies... done!

Total: 0 packages, Size of downloads: 0 KiB

Once I change the use flags I get output like this:

nas /etc/portage/package.use # emerge -U --noreplace dhcp

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] net-misc/dhcp-4.4.3_p1::gentoo  USE="(selinux) server ssl -client* -ipv6 -ldap -vim-syntax" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 1) net-misc/dhcp-4.4.3_p1::gentoo
>>> Jobs: 0 of 1 complete, 1 running                Load avg: 0.65, 0.39, 0.18nas /etc/portage/package.use # emerge -U --noreplace dhcp

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] net-misc/dhcp-4.4.3_p1::gentoo  USE="(selinux) server ssl -client* -ipv6 -ldap -vim-syntax" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 1) net-misc/dhcp-4.4.3_p1::gentoo
>>> Jobs: 0 of 1 complete, 1 running                Load avg: 0.65, 0.39, 0.18

Full log:

nas /etc/portage/package.use # emerge -U --noreplace dhcp

These are the packages that would be merged, in order:

Calculating dependencies... done!

Total: 0 packages, Size of downloads: 0 KiB

nas /etc/portage/package.use # nano dhcp
nas /etc/portage/package.use # emerge -U --noreplace dhcp

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] net-misc/dhcp-4.4.3_p1::gentoo  USE="client* (selinux) server ssl -ipv6 -ldap -vim-syntax" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 1) net-misc/dhcp-4.4.3_p1::gentoo
>>> Installing (1 of 1) net-misc/dhcp-4.4.3_p1::gentoo
>>> Jobs: 1 of 1 complete                           Load avg: 1.76, 0.48, 0.17

 * Messages for package net-misc/dhcp-4.4.3_p1:

 * The client and relay functionality will be removed in the next release!
 * Upstream have decided to discontinue this functionality.

 * GNU info directory index is up-to-date.
nas /etc/portage/package.use # cat dhcp
net-misc/dhcp server client
nas /etc/portage/package.use # nano dhcp
nas /etc/portage/package.use # cat dhcp
net-misc/dhcp server client
nas /etc/portage/package.use # emerge -U --noreplace dhcp

These are the packages that would be merged, in order:

Calculating dependencies... done!

Total: 0 packages, Size of downloads: 0 KiB

nas /etc/portage/package.use # nano dhcp
nas /etc/portage/package.use # cat dhcp
net-misc/dhcp server -client
nas /etc/portage/package.use # emerge -U --noreplace dhcp

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] net-misc/dhcp-4.4.3_p1::gentoo  USE="(selinux) server ssl -client* -ipv6 -ldap -vim-syntax" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 1) net-misc/dhcp-4.4.3_p1::gentoo
>>> Jobs: 0 of 1 complete, 1 running                Load avg: 0.28, 0.31, 0.15

I did testing running --noreplace -U as I wanted to execute the command I thought ansible was running, but it turns out that ansible doesn't seem to run this command at all since it determines there is nothing that needs to be done