Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add refresh-exclude-older flag to only transfer new snapshots during instance/volume refresh #1365

Merged

Conversation

ps-gill
Copy link
Contributor

@ps-gill ps-gill commented Nov 10, 2024

This pull request resolves issue #745

@github-actions github-actions bot added Documentation Documentation needs updating API Changes to the REST API labels Nov 10, 2024
@ps-gill ps-gill force-pushed the feature/745/allow-excluding-older-snapshots branch 5 times, most recently from 3fbd40c to ea9b60f Compare November 10, 2024 15:47
@ps-gill ps-gill marked this pull request as ready for review November 10, 2024 16:05
@ps-gill ps-gill requested a review from stgraber as a code owner November 10, 2024 16:05
@stgraber
Copy link
Member

Looking into this one now

@stgraber stgraber force-pushed the feature/745/allow-excluding-older-snapshots branch from ea9b60f to f5d88c7 Compare November 14, 2024 06:29
@stgraber
Copy link
Member

Just did a few minor tweaks so far:

  • Split the doc/rest-api.yaml changes to its own commit
  • Fixed a missing API extension comment in shared/api
  • Re-titled some commits

@stgraber
Copy link
Member

There seems to be a problem with the actual logic:

root@v1:~# incus snapshot list a1
+-------+----------------------+------------+----------+
| NAME  |       TAKEN AT       | EXPIRES AT | STATEFUL |
+-------+----------------------+------------+----------+
| snap0 | 2024/11/14 07:03 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap1 | 2024/11/14 07:03 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap2 | 2024/11/14 07:04 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap3 | 2024/11/14 07:04 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap4 | 2024/11/14 07:05 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap5 | 2024/11/14 07:05 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap6 | 2024/11/14 07:05 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap7 | 2024/11/14 07:05 UTC |            | NO       |
+-------+----------------------+------------+----------+
root@v1:~# incus snapshot list v2:a1
+-------+----------------------+------------+----------+
| NAME  |       TAKEN AT       | EXPIRES AT | STATEFUL |
+-------+----------------------+------------+----------+
| snap6 | 2024/11/14 07:05 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap7 | 2024/11/14 07:05 UTC |            | NO       |
+-------+----------------------+------------+----------+
root@v1:~# incus copy a1 v2: --refresh-exclude-older --refresh
root@v1:~# incus snapshot list v2:a1
+-------+----------------------+------------+----------+
| NAME  |       TAKEN AT       | EXPIRES AT | STATEFUL |
+-------+----------------------+------------+----------+
| snap0 | 2024/11/14 07:03 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap1 | 2024/11/14 07:03 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap2 | 2024/11/14 07:04 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap3 | 2024/11/14 07:04 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap4 | 2024/11/14 07:05 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap5 | 2024/11/14 07:05 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap6 | 2024/11/14 07:05 UTC |            | NO       |
+-------+----------------------+------------+----------+
| snap7 | 2024/11/14 07:05 UTC |            | NO       |
+-------+----------------------+------------+----------+
root@v1:~# 

@stgraber stgraber force-pushed the feature/745/allow-excluding-older-snapshots branch from f5d88c7 to fba6816 Compare November 14, 2024 07:37
@stgraber
Copy link
Member

I fixed the issue. It was the exclude flag not being passed around through migrationSinkArgs combined with then a bad check in utils.go.

@stgraber stgraber force-pushed the feature/745/allow-excluding-older-snapshots branch 2 times, most recently from 6dd5e73 to 6aa57bc Compare November 14, 2024 07:41
Signed-off-by: Parm Gill <pgdev@daak.ca>
…meSource

Signed-off-by: Parm Gill <pgdev@daak.ca>
… InstanceCopyArgs

Signed-off-by: Parm Gill <pgdev@daak.ca>
…e copy'

Signed-off-by: Parm Gill <pgdev@daak.ca>
Signed-off-by: Parm Gill <pgdev@daak.ca>
Signed-off-by: Parm Gill <pgdev@daak.ca>
Signed-off-by: Parm Gill <pgdev@daak.ca>
Signed-off-by: Parm Gill <pgdev@daak.ca>
@stgraber stgraber force-pushed the feature/745/allow-excluding-older-snapshots branch from 6aa57bc to 0779d5c Compare November 14, 2024 13:44
@stgraber stgraber enabled auto-merge November 14, 2024 13:44
@stgraber stgraber merged commit ccaa4de into lxc:main Nov 14, 2024
29 of 30 checks passed
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Nov 22, 2024
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [lxc/incus](https://github.com/lxc/incus) | minor | `v6.6.0` -> `v6.7.0` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>lxc/incus (lxc/incus)</summary>

### [`v6.7.0`](https://github.com/lxc/incus/releases/tag/v6.7.0): Incus 6.7

[Compare Source](lxc/incus@v6.6.0...v6.7.0)

#### What's Changed

-   fix live update VM's limits.memory configuration when use a percentage value by [@&#8203;itviewer](https://github.com/itviewer) in lxc/incus#1287
-   fix: fix slice init length by [@&#8203;cuishuang](https://github.com/cuishuang) in lxc/incus#1285
-   incusd/instance/lxc: Remove restrictions on /run by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1288
-   Correct macvlan mode names by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1284
-   Translations update from Hosted Weblate by [@&#8203;weblate](https://github.com/weblate) in lxc/incus#1290
-   Translations update from Hosted Weblate by [@&#8203;weblate](https://github.com/weblate) in lxc/incus#1295
-   Translations update from Hosted Weblate by [@&#8203;weblate](https://github.com/weblate) in lxc/incus#1304
-   incus-simplestreams: Fix list -f json by [@&#8203;melato](https://github.com/melato) in lxc/incus#1310
-   Profile performance improvements by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1314
-   incus-agent: Add timeout for DNS query by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1313
-   incusd/instance/qemu: Don't fail on console retrival issue by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1316
-   Allow changing the parent value on physical networks by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1317
-   incus: Fix display of current project in projects list by [@&#8203;montag451](https://github.com/montag451) in lxc/incus#1318
-   Add `--format` to `incus admin sql` by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1319
-   incusd/internal/server/instance/drivers:  support for Chimera Linux (qemu/edk2) pkg layout by [@&#8203;mwyvr](https://github.com/mwyvr) in lxc/incus#1298
-   incusd/instance/common: Cleanup volatile on device add failure by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1323
-   incusd/network/bgp: Only advertise networks with BGP configuration by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1325
-   Make revert library shared by [@&#8203;gibmat](https://github.com/gibmat) in lxc/incus#1326
-   Fix to the cluster resources caching mechanism by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1324
-   Fix idmap issues by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1327
-   Make ask library shared by [@&#8203;gibmat](https://github.com/gibmat) in lxc/incus#1329
-   doc/network/resolved: Add disabling DNSSEC and DNSOverTLS by [@&#8203;ntnn](https://github.com/ntnn) in lxc/incus#1328
-   Add some application container documentation by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1331
-   incusd/device/nic/bridged: Handle invalid configuration by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1330
-   Fix handling of custom volume snapshot patterns by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1333
-   Add OCI DHCP renewal by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1334
-   doc/installing: Update for Chimera Linux by [@&#8203;mwyvr](https://github.com/mwyvr) in lxc/incus#1335
-   shared/cgo: Don't use strlcpy by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1337
-   Implement `incus webui` by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1338
-   incusd/scriptlet: Make set_target fail with invalid members by [@&#8203;bensmrs](https://github.com/bensmrs) in lxc/incus#1339
-   Export QMP functions by [@&#8203;bensmrs](https://github.com/bensmrs) in lxc/incus#1340
-   incusd/network/ovn: Add support to ipv4.dhcp.ranges by [@&#8203;jonatas-lima](https://github.com/jonatas-lima) in lxc/incus#1341
-   internal/server: Log QMP interaction to a file by [@&#8203;bensmrs](https://github.com/bensmrs) in lxc/incus#1345
-   incusd/instance/qemu: Log QEMU command line by [@&#8203;bensmrs](https://github.com/bensmrs) in lxc/incus#1346
-   Improve cluster instance placement by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1344
-   incusd/instance_logs: Update log file list by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1347
-   Add infrastructure for OVN events by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1349
-   Fix QEMU feature checks during startup by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1350
-   incusd/instance/lxc: Fix LXCFS per-instance path by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1352
-   doc/idmap: Clarify subuid/subgid configuration by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1353
-   incusd/instance/qmp: Fix logging with no log file by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1355
-   Add a GetOIDCTokens() method by [@&#8203;gibmat](https://github.com/gibmat) in lxc/incus#1357
-   Add get-current to show current project by [@&#8203;maveonair](https://github.com/maveonair) in lxc/incus#1356
-   incus/file/create: Use SFTP client instead of file API by [@&#8203;HassanAlsamahi](https://github.com/HassanAlsamahi) in lxc/incus#1354
-   internal/instance: Allow 0 as value to limits.cpu.nodes by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1358
-   Translations update from Hosted Weblate by [@&#8203;weblate](https://github.com/weblate) in lxc/incus#1361
-   Translations update from Hosted Weblate by [@&#8203;weblate](https://github.com/weblate) in lxc/incus#1362
-   Translations update from Hosted Weblate by [@&#8203;weblate](https://github.com/weblate) in lxc/incus#1368
-   Improve agent interface listing performance by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1367
-   Make `incus top` output configurable through options by [@&#8203;presztak](https://github.com/presztak) in lxc/incus#1370
-   Automatic live-migration to balance load on cluster by [@&#8203;presztak](https://github.com/presztak) in lxc/incus#1369
-   gomod: Update dependencies by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1372
-   Add refresh-exclude-older flag to only transfer new snapshots during instance/volume refresh by [@&#8203;ps-gill](https://github.com/ps-gill) in lxc/incus#1365
-   incusd/instances/publish: Fix base metadata by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1374
-   Fix TPM with long instance names by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1377
-   Don't BGP advertise OVN load-balancers when all backends are offline by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1376
-   incusd/instance/qemu: Don't take over operations on console retrieval by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1379
-   Tweak to cluster internal relocation by [@&#8203;stgraber](https://github.com/stgraber) in lxc/incus#1378

#### New Contributors

-   [@&#8203;cuishuang](https://github.com/cuishuang) made their first contribution in lxc/incus#1285
-   [@&#8203;mwyvr](https://github.com/mwyvr) made their first contribution in lxc/incus#1298
-   [@&#8203;ntnn](https://github.com/ntnn) made their first contribution in lxc/incus#1328
-   [@&#8203;jonatas-lima](https://github.com/jonatas-lima) made their first contribution in lxc/incus#1341

**Full Changelog**: lxc/incus@v6.6.0...v6.7.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Changes to the REST API Documentation Documentation needs updating
Development

Successfully merging this pull request may close these issues.

2 participants