Regenerate network and system UCI configs on every reconfigure, switch to role-based interface configuration #2372

neocturne · 2022-01-09T16:00:18Z

~~This PR is now based on #2382 (therefore blocked)~~

lan_ifname, wan_ifname and setup_ifname are reset on every reconfigure
- This makes it impossible to change these manually; instead, the roles of these logical interfaces can be modified on a higher level (see below). Changing the config mode interface (setup_ifname) is not possible.
- When only one interface exists, it is called single_ifname, and lan_ifname/wan_ifname are unset to allow separate configuration via site.conf for this case
Generate fresh network config
- Settings that are configurable in the Web UI are preserved, + minor additions
Generate fresh system config
- Only the "system" section (hostname, remote syslog), NTP servers and GPIO (PoE passthrough) settings are preserved
Interface configuration is now role-based. A list of roles ("client", "uplink", "mesh") can be assigned to each of the logical interfaces (LAN/WAN or single). See the individual commit messages, in particular "gluon-core: initialize interfaces role configuration" and "treewide: use interface roles as basis for network configuration", for details
Modifying port assignment on swconfig-based devices is not possible at the moment (preserve flag only allows adding new custom sections, not modifying existing ones)

TODO:

~~Preserve manually added sections if they have a 'gluon_preserve' marker option~~
~~Migrate old configuration to role-based settings~~
~~Much more testing~~
~~Documentation~~
~~Blocked: Rebase after gluon-config-mode-core: discard gluon-reconfigure output #2395~~

To test:

Migrations from 2021.1.x
- Correct migration of Mesh-on-LAN/WAN to mesh role on 2-interface devices
- Correct migration of Mesh-on-WAN on 1-interface devices
- Correct migration of Mesh-on-LAN on 1-interface devices that were originally set up with single_as_lan = true
Network config works for DSA-based devices
Network config works for swconfig-based devices
Other things?

mkg20001 · 2022-01-13T07:16:44Z

This kills the loopback interfacce

neocturne · 2022-01-14T15:19:48Z

This kills the loopback interfacce

Oops, fixed.

neocturne · 2022-01-23T11:55:40Z

Proposal for new site.conf settings for default interface assignment, superseding mesh_on_wan, mesh_on_lan and single_as_wan:

	interfaces = {
		lan = {
			default_roles = { 'client' },
		},
		wan = {
			default_roles = { 'uplink', 'mesh' },
		},
		single = {
			default_roles = { 'uplink', 'mesh' },
		},
	},

single will default to the same setting used for wan, but can be configured independently.

neocturne · 2022-01-23T21:22:59Z

Migration of Mesh-on-WAN/LAN setting from the old configuration is implemented now.

I've also added a simple migration for single -> WAN and vice-versa to account for added and removed interfaces. ~~This is currently not fully working without an upgrade, as /etc/board.json is not regenerated by default. We could account for this unusual case by regenerating board.json on every reconfigure.~~ Edit: Resetting board.json on reconfigure is implemented now.

AiyionPrime · 2022-01-26T22:03:30Z

http://lilienstra-e.n.ffh.zone/ runs the current version of the PR.
mesh-on-wan does apparently still work, gluon-reconfigure does not break the device and dmesg does not throw obvious errors.

Wireless mesh is disabled as it has been before the update.

https://hannover.freifunk.net/karte/#/en/map/b44bd6202778

This appears not to break a device that has nothing to migrate, which is nice.
What else should be tested?

neocturne · 2022-01-29T16:24:11Z

The PR description now contains a list of things to test. I have two devices for tests that actually need migration of interface names (a TP-LINK CPE510 v1 and a D-Link DIR-860L), but I don't think I have a single-interface device like that. Might be sufficient to test with x86 VMs, as /etc/config/network is reset anyways, so device-specific migration issues are unlikely (and if there are device-specific issues, they should also happen without migration).

I have also pushed a small change that should handle migration of the Mesh-on-LAN/WAN flag more correctly for single-interface devices.

belzebub40k · 2022-02-11T07:43:49Z

Should be fixed now.

The problem with the selection of the mesh interfaces is fixed but the "Bad Gateway" message still shows up when clicking save & reboot.

Fixed as well now. And now I'm confused why this worked at all in Gluon 2021.1.x...

Strange but I can confirm it is fixed now.

belzebub40k · 2022-02-14T09:55:31Z

I noticed a new problem. When using this branch I'm not able to change the hostname via the config-mode but pretty-hostname on the shell works. I did not notice any other options which are not saved. I tried to reproduce the problem with the master branch but there it is working fine.

EDIT: Never mind! I just noticed the conversation on IRC regarding the same problem. Can confirm it is fixed.

AiyionPrime · 2022-02-15T14:22:40Z

Starting a build for the CPE210 now, @belzebub40k sorry I effectively put the hostname-discussion in the wrong PR.

Do not write files when the content is unchanged. Avoids a few unnecessary filesystem writes when resetting ifname sysconfigs on each upgrade.

The new configuration generates sections iface_single/lan/wan in /etc/config/gluon. These sections usually refer to a sysconfig-controlled interface list, but adding custom sections with verbatim interfaces names is also possible. Each interface section contains a list of roles. The supported roles are 'client', 'uplink' and 'mesh'. Multiple roles can be configured on the same interface (for example the old 'mesh_on_wan' setting would become 'uplink'+'mesh'). 'client' is subsumed by any other role configured on the same interface ('client'+'mesh' is equivalent to 'mesh'). This property is important, as it allows the Wired Mesh settings in gluon-web-network to simply add and remove the mesh role without having to care what other roles are set - so in the default setup, this would switch between 'client' and 'client'+'mesh' for the LAN interface. By default, the WAN interface has role 'uplink' and the LAN interface 'client'; if only a single interface exists, the roles from the WAN interface are used by default. The default for each of the three interfaces (WAN/LAN/single) can be changed separated in site.conf, superseding the old mesh_on_wan, mesh_on_lan and single_as_lan settings.

With the new role-based interface configuration, it would be better to rename the wan/wan6 interfaces to uplink/uplink6, but that would cause unnecessary churn for the firewall configuration, so it is left for a later update. As all interfaces with the 'uplink' role are in the br-wan bridge, it is not possible to assign these to the 'mesh' role independently - instead, br-wan is added as a mesh interface as soon as a single interface has both the 'uplink' and 'mesh' roles. The UCI section for this configuration is now called 'mesh_uplink' instead of 'mesh_wan'. For all interfaces that have the 'mesh', but not the 'uplink' role a second configuration 'mesh_other' is created. If there is more than one such interface, all these interfaces are bridged as well (creating a bridge 'br-mesh_other'). This replaces the 'mesh_lan' section with its optional 'br-mesh_lan' bridge, but can also include interfaces that were not considered "LAN" when interfaces roles are modified (via site.conf or manually).

… introduce single_ifname Allow interface names to change on updates to handle hwconfig -> DSA and similar migrations. On devices with only a single interface, a sysconfig single_ifname is created instead of wan_ifname or lan_ifname to allow separate configuration in site.conf.

…iguration

A section can be marked as preseved by setting the gluon_preserve option to 1. In addition the following conditions must hold: - The preserved section must not already exist after OpenWrt's and Gluons setup scripts run. Modifying existing sections is currently unsupported. - Preserved sections must be named, so it can be detected whether a section conflicts with a preexisting one.

neocturne · 2022-02-22T18:00:39Z

Rebased and unblocked again.

lemoer · 2022-03-17T22:08:29Z

What is the current state here? How are we going to proceed?

neocturne · 2022-03-18T16:17:54Z

From my side this is ready to merge.

AiyionPrime · 2022-03-20T16:59:53Z

Thanks!

neocturne force-pushed the uci-regen branch from 9924bb1 to 976e654 Compare January 14, 2022 15:19

neocturne mentioned this pull request Jan 18, 2022

ath79-generic: Re-add OpenMesh devices #2325

Merged

neocturne mentioned this pull request Jan 23, 2022

Removal of more obsolete migrations, minor cleanup and improvements to lsupgrade.sh #2382

Merged

neocturne force-pushed the uci-regen branch from 976e654 to f6ec9ec Compare January 23, 2022 17:21

github-actions bot added the 3. topic: config-mode This is about the configuration mode label Jan 23, 2022

neocturne force-pushed the uci-regen branch from f6ec9ec to c679a4c Compare January 23, 2022 21:17

neocturne force-pushed the uci-regen branch 3 times, most recently from 096478e to afe399f Compare January 23, 2022 22:34

github-actions bot added 3. topic: docs Topic: Documentation 3. topic: multidomain labels Jan 23, 2022

neocturne marked this pull request as ready for review January 23, 2022 23:19

neocturne changed the title ~~[WIP] Regenerate network and system UCI configs on every reconfigure~~ Regenerate network and system UCI configs on every reconfigure, switch to role-based interface configuration Jan 23, 2022

neocturne force-pushed the uci-regen branch from a0be061 to a339be8 Compare January 29, 2022 16:17

lemoer mentioned this pull request Feb 11, 2022

Wishlist: Config-Mode UI for Interface Roles Feature #2393

Open

neocturne force-pushed the uci-regen branch from 5166f5e to 7b331c2 Compare February 13, 2022 17:30

neocturne mentioned this pull request Feb 15, 2022

gluon-config-mode-core: discard gluon-reconfigure output #2395

Merged

neocturne added the 2. status: blocked Marked as blocked because it's waiting on something label Feb 15, 2022

neocturne mentioned this pull request Feb 15, 2022

Backports for v2021.1.x #2390

Merged

neocturne added 8 commits February 22, 2022 18:52

treewide: regenerate network and system configs on every reconfigure

c2a5f57

gluon-core: sysconfig: avoid unnecessary writes

05c2951

Do not write files when the content is unchanged. Avoids a few unnecessary filesystem writes when resetting ifname sysconfigs on each upgrade.

gluon-web-network: update wired mesh settings for new role-based conf…

6dcee2e

…iguration

docs: update for role-based interface configuration

210cacd

neocturne force-pushed the uci-regen branch from 7b331c2 to 210cacd Compare February 22, 2022 17:52

neocturne removed the 2. status: blocked Marked as blocked because it's waiting on something label Feb 22, 2022

herbetom mentioned this pull request Mar 14, 2022

ath79-generic: add AVM FRITZ!WLAN Repeater 300E support #2406

Merged

19 tasks

blocktrron approved these changes Mar 20, 2022

View reviewed changes

blocktrron merged commit a3203b2 into master Mar 20, 2022

blocktrron deleted the uci-regen branch March 20, 2022 16:54

rotanid removed the 2. status: waiting-on-review Awaiting review from the assignee but also interested parties. label Apr 13, 2022

herbetom mentioned this pull request Apr 14, 2022

docs: change Commandline reference of Wired mesh needs update #2474

Closed

AiyionPrime mentioned this pull request Jun 3, 2022

RFC: use vlans on lan #1208

Closed

lemoer mentioned this pull request Oct 30, 2022

Config-Mode UI for Interface Role Assignment (v2) #2688

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Regenerate network and system UCI configs on every reconfigure, switch to role-based interface configuration #2372

Regenerate network and system UCI configs on every reconfigure, switch to role-based interface configuration #2372

neocturne commented Jan 9, 2022 •

edited

Loading

mkg20001 commented Jan 13, 2022

neocturne commented Jan 14, 2022

neocturne commented Jan 23, 2022

neocturne commented Jan 23, 2022 •

edited

Loading

AiyionPrime commented Jan 26, 2022 •

edited

Loading

neocturne commented Jan 29, 2022

belzebub40k commented Feb 11, 2022

belzebub40k commented Feb 14, 2022 •

edited

Loading

AiyionPrime commented Feb 15, 2022

neocturne commented Feb 22, 2022

lemoer commented Mar 17, 2022

neocturne commented Mar 18, 2022

AiyionPrime commented Mar 20, 2022

Regenerate network and system UCI configs on every reconfigure, switch to role-based interface configuration #2372

Regenerate network and system UCI configs on every reconfigure, switch to role-based interface configuration #2372

Conversation

neocturne commented Jan 9, 2022 • edited Loading

mkg20001 commented Jan 13, 2022

neocturne commented Jan 14, 2022

neocturne commented Jan 23, 2022

neocturne commented Jan 23, 2022 • edited Loading

AiyionPrime commented Jan 26, 2022 • edited Loading

neocturne commented Jan 29, 2022

belzebub40k commented Feb 11, 2022

belzebub40k commented Feb 14, 2022 • edited Loading

AiyionPrime commented Feb 15, 2022

neocturne commented Feb 22, 2022

lemoer commented Mar 17, 2022

neocturne commented Mar 18, 2022

AiyionPrime commented Mar 20, 2022

neocturne commented Jan 9, 2022 •

edited

Loading

neocturne commented Jan 23, 2022 •

edited

Loading

AiyionPrime commented Jan 26, 2022 •

edited

Loading

belzebub40k commented Feb 14, 2022 •

edited

Loading