docs: What configuration in config.yml is necessary?

[tukusejssirs]

Related: #25

---

Current config.yml contains the following config per machine:

machines:
  - id: sim
    enabled: !!bool false
    type: l99.driver.fanuc.FanucMachine, fanuc
    strategy: l99.driver.fanuc.collectors.UseCase01, fanuc
    handler: l99.driver.fanuc.handlers.Native, fanuc
    
    broker:
      enabled: !!bool true
      net_ip: 10.20.30.102
      net_port: !!int 1883
      auto_connect: !!bool true
      publish_status: !!bool true
      publish_arrivals: !!bool true
      publish_changes: !!bool true
      publish_disco: !!bool true
      disco_base_topic: fanuc
      anonymous: !!bool true
      user: 
      password: 

    l99.driver.fanuc.FanucMachine, fanuc:
      sweep_ms: !!int 1000
      net_ip: 10.20.30.101
      net_port: !!int 8193
      net_timeout_s: !!int 3

As I understand it, the following properties are required:

• machine.id unique for each machine;
• machine.enabled if we shoud gather the data (it does not need to be required if we set a default value);
• machine.type;
• machine.strategy;
• machine.handler;

The following properties are optional:

• machine.broker: configure MQTT:
  • actually, we don’t configure the broker (MQTT server), but the MQTT client created and used by fanuc-driver, therefore I suggest to rename it the machine.mqtt;
  • if machine.broker is to be defined, the following properties are required:
    • machine.broker.enabled if we should use MQTT to publish the gathered data (it does not need to be required if we set a default value);
    • machine.broker.net_ip IP address or domain where the MQTT server is located;
    • machine.broker.net_port port which is used by the MQTT server;
  • optional properties (unless there is no default value set yet):
    • machine.broker.auto_connect;
    • machine.broker.publish_status;
    • machine.broker.publish_arrivals;
    • machine.broker.publish_changes;
    • machine.broker.publish_disco;
    • machine.broker.disco_base_topic;
    • machine.broker.anonymous;
    • machine.broker.user;
    • machine.broker.password.

---

I have no idea what is l99.driver.fanuc.FanucMachine, fanuc for. Is it required? Does its properties have their default values? Is it only used when using a simulator?

[MRIIOT]

l99.driver.fanuc.FanucMachine, fanuc: are the configuration properties for machines[id].type.

[MRIIOT]

Agreed, machine.broker is misleading.

[tukusejssirs]

In my opinion, there should be a global object (at the top level) where we could define global configuration and then we should use configuration of individual machines as overrides of the global config.

That way we could configure everything once unless we have machines with special needs.

[MRIIOT]

@tukusejssirs

configuration has changed. It is somewhat modular.

https://github.com/Ladder99/fanuc-driver/blob/develop/fanuc/config-example.yml

[tukusejssirs]

Thanks, @MRIIOT, for letting me know! I’d like to watch for any new commits in fanuc-driver, but I couldn’t find a way to do so on GitHub. Anyway, thanks for the awesome work! 🙏

As for the new config, could you better describe it? Or even better: update the docs and move them to fanuc-driver docs (or at least provide a link to them in there)?

---

Specific questions and/or statements that needs to be validated if they are true:

1. As I understand it, system is used to add global configuration to fanuc-driver, i.e. the base configuration which can be overrided in user and machines.
2. system['machine-disabled']: Why is it called disabled if it contains a property named enabled? How does it actually work? Wouldn’t it be easier (and much more easier to understand) to defined disabled-machines array that would contain a list of IDS of disabled machines? If it is empty (or undefined), monitoring on all machines would be enabled.

Is this config monitored for content changes? If not, I see a potential problem: how could I toggle the monitoring of a machine at runtime? I think it might be a good idea to implement a basic API (e.g. REST API) to enable runtime monitoring toggling, as well as shutting down fanuc-driver.

After reading L142 (and the comments at the begingging of machines object), as I understand it, machine-disabled is somehow ‘imported’ into a particular machine. Threfore I simply need to modify config.yml and add this line to a machine that is the monitoring being disabled and remove it if it is being enabled? I will need to find a way (a utility?) to modify YAML files from Node.js, as manually editing (i.e. regex replacing) the file would feel cumbersome.

3. system['machine-base-${transport}']: As I understand it, it defined the base config of the properties, which can be overridden in user and/or machines. My question is: why would you want to define the base config per transport? Wouldn’t it be better to define what transport to use (e.g. transport: 'mqtt' | 'influx' | 'shdr') and than the default values of type/strategy/handler?

4. What exactly is system['json-transfer'] for? I think I don’t need to change it, but do I need to define it in my config.yml?

5. I presume that user can be used to define that collectors to use, and thus the collectors in fanuc/collectors will be more or less obsolete and eventually removed, right?

6. What is the purpose of user['source-${id}' and user['target-${transport}-${id}'?

[MRIIOT]

As for the new config, could you better describe it? Or even better: update the docs and move them to fanuc-driver docs (or at least provide a link to them in there)?

I'll update the docs eventually.

Specific questions and/or statements that needs to be validated if they are true:

1. As I understand it, system is used to add global configuration to fanuc-driver, i.e. the base configuration which can be overrided in user and machines.

Correct, and user that is adding Fanuc machines should not touch system section.

2. system['machine-disabled']: Why is it called disabled if it contains a property named enabled? How does it actually work? Wouldn’t it be easier (and much more easier to understand) to defined disabled-machines array that would contain a list of IDS of disabled machines? If it is empty (or undefined), monitoring on all machines would be enabled.

Ok, noted.

Is this config monitored for content changes? If not, I see a potential problem: how could I toggle the monitoring of a machine at runtime? I think it might be a good idea to implement a basic API (e.g. REST API) to enable runtime monitoring toggling, as well as shutting down fanuc-driver.

No, this is not possible yet.

After reading L142 (and the comments at the begingging of machines object), as I understand it, machine-disabled is somehow ‘imported’ into a particular machine. Threfore I simply need to modify config.yml and add this line to a machine that is the monitoring being disabled and remove it if it is being enabled? I will need to find a way (a utility?) to modify YAML files from Node.js, as manually editing (i.e. regex replacing) the file would feel cumbersome.

Yes, << : *ref merges and overrides whatever values might already be present. YAML is not really configuration markup, so we do the best we can with the syntax that YAML offers and the current state of how the config is processed.

3. system['machine-base-${transport}']: As I understand it, it defined the base config of the properties, which can be overridden in user and/or machines. My question is: why would you want to define the base config per transport? Wouldn’t it be better to define what transport to use (e.g. transport: 'mqtt' | 'influx' | 'shdr') and than the default values of type/strategy/handler?

Made a change to this. Moved the transport specific stuff higher.

fanuc-driver/fanuc/config-example.yml

Lines 83 to 91 in cd08982

	target-mqtt-1: &target-mqtt-1
	# send to tcp mqtt broker
	<< : *transport-base-mqtt
	l99.driver.fanuc.transports.MQTT, fanuc:
	enabled: !!bool true
	net:
	type: tcp
	ip: 10.1.10.210
	port: !!int 1883

fanuc-driver/fanuc/config-example.yml

Lines 138 to 155 in cd08982

	- id: example-mqtt
	<< : *machine-base # base Fanuc configuration
	<< : *machine-disabled # omit this machine setup from runtime; remove to make active
	<< : *source-1 # where to get data
	<< : *collector-1 # what data to get and how to transform data
	<< : *target-mqtt-1 # where to send data
	- id: example-influx
	<< : *machine-base
	<< : *machine-disabled
	<< : *source-1
	<< : *collector-1
	<< : *target-influx-1
	- id: example-shdr
	<< : *machine-base
	<< : *machine-disabled
	<< : *source-1
	<< : *collector-1
	<< : *target-shdr-1

4. What exactly is system['json-transfer'] for? I think I don’t need to change it, but do I need to define it in my config.yml?

Determines if the Focas data is passed as JSON or .NET POCO to transport classes. This will be removed in the future.

5. I presume that user can be used to define that collectors to use, and thus the collectors in fanuc/collectors will be more or less obsolete and eventually removed, right?

No. The YAML only defines which classes in fanuc/collectors to instantiate at runtime.

6. What is the purpose of user['source-${id}' and user['target-${transport}-${id}'?

This is where you define the Fanuc connection details and MQTT,Influx,SHDR, etc connection details.

[tukusejssirs]

@MRIIOT, can’t I simply move the user['source-1'] config into machines[id]? Like changing this:

user:
  # focas sources
  source-1: &source-1
  # example fanuc source
  l99.driver.fanuc.FanucMachine, fanuc:
    sweep_ms: !!int 1000
    net:
      # Machine IP address
      ip: 1.2.3.4
      port: !!int 8193
      timeout_s: !!int 0

machines:
  - id: m13
    << : *machine-base        # base Fanuc configuration
    << : *machine-disabled    # omit this machine setup from runtime; remove to make active
    << : *source-1            # where to get data
    << : *collector-1         # what data to get and how to transform data
    << : *target-mqtt-1       # where to send data

into this:

machines:
  - id: m13
    << : *machine-base        # base Fanuc configuration
    << : *machine-disabled    # omit this machine setup from runtime; remove to make active
    # << : *source-1            # where to get data
    << : *collector-1         # what data to get and how to transform data
    << : *target-mqtt-1       # where to send data
    source-1:
      l99.driver.fanuc.FanucMachine, fanuc:
        sweep_ms: !!int 1000
        net:
          # Machine IP address
          ip: 1.2.3.4
          port: !!int 8193
          timeout_s: !!int 0

I think that adding a separate user ‘stage’ does not help anything. IMHO all we need is system (which I’d rather name global or general) and machine.

• fanuc-driver should have default configuration when a config is missing in config.yml (e.g. default strategy/handler/source/…);
• in system the user would configure the overrides to the default config in fanuc-driver:
  • nothing would need to be defined, as non-defined properties have the default config;
• in machines we would define the config per machine which would override the default config in fanuc-driver and well as system.

[MRIIOT]

Yes, without the source-1

machines:
  - id: m13
    << : *machine-base        # base Fanuc configuration
    << : *machine-disabled    # omit this machine setup from runtime; remove to make active
    << : *collector-1         # what data to get and how to transform data
    << : *target-mqtt-1       # where to send data
    l99.driver.fanuc.FanucMachine, fanuc:
      sweep_ms: !!int 1000
      net:
        # Machine IP address
        ip: 1.2.3.4
        port: !!int 8193
        timeout_s: !!int 0

[tukusejssirs]

I think that the configuration should be less bloated, less circular (currently we need to define something in system or user and reference it in machine). Here are some suggestions how to improve this. I have already told some of these suggestions. At the end of this comment, I provide you a sample configuration how I would try to implement this.

1. Within fanuc-driver, there should be a default value for every configuration item but machine ID and machine IP.
2. config.yml should be divided into two groups:
   • global:
     • global/general configuration which overrides the default configuration defined within fanuc-driver;
     • no configuration item is required in this group; if no item is provided, not even global is needed to be present in the config;
   • machines:
     • inherits from default and global;
     • configuration per machine, which overrides both default configuration and global unless a specific configuration item is defined;
     • the only required configuration items are machine ID and IP address;
3. Simplify the configuration item values.
   • For example, why would we need to define l99.driver.fanuc.FanucMachine, fanuc if all that changes is FanucMachine? If we actually need fanuc to configure that the machine is a Fanuc and not (e.g.) a Mazak, then let us provide a new configuration item for this.

Here is an example of the config structure I suggest. I might have overloops some use cases. Note that I provided some configuration items (even though they should be inherited) just to provide some extra information.

suggested_config.yml

---
global:
  enabled: !!bool true
  type: fanuc
  strategy: multi-strategy  # I Presume that `strategy` is used in Mazaks/MTConnect too, though with different values.
  handler: fanuc-one  # I Presume that `handler` is used in Mazaks/MTConnect too, though with different values.
  sweep_ms: !!int 1000  # I Presume that `sweep_ms` is used in Mazaks/MTConnect too
  net:  # Note: net.ip` cannot be defined in `global` nor `default`
      port: !!int 8193
      timeout_s: !!int 5
  target-groups:
    mqtt-1:  # User-provided target name (just like collector name is right now)
      type: mqtt  # mqtt | influx | shdr
      enabled: !!bool true
      login:  # Or choose a better name
        anonymous: !!bool true
        password:
        user:
      net:
        ip: localhost
        port: !!int 8193
        type: tcp
      topic: "fanuc/{{machine.Id}}/{{veneer.Name}}{{if veneer.SliceKey}}/{{veneer.SliceKey}}{{end}}"
    mqtt-2:  # User-provided target name (just like collector name is right now)
      type: mqtt  # mqtt | influx | shdr
      enabled: !!bool true
      login:  # Or choose a better name
        anonymous: !!bool true
        password:
        user:
      net:
        ip: localhost
        port: !!int 1883
        type: tcp
      topic: "fanuc/{{machine.Id}}/{{veneer.Name}}{{if veneer.SliceKey}}/{{veneer.SliceKey}}{{end}}"
  trasport: mqtt
  collector-groups:  # There should be a single default collector group per machine type (Fanuc, Mazak, ...); There might be multiple pre-defined collector groups though.
    diagnostics:  # User-provided collector name (just like it is right now)
      - type: fanuc
      - strategy: multi-strategy
      - collectors:
        - parameter-dump
        - diagnosis-dump
        - pmc-dump
    info:  # User-provided collector name (just like it is right now)
      - type: fanuc
      - strategy: multi-strategy
      - collectors:
        - machine-info
    monitoring:  # User-provided collector name (just like it is right now)
      - type: fanuc
      - strategy: multi-strategy
      - collectors:
        - alarms
        - axis-data  # Consider removing `-data` suffix from collector names
        - g-code-data
        - messages
        - production-data
        - spindle-data
        - state-data
        - tool-data

machines:
  - id: m1
    enabled: !!bool true  # This is unnecessary, as it could be inherited from `global`, and probably even in `default`
    handler: fanuc-one  # This is unnecessary, as it could be inherited from `global`
    net:
      ip: 10.14.7.2
      port: !!int 8193  # This is unnecessary, as it could be inherited from `global`; Actually, this config item might be removed altogether, as the port of `transport` should be always used.
      timeout_s: !!int 5  # This is unnecessary, as it could be inherited from `global`
    strategy: multi-strategy  # This is unnecessary, as it could be inherited from `global`
    sweep_ms: !!int 1000
    transport: mqtt-1
    type: fanuc  # This is unnecessary, as it could be inherited from `global`
  - id: m2
    enabled: !!bool false
    handler: fanuc-one  # This is unnecessary, as it could be inherited from `global`
    net:
      ip: 10.14.7.2
      port: !!int 1883  # This is unnecessary, as it could be inherited from `global`; Actually, this config item might be removed altogether, as the port of `transport` should be always used.
      timeout_s: !!int 5  # This is unnecessary, as it could be inherited from `global`
    strategy: multi-strategy  # This is unnecessary, as it could be inherited from `global`
    sweep_ms: !!int 2000
    transport: mqtt-2
    type: fanuc  # This is unnecessary, as it could be inherited from `global`