How about a ID or name flag?

[crashr]

Hi.

I just tried paddler and it works great. I implemted a nice way in gppm (dev branch) to launch paddler agents via a post ready hook when the llama.cpp instance is up and running. I simply configure it like this:

and a view seconds later an up and running HA setup pops out.

I love it!
But as the title says it would be nice to be able to set a name or identifier via agent flag that is visible in the dashboard.

Greets

[mcharytoniuk]

@crashr Thank you for the kind words and for suggesting the feature to be able to name the agents. :)

I have already added it to the backlog. That should be a quick thing to add for me: #15

Also, you have a really nice project. I really appreciate that you tried it with Paddler.

If you add an example to your repo on integrating it with Paddler, I will be happy to feature gppm in Paddler's readme. If you also want to do it, just issue a Pull Request to change the README and add the "Featured Projects" section before the "Changelog" section and list gppm there.

[mcharytoniuk]

@crashr I just pushed that feature as a pre-release. You can add a --name flag to the agents, which shows up in the dashboard. Please let me know if that is what you have in mind. If so, I will make a proper release out of it:

https://github.com/distantmagic/paddler/releases/tag/v0.6.0-rc1

[crashr]

@mcharytoniuk Thank you for the quick implementation. This is exactly what I was asking for. As soon as I have my documentation ready in the next few days, there will definitely be an example of how to use it with paddler. It is already a central component of my personal setup.

[crashr]

This is a real world example that I use:

- name: "Codestral-22B-v0.1-Q8_0 (paddler balancer)"
  enabled: True
  command:
    "/usr/local/bin/paddler balancer \
      --management-host 0.0.0.0 \
      --management-port 8085 \
      --management-dashboard-enable=true \
      --reverseproxy-host 192.168.178.56 \
      --reverseproxy-port 8081"

- name: "Codestral-22B-v0.1-Q8_0 (llama.cpp 01)"
  enabled: True
  env:
    CUDA_VISIBLE_DEVICES: "0,1,2"
  command:
    "/usr/local/bin/llama-server \
      --host 0.0.0.0 \
      -ngl 100 \
      -m /models/Codestral-22B-v0.1-Q8_0.gguf \
      --port 8082 \
      -fa \
      -sm row \
      -mg 0 \
      --no-mmap \
      --log-format json"
  post_launch_hooks:
  - name: Codestral-22B-v0.1-Q8_0_(paddler_01)
    enabled: True
    command:
      "/usr/local/bin/paddler agent \
        --name 'Codestral-22B-v0.1-Q8_0 (llama.cpp 01)' \
        --external-llamacpp-host 192.168.178.56 \
        --external-llamacpp-port 8082 \
        --local-llamacpp-host 192.168.178.56 \
        --local-llamacpp-port 8082 \
        --management-host 192.168.178.56 \
        --management-port 8085"

- name: "Codestral-22B-v0.1-Q8_0_(llama.cpp_02)"
  enabled: True
  env:
    CUDA_VISIBLE_DEVICES: "0,1,2"
  command:
    "/usr/local/bin/llama-server \
      --host 0.0.0.0 \
      -ngl 100 \
      -m /models/Codestral-22B-v0.1-Q8_0.gguf \
      --port 8083 \
      -fa \
      -sm row \
      -mg 1 \
      --no-mmap \
      --log-format json"
  post_launch_hooks:
  - name: "Codestral-22B-v0.1-Q8_0_Paddler_02"
    enabled: True
    command:
      "/usr/local/bin/paddler agent \
        --name 'Codestral-22B-v0.1-Q8_0 (llama.cpp 02)' \
        --external-llamacpp-host 192.168.178.56 \
        --external-llamacpp-port 8083 \
        --local-llamacpp-host 192.168.178.56 \
        --local-llamacpp-port 8083 \
        --management-host 192.168.178.56 \
        --management-port 8085"

Launching the balancer like this is actually a bit hacky since gppm wasn't intended to do this but it works and I like to have everything bundled that belongs together.