Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DHT expose and search #770

Open
mihaisc opened this issue Dec 2, 2024 · 2 comments
Open

DHT expose and search #770

mihaisc opened this issue Dec 2, 2024 · 2 comments

Comments

@mihaisc
Copy link
Contributor

mihaisc commented Dec 2, 2024

  • include compute env in dht
  • create command to search dht
@alexcos20
Copy link
Member

alexcos20 commented Jan 13, 2025

We need to expose compute capabilities of a node to DHT (so users can search for them)
Since dht does not support any types of queries, there is no way to perform something like: "any node which has between 10 and 20 cpus with or without gpu"

Another caveat is the DHT content provider only works with hashes.

So, we need to expose the full capabilities in small steps, by exposing hashes of well known json structures.

Example:

By calculating the hash of the below string, we can serve as a provider of 2 CPUS, 1 Gb of RAM

{ "arch":"x86_64","cpus":2 , "memory": 1000000000}

If a host has 12 cpus and 10 gb of RAM, they will announce all combinations:

  • 1 cpu, 1 GB of RAM
  • 2 cpu, 1 GB of RAM
  • 3 cpu, 1 GB of RAM
    .....
  • 12 cpu, 1 GB of RAM
  • 1 cpu, 2 GB of RAM
  • 2 cpu, 2 GB of RAM
    ...
    On pseudocode, this will look like:
  for arch in SUPPORTED_ARCHS:
     for cpu=1 to TOTAL_CPUS in steps of 1:
       for ram = 1Gb to TOTAL_RAM in steps of 1GB:
             hash=calculate_hash(JSON.stringify({ "arch":arch,"cpus":cpu , "memory": ram})
             announce provider for hash

On the client side, the flow will be the following:

  • get from user the required specs
  • search DHT and get a list with nodes providing that type of resource
  • do additional check through status and getComputeEnvs

@alexcos20
Copy link
Member

let's do #801 first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants