Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: proxied models (remotely learned models used in DefaultRegistry) #1764

Merged
merged 1 commit into from
Dec 1, 2020

Conversation

milanlenco
Copy link
Collaborator

This small PR allows to register models learned remotely (i.e. over gRCP using the meta service) into the LocalRegistry.

Why is this useful?
This change can be used as a building block for integration of multiple (distributed) CNFs (pairs of agent + some data-plane component) into one appliance with a single API endpoint. A central agent can dynamically discover all CNFs, learn their models over gRPC and generate kv descriptors for them that would mostly just forward CRUD operations via remoteclient.
So basically the change allows the following configuration sequence:

agentctl/etcd/... -> [proxy agent(s) ->]  target agent

To proxy a given model one would write something like:

client, err = remoteclient.NewClientGRPC(grpcConn, remoteclient.UseRemoteRegistry("config"))
models, err := swMod.cfgClient.KnownModels("config")
for _, model := range models {
    if !proxyThisModel {
      continue
    }
    spec := models.ToSpec(cnfModel.info.Spec)
    _, err := models.DefaultRegistry.Register(cnfModel.info, spec)
    // build descriptor for it, etc...
}

Signed-off-by: Milan Lenco milan.lenco@pantheon.tech

@milanlenco milanlenco self-assigned this Nov 27, 2020
@milanlenco milanlenco changed the title feat: proxied models (remotely learned models used from DefaultRegistry) feat: proxied models (remotely learned models used in DefaultRegistry) Nov 27, 2020
// LocallyKnownModel is used for proto message with go type generated from imported proto file and known
// at compile time, while RemotelyKnownModel can't produce such go typed instances (we know the name of go type,
// but can't produce it from remote information) so dynamic proto message must be enough (*dynamicpb.Message))
model, err := GetModelFromModelRegistryForItem(item, modelRegistry)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This call is the same as one above. I understand that the first one is for existence checking and this one is for model itself, but lets call it only once and just make a note in comment that "err != nil" check also for nonexistence of model.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

return typeRegistry
}

// Register registers a protobuf message with given model specification.
// If spec.Class is unset empty it defaults to 'config'.
func (r *LocalRegistry) Register(x interface{}, spec Spec, opts ...ModelOption) (KnownModel, error) {
goType := reflect.TypeOf(x)
// check if the model was learned remotely
if modelInfo, isProxied := x.(*ModelInfo); isProxied {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The syntax of the "x" paramater doesn't inforce it, but the documentation says that the protobuf message is expected. So i would prefer a proto message as input. In your case you can use this to create new empty proto message:
dynamicpb.NewMessageType(modelinfo.MessageDescriptor).New().Interface()
(maybe you can put it somewhere as conversion helper function)
This will create a dynamicpb.Message instance and you can use that type here as type check switch.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can update the function documentation, but I don't think that I can use dynamicpb here since RemoteRegistry requires ModelInfo.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately true. So please just update to docs (the registry interface docs probably also need some update).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Signed-off-by: Milan Lenco <milan.lenco@pantheon.tech>
@ondrej-fabry ondrej-fabry merged commit 58571ed into ligato:master Dec 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants