Skip to content

Conversation

CorieW
Copy link
Member

@CorieW CorieW commented Sep 10, 2025

Resolves #3444

What this does

  • Upgrades Genkit to the latest version
  • Migrates the plugin API to v2 (doc)

Testing

  • Checked that the evaluators load with the same names/prefixes on the UI.
  • Checked that at least one evaluator works consistent with pre-migration on the UI.
  • Using an evaluator object, checked using a flow works consistent with pre-migration.
  • Using an evaluator string ('pluginName/evaluatorName'), checked using a flow works consistent with pre-migration.
  • Tested error response from embedder.
  • Tested error response from model.

Copy link
Contributor

@cabljac cabljac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, one small nit

@CorieW CorieW marked this pull request as ready for review September 17, 2025 13:55
return checksEvaluators(googleAuth, metrics, await projectId);
},
list: async () => {
return [checksEvaluators(googleAuth, metrics, await projectId).__action];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm are we definitely meant to use __action

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, I'm not too sure. It did seem to work, but I'll look into this.

@cabljac cabljac marked this pull request as draft October 6, 2025 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

refactor: migrate checks plugin to V2 Plugin API
3 participants