Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add custom providers #46

Merged
merged 7 commits into from
Feb 5, 2025
Merged

WIP: Add custom providers #46

merged 7 commits into from
Feb 5, 2025

Conversation

Niduank
Copy link
Collaborator

@Niduank Niduank commented Feb 5, 2025

Custom endpoint provider

Settings

We can add custom provider in Settings which will fetch models.
We can toggle a custom provider
We can toggle a model from custom provider
Data is persistent

ModelSelection

We can see models from custom provider
We can select/use model from custom provider

SetUp Remote Models (dialog and warning)

Models from custom provider are considered as remote models.
If we configure a custom provider, we consider that remote models are configured.

Bugs

⚠️ Onit models can have same id than models from custom provider

  • Disable one in settings will disable all -> due to visibleModelIds
  • All model from custom provider are deprecated in Settings

Testing

Let's set OpenAI & this custom provider in Settings

Improvement

  • Unselect from Settings the current model should pick a default one ?
  • Find a way to skip model which doesn’t work on chat/completion ?

openhands-agent and others added 7 commits February 5, 2025 11:23
- Add CustomChatEndpoint and CustomModelsEndpoint
- Add CustomProvider model and UI components
- Support custom providers in model selection
- Display custom models in settings and model selection view
- Add migration from legacy model IDs to unique provider-based IDs
- Add testing utilities for verifying migration
- Update model visibility tracking to use unique IDs
@timlenardo
Copy link
Contributor

I'm made a number of changes here, and as far as I can tell, this is working as expected. A summary of changes:

  • Changed CustomModelsResponse to work for response from other providers (we only require an 'id' now). I've tested this with OpenRouter and Groq.
  • Added uniqueId property to AIModel to handle model identification across providers
  • Added a migration, so old versions of visibleModelIds are migrated to uniqueId instead of the original model ID
  • Moved the CustomChatEndpoint logic into the main "chat" function with the other providers.
  • Updated the settings UI to include: a search bar, a scrollview, a TokenView (plus refresh), and a trash button to remove a custom provider.

Some remaining tasks:

  • We aren't using the token validation logic, which I actually might prefer.
  • Oddly, through OpenHands, you can send SystemPrompts to models (like o1) that don't support system prompts. So, I'm including the system prompts in all the custom model chats, but this may not be the correct solution. We may need to let users enable/disable this on a per model basis.

@timlenardo timlenardo merged commit 0a3e398 into main Feb 5, 2025
@timlenardo timlenardo deleted the add-custom-providers branch February 6, 2025 00:34
@timlenardo timlenardo mentioned this pull request Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants