Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new Multimodal AI Block utilizing openrouter's API Service. #8290

Closed
wants to merge 42 commits into from

Conversation

ymrohit
Copy link
Contributor

@ymrohit ymrohit commented Oct 10, 2024

This PR introduces the MultimodalAIBlock class, a new block that integrates multimodal AI models via OpenRouter API. It allows users to input a text prompt and an image URL, and select from several multimodal models for analysis. The block processes the inputs and returns a response based on the selected AI model.

ymrohit and others added 29 commits September 29, 2024 07:27
@ymrohit ymrohit requested a review from a team as a code owner October 10, 2024 13:55
@github-actions github-actions bot added size/l and removed size/m labels Oct 10, 2024
@aarushik93 aarushik93 changed the base branch from master to dev October 14, 2024 15:27
@aarushik93
Copy link
Contributor

thanks for the PR @ymrohit - just curious there's no speech modality models available via Openrouter as of now right?

@ymrohit
Copy link
Contributor Author

ymrohit commented Oct 17, 2024

@aarushik93 I don't think there is any speech modality models available via openrouter at the moment.

aarushik93
aarushik93 previously approved these changes Oct 17, 2024
@github-actions github-actions bot added the platform/frontend AutoGPT Platform - Front end label Oct 24, 2024
@github-actions github-actions bot removed the platform/frontend AutoGPT Platform - Front end label Oct 25, 2024
@ymrohit ymrohit requested a review from a team as a code owner October 30, 2024 10:42
@github-actions github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Oct 30, 2024
Copy link
Contributor

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

DATABASE_URL: "AgAGn36yLuB4z1eh1MuBkUt3EydFYVoFi4v/XC7JgfdVyt5cMI9c59rv1Jztrep/TejpeJVPYKuyR07TH9d96Lc+8ALXSNZKCD8Q57/vqnaMRjPvnSwVaLzdx57F66953y2p6244QXpbLHaVGHG0ZowV0rY4XpU6VcmQZnSbSVyBXS+ZcYjF5AYN31+0TWTG8RRVAJ1WFA6Jq4fUtLokaZTr5+pPy4FGoEdBZJVAgNVBPNe6XIrCTJL7N/zwJxUWkqF1nUTWkr017SAodgicQbogcvOcHclgZLEf60nRPbxYUiLJ1uzcBGMT1Ze8n3TdIi5uMvqQ5fb/rHqyEBGjpcOPhZR5yZTeLVhdxOPoQ2gzMVfASGbfzrueGFq0SGKn1P6Vm1zboVD39/EunEfZ4iXULRVeqIXBZ7jhBIkaYx/NwX7LkaUfv+mpEI6xhYewHT9QH/QXlTjN5XAMERryJrAlOuEe6cv0nz7Up+sLkesPLKDaWXkcoKDhgA3XP1aTTgn2i8nzAOzB9q+Y05amEGl7Z1YYtVD/2pR96sZhaL1YcS32WP03HyZ3+ZRM1cq9AgEE776iMsOAgzbETj+Li6l/Oc2i8Nb9dSwjdz/fAjR9P6v3+JPl0Zl71P8Y7RKDawXyzfaNGOOl1lBeY3CVe+t3duSOdO7PuQOVxbGYIlqgSs5hdLRmTeQGywv2ucxS4p9IZTqooI1BoXHGW/JE5GbzxVMVanuY1eM+F/G1xY6W2tIyNo8a4qVjZ1ZbrdUFqNZkBPnJcekNFyo2PErfVMLJDVwhU8hOTbPmmlkYf9EAEYDkTFIeqXimN+oIK6xzYiomppECl8fTJ86doMK6fjD/DtdBfGuM35eGu9jQYpLoHv3XTpO0stx6utk="
SENTRY_DSN: "AgAd4XlyuPNl5fqxIGHg5L7y2nzjujpKoHUP9IUAOPiGEobtjiP1M5M8EB/0anjqrIj9GC+oqPWuLvL4rPwPMrTeiEyRmeHwlA9eQDJPi+C3sxu4cbXwUdh/Un5hLZkE/GuqMuOgALwZ599CmHQJVXye9mWbs8xWmyGQd6BJsEE32KMRIbI3Wk21luIzypmyeTg1WdpNuAwecwK4u+CkadwGWaJOyZJacJDRwvNzGWbjzGHAHHsllXk2gZFz4CCGNDyDVZrGPG0rCmJji72pu1jcjipoqYXte1WWfs9Yr9Nc1YU/yaOqz7/U3NhPzezQtvzbTF7RRJaBV4nOIjoOwSFBIeoYPjr8BSFNyCc3/8F+2nWJ7pDSWXbV5L2aDIP7ba0tLeqsuCxevvooGNDki1iGBykEp/NXOCQeZ0BCQYS9OuqCHzrrFA6jjkA6Ojxf2E7WZ8Y8LGVW9LGMggh1mM2c+fbV1za2psRrCQXR2ml46BOIt9/24Os56dggt/vbgXJ7l1mXQXQm+XU/S51ifK4g+HqkTE6PhAafm/sTLqeQKMDsy1/iEsnb8yphjpsydxBDgp+Rcp/zQOt0JuD4TTvpQE8NNwVQZJ2I6zMYwTu2tg+IIiv1fFtYt76BmOqjS/4USk9sjlHQRf562th3TRMENcg8crOOBmOmLaaEI/JCrtnpPMO8N8v26Q8bFSAzBFkyvO2kSUFNMkfKf2kBdtcufh3MysGtu/fL3Gh3BAOrefEDnHYFvvPx1w22FJsypl82mbU55wj2T/v9/UIxqeJpSW0/6HraNKzYAQ5Z9Fx2sfDBafMihiRWsURPfOqZkw=="
SUPABASE_SERVICE_ROLE_KEY: "AgAZKuZI0N/fmap8PaEoYti567yK5rugVVtMY0au4HTQBKKz336rcoMz/aGWWlPf/8AfpdEq1iRNKNRG+Xcz7MvCM9vjophOxCSoIgDts8zaKJ835NSKZZTtmQJRWdq8hAtTzA4QgKVq3uGmMiNi/8k3pKlif33sAdYRZoAOZ+RWb1lrLX7E3NTRcLfLbnQqoiMCt99Ks6npDgyxH2/ugBQGgP3DxmLPR2LA7FHOSlXbvLl5sA5FtxPXfcogjAL4u37FrG4yJ+7B+n+f9DpIPGpCpo7SyQNJ1iaY8OU1I6/PxmMr/ECshcQVWsLtKRgqSpZf6nO6DHXFzUFYufM4xJ2hELxeo4ihNbqt/GrZK9FR26vEohSPblL4pC3i5qdAbGkHhUAui0yzs4RApN+o/RhvE1Ip98PJoSBzz/TOUE9PASRa7d8kzYQGFBQ7o703vyAdWOXopoOkmbiZ3kmVxYxi5lXqi6xaN0jzjVldelk8p6SCcWadR1MEZjpyiY6bPM5/kLgHiq25UsU4N/ssqA+ZXma7zUMydbxbfxqW9D8kSs5H7Hcbbd07HDYPmBNSc3w5MTI5QPZzH/yPbRA0pKUX3yF05BhIbgDuhRYQG582IYk8nI+PmvqNxDFLEDEauzoX90qNUcidZS2jgjJ0BwmkB2zgBdoU13zELjdhdewoOJJkAVm6wIT83tUnkwpEW2nJg5bE1dmvKXQuoT85/03hIf36BXQ815m6Cp1lf4W4goOfF3U3HM7CGNFRC8st3l7Rem49AE7sPWBVc00a8NnKVzdbFohm0l+NRzRwFagU/gYkXHILpP0YPdfWHI8mptv0+UlirwuJXW0EUYU9hu9rCMB3u+ooSUFLK3ojlA7A34s7lJ+7VW0haxl/Wt0PQGwGptDT8ZBrLFv7NtCGq7FRlBfdlHEKouuaqgzIJq7THwEXyvOAkmZMhrEoQoOZol/cMXYntglTO9eZ5Vhapp6ttmO8SxQieAilImY="
DATABASE_URL: "AgCe47Y6QFnZRmNNpc3iVIcJ9zAZMPr/1OU61Slr0oM1yjneT7LgNTrAD9+QSu7Q0q2u+533FImOu8o/pKG0gNzEtyeSEEsegmCiaxs4Wi+/jVodU8CgfrYN3xkgi7QuD2fMs/zz+sv25LDg8LR1mIvkv+W/PkbqoVx3UAUXL7ZM+UvkFqie+69C/kwoH51xPaYi+FajSzql+VdhDc28aFkwrDApenyLbyKAHSnplWqpFDVtcQK4dL7K9EhRkUnWORNHiYT+N0Ras5tw1ief481nDud29nhlQ+5HIHGK5QouKmA1yJIJqavbzsDf7n+nWCwULIWXGi5d8A23e8vZAuF2CInUHAmP8d+xBDRx8aATZigoOM0GpHn5ljxNs0TCZ1HvQ8TjzQOpcgmBZ9l5Eod3y/PRhjGBRrcR550PAxpkjRj9VjfMN5NSiHVsGw7xKQr2+4yghv3VgSZFvOg1qUxmleJ9khwZq2Gel0uvhGStYw50x/C+EDMaJNQUaBDDbo5Bjy6/RNAEgrX0snQAZWCZSg4bsb4+2jRvo50W1wMr9076qOFViNn2YM+exIGXdmgggAKk7foZDZz1KgG+dAhKBXoUhXwHO9P450gq4xxDGSk0ZCe10OiszNkXVtmmE+Ryi5JjnbDsxLJP1tynrljFUb29VePt0zwZAhdnz0MQU1T9IVbjcKYB92h19lhcdgE="
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove

@@ -250,7 +250,7 @@ class Secrets(UpdateTrackingModel["Secrets"], BaseSettings):
replicate_api_key: str = Field(default="", description="Replicate API Key")
unreal_speech_api_key: str = Field(default="", description="Unreal Speech API Key")
ideogram_api_key: str = Field(default="", description="Ideogram API Key")

openrouter_api_key: str = Field(default="", description="OpenRouter API Key")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please migrate to new credentials system

Copy link
Contributor

@aarushik93 aarushik93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see comments

@aarushik93
Copy link
Contributor

@ymrohit we've now added OpenRouter to the new platform and so this PR is no longer needed....we'd still love for you to contribute other blocks and integrations 🙏🏽

@aarushik93 aarushik93 closed this Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
conflicts Automatically applied to PRs with merge conflicts platform/backend AutoGPT Platform - Back end platform/blocks size/l
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

6 participants