-
Notifications
You must be signed in to change notification settings - Fork 710
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🎉 Add Azure Content Moderator service for image analysis #143
🎉 Add Azure Content Moderator service for image analysis #143
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DocumentImportController is within a week or two of being deleted and all of this work will need to be ported to the semantic-memory service. I wonder if just landing this in semantic-memory as a first step doesn't make more sense?
Definitely makes sense to coordinate this feature within the semantic-memory work. Pausing this PR until then. |
### Motivation and Context <!-- Thank you for your contribution to the copilot-chat repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This pull request adds content moderation and image handling features to the project. Specifically, it adds a new ContentModeratorController with endpoints for detecting sensitive image content and getting the status of content moderation. It also adds image text validation to the DocumentImportController, throwing an exception if an image does not contain text. Additionally, this pull request adds an AzureContentModerator service and a ContentModeratorOptions class for configuring content moderation. The Azure Content Moderator service allows for the analysis of images to detect harmful content, such as hate speech, sexual content, self-harm, and violence. The pull request also includes updates to the ChatInput and DocumentsTab components in the webapp to support the new features, including the use of the new useContentModerator hook for content moderation and the useFile hook for image upload handling. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> webapp - Add image moderation and import handling to useFile hook: Add loadImage and handleImport functions to useFile hook. - loadImage function loads an image from a URL and returns a promise. - handleImport function handles importing a file and returns a promise. - Add ContentModerationService for image moderation. - analyzeImageAsync method analyzes an image for content moderation. - getContentModerationStatusAsync method checks the status of the content moderation service. webapi - The `ChatSkill` now includes an instance of `AzureContentModerator` which can be used to moderate user input when an image is uploaded. - The `appsettings.json` file has been updated to include configuration options for the service. - The `ChatInput` component has been updated to include a new hook `useContentModerator` which handles document import, including using the Azure Content Moderator API to analyze image content for offensive or unwanted elements. - The `handleImport` function in `ChatInput` has been removed and replaced with a shared function in `useFile` that handles file imports. Also refactored document import in DocumentsTab component to use useFile hook. ![image](https://github.com/microsoft/semantic-kernel/assets/125500434/7d9b3ae0-9974-4733-831a-a980bffb43f5) ![image](https://github.com/microsoft/semantic-kernel/assets/125500434/712137cb-72f8-4e8f-ad85-d63fad6ecc87) ![AzureContentSafety](https://github.com/microsoft/semantic-kernel/assets/125500434/3fd59bae-2a9c-4a7f-af04-85367be3631e) ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [Contribution Guidelines](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md#development-scripts) raises no violations ~~- [ ] All unit tests pass, and I have added new tests where possible~~ - [x] I didn't break anyone 😄
### Motivation and Context <!-- Thank you for your contribution to the copilot-chat repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This pull request adds content moderation and image handling features to the project. Specifically, it adds a new ContentModeratorController with endpoints for detecting sensitive image content and getting the status of content moderation. It also adds image text validation to the DocumentImportController, throwing an exception if an image does not contain text. Additionally, this pull request adds an AzureContentModerator service and a ContentModeratorOptions class for configuring content moderation. The Azure Content Moderator service allows for the analysis of images to detect harmful content, such as hate speech, sexual content, self-harm, and violence. The pull request also includes updates to the ChatInput and DocumentsTab components in the webapp to support the new features, including the use of the new useContentModerator hook for content moderation and the useFile hook for image upload handling. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> webapp - Add image moderation and import handling to useFile hook: Add loadImage and handleImport functions to useFile hook. - loadImage function loads an image from a URL and returns a promise. - handleImport function handles importing a file and returns a promise. - Add ContentModerationService for image moderation. - analyzeImageAsync method analyzes an image for content moderation. - getContentModerationStatusAsync method checks the status of the content moderation service. webapi - The `ChatSkill` now includes an instance of `AzureContentModerator` which can be used to moderate user input when an image is uploaded. - The `appsettings.json` file has been updated to include configuration options for the service. - The `ChatInput` component has been updated to include a new hook `useContentModerator` which handles document import, including using the Azure Content Moderator API to analyze image content for offensive or unwanted elements. - The `handleImport` function in `ChatInput` has been removed and replaced with a shared function in `useFile` that handles file imports. Also refactored document import in DocumentsTab component to use useFile hook. ![image](https://github.com/microsoft/semantic-kernel/assets/125500434/7d9b3ae0-9974-4733-831a-a980bffb43f5) ![image](https://github.com/microsoft/semantic-kernel/assets/125500434/712137cb-72f8-4e8f-ad85-d63fad6ecc87) ![AzureContentSafety](https://github.com/microsoft/semantic-kernel/assets/125500434/3fd59bae-2a9c-4a7f-af04-85367be3631e) ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [Contribution Guidelines](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md#development-scripts) raises no violations ~~- [ ] All unit tests pass, and I have added new tests where possible~~ - [x] I didn't break anyone 😄
Motivation and Context
This pull request adds content moderation and image handling features to the project. Specifically, it adds a new ContentModeratorController with endpoints for detecting sensitive image content and getting the status of content moderation. It also adds image text validation to the DocumentImportController, throwing an exception if an image does not contain text.
Additionally, this pull request adds an AzureContentModerator service and a ContentModeratorOptions class for configuring content moderation. The Azure Content Moderator service allows for the analysis of images to detect harmful content, such as hate speech, sexual content, self-harm, and violence.
The pull request also includes updates to the ChatInput and DocumentsTab components in the webapp to support the new features, including the use of the new useContentModerator hook for content moderation and the useFile hook for image upload handling.
Description
webapp
webapi
ChatSkill
now includes an instance ofAzureContentModerator
which can be used to moderate user input when an image is uploaded.appsettings.json
file has been updated to include configuration options for the service.ChatInput
component has been updated to include a new hookuseContentModerator
which handles document import, including using the Azure Content Moderator API to analyze image content for offensive or unwanted elements. - ThehandleImport
function inChatInput
has been removed and replaced with a shared function inuseFile
that handles file imports. Also refactored document import in DocumentsTab component to use useFile hook.Contribution Checklist
- [ ] All unit tests pass, and I have added new tests where possible