forked from microsoft/chat-copilot
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
🎉 Add Azure Content Moderator service for image analysis (microsoft#143)
### Motivation and Context <!-- Thank you for your contribution to the copilot-chat repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> This pull request adds content moderation and image handling features to the project. Specifically, it adds a new ContentModeratorController with endpoints for detecting sensitive image content and getting the status of content moderation. It also adds image text validation to the DocumentImportController, throwing an exception if an image does not contain text. Additionally, this pull request adds an AzureContentModerator service and a ContentModeratorOptions class for configuring content moderation. The Azure Content Moderator service allows for the analysis of images to detect harmful content, such as hate speech, sexual content, self-harm, and violence. The pull request also includes updates to the ChatInput and DocumentsTab components in the webapp to support the new features, including the use of the new useContentModerator hook for content moderation and the useFile hook for image upload handling. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> webapp - Add image moderation and import handling to useFile hook: Add loadImage and handleImport functions to useFile hook. - loadImage function loads an image from a URL and returns a promise. - handleImport function handles importing a file and returns a promise. - Add ContentModerationService for image moderation. - analyzeImageAsync method analyzes an image for content moderation. - getContentModerationStatusAsync method checks the status of the content moderation service. webapi - The `ChatSkill` now includes an instance of `AzureContentModerator` which can be used to moderate user input when an image is uploaded. - The `appsettings.json` file has been updated to include configuration options for the service. - The `ChatInput` component has been updated to include a new hook `useContentModerator` which handles document import, including using the Azure Content Moderator API to analyze image content for offensive or unwanted elements. - The `handleImport` function in `ChatInput` has been removed and replaced with a shared function in `useFile` that handles file imports. Also refactored document import in DocumentsTab component to use useFile hook.    ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [Contribution Guidelines](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md#development-scripts) raises no violations ~~- [ ] All unit tests pass, and I have added new tests where possible~~ - [x] I didn't break anyone 😄
- Loading branch information
1 parent
b072121
commit b2e92fb
Showing
17 changed files
with
533 additions
and
99 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| // Copyright (c) Microsoft. All rights reserved. | ||
|
|
||
| using System.Text.Json.Serialization; | ||
| using CopilotChat.WebApi.Services; | ||
|
|
||
| namespace CopilotChat.WebApi.Models.Response; | ||
|
|
||
| /// <summary> | ||
| /// Response definition to the /contentsafety/image:analyze | ||
| /// endpoint made by the AzureContentSafety. | ||
| /// </summary> | ||
| public class ImageAnalysisResponse | ||
| { | ||
| /// <summary> | ||
| /// Gets or sets the AnalysisResult related to hate. | ||
| /// </summary> | ||
| [JsonPropertyName("hateResult")] | ||
| public AnalysisResult? HateResult { get; set; } | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the AnalysisResult related to self-harm. | ||
| /// </summary> | ||
| [JsonPropertyName("selfHarmResult")] | ||
| public AnalysisResult? SelfHarmResult { get; set; } | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the AnalysisResult related to sexual content. | ||
| /// </summary> | ||
| [JsonPropertyName("sexualResult")] | ||
| public AnalysisResult? SexualResult { get; set; } | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the AnalysisResult related to violence. | ||
| /// </summary> | ||
| [JsonPropertyName("violenceResult")] | ||
| public AnalysisResult? ViolenceResult { get; set; } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| // Copyright (c) Microsoft. All rights reserved. | ||
|
|
||
| using System; | ||
| using System.ComponentModel.DataAnnotations; | ||
|
|
||
| namespace CopilotChat.WebApi.Options; | ||
|
|
||
| /// <summary> | ||
| /// Configuration options for content safety. | ||
| /// </summary> | ||
| public class ContentSafetyOptions | ||
| { | ||
| public const string PropertyName = "ContentSafety"; | ||
|
|
||
| /// <summary> | ||
| /// Whether to enable content safety. | ||
| /// </summary> | ||
| [Required, NotEmptyOrWhitespace] | ||
| public bool Enabled { get; set; } = false; | ||
|
|
||
| /// <summary> | ||
| /// Azure Content Safety endpoints | ||
| /// </summary> | ||
| [RequiredOnPropertyValue(nameof(Enabled), true)] | ||
| public string Endpoint { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Key to access the content safety service. | ||
| /// </summary> | ||
| [RequiredOnPropertyValue(nameof(Enabled), true)] | ||
| public string Key { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Set the violation threshold. See https://learn.microsoft.com/en-us/azure/ai-services/content-safety/quickstart-image for details. | ||
| /// </summary> | ||
| [Range(0, 6)] | ||
| public short ViolationThreshold { get; set; } = 4; | ||
| } |
Oops, something went wrong.