Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 Add Azure Content Moderator service for image analysis #143

Merged
merged 15 commits into from
Aug 18, 2023

Conversation

teresaqhoang
Copy link
Contributor

Motivation and Context

This pull request adds content moderation and image handling features to the project. Specifically, it adds a new ContentModeratorController with endpoints for detecting sensitive image content and getting the status of content moderation. It also adds image text validation to the DocumentImportController, throwing an exception if an image does not contain text.

Additionally, this pull request adds an AzureContentModerator service and a ContentModeratorOptions class for configuring content moderation. The Azure Content Moderator service allows for the analysis of images to detect harmful content, such as hate speech, sexual content, self-harm, and violence.

The pull request also includes updates to the ChatInput and DocumentsTab components in the webapp to support the new features, including the use of the new useContentModerator hook for content moderation and the useFile hook for image upload handling.

Description

webapp

  • Add image moderation and import handling to useFile hook: Add loadImage and handleImport functions to useFile hook.
    • loadImage function loads an image from a URL and returns a promise.
    • handleImport function handles importing a file and returns a promise.
  • Add ContentModerationService for image moderation.
    • analyzeImageAsync method analyzes an image for content moderation.
    • getContentModerationStatusAsync method checks the status of the content moderation service.

webapi

  • The ChatSkill now includes an instance of AzureContentModerator which can be used to moderate user input when an image is uploaded.
  • The appsettings.json file has been updated to include configuration options for the service.
  • The ChatInput component has been updated to include a new hook useContentModerator which handles document import, including using the Azure Content Moderator API to analyze image content for offensive or unwanted elements. - The handleImport function in ChatInput has been removed and replaced with a shared function in useFile that handles file imports. Also refactored document import in DocumentsTab component to use useFile hook.

image

image
AzureContentSafety

Contribution Checklist

@github-actions github-actions bot added webapp Pull requests that update Typescript code webapi Pull requests that update .net code PR: ready for review labels Aug 9, 2023
@teresaqhoang teresaqhoang self-assigned this Aug 9, 2023
@teresaqhoang teresaqhoang marked this pull request as draft August 14, 2023 20:00
@teresaqhoang teresaqhoang added the PR: paused For PRs that have been converted to draft, are facing blockers or have no active development planned label Aug 14, 2023
@teresaqhoang teresaqhoang requested a review from glahaye August 17, 2023 14:32
@teresaqhoang teresaqhoang removed the PR: paused For PRs that have been converted to draft, are facing blockers or have no active development planned label Aug 17, 2023
@teresaqhoang teresaqhoang requested a review from alliscode August 17, 2023 14:33
@teresaqhoang teresaqhoang marked this pull request as ready for review August 17, 2023 14:46
Copy link
Contributor

@crickman crickman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DocumentImportController is within a week or two of being deleted and all of this work will need to be ported to the semantic-memory service. I wonder if just landing this in semantic-memory as a first step doesn't make more sense?

webapi/Controllers/ContentModeratorController.cs Outdated Show resolved Hide resolved
@teresaqhoang teresaqhoang marked this pull request as draft August 17, 2023 15:23
@teresaqhoang teresaqhoang added the PR: paused For PRs that have been converted to draft, are facing blockers or have no active development planned label Aug 17, 2023
@teresaqhoang
Copy link
Contributor Author

DocumentImportController is within a week or two of being deleted and all of this work will need to be ported to the semantic-memory service. I wonder if just landing this in semantic-memory as a first step doesn't make more sense?

As part of the semantic-memory work, nearly all of the image processing logic is being moved to semantic-memory, in the relatively near-term. I wonder if it makes sense to coordinate this feature within that scope?

Definitely makes sense to coordinate this feature within the semantic-memory work. Pausing this PR until then.

crickman
crickman previously approved these changes Aug 17, 2023
@teresaqhoang teresaqhoang marked this pull request as ready for review August 18, 2023 13:54
@teresaqhoang teresaqhoang requested a review from alliscode August 18, 2023 13:54
@teresaqhoang teresaqhoang removed the PR: paused For PRs that have been converted to draft, are facing blockers or have no active development planned label Aug 18, 2023
alliscode
alliscode previously approved these changes Aug 18, 2023
@teresaqhoang teresaqhoang dismissed stale reviews from alliscode and crickman via 5914fcc August 18, 2023 16:18
@teresaqhoang teresaqhoang requested a review from crickman August 18, 2023 16:19
@crickman crickman requested a review from alliscode August 18, 2023 17:13
@teresaqhoang teresaqhoang added this pull request to the merge queue Aug 18, 2023
Merged via the queue into microsoft:main with commit 8bc548f Aug 18, 2023
@teresaqhoang teresaqhoang deleted the azure-content-safety branch August 18, 2023 17:20
teamleader-dev pushed a commit to vlink-group/chat-copilot that referenced this pull request Oct 7, 2024
### Motivation and Context

<!-- Thank you for your contribution to the copilot-chat repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
This pull request adds content moderation and image handling features to
the project. Specifically, it adds a new ContentModeratorController with
endpoints for detecting sensitive image content and getting the status
of content moderation. It also adds image text validation to the
DocumentImportController, throwing an exception if an image does not
contain text.

Additionally, this pull request adds an AzureContentModerator service
and a ContentModeratorOptions class for configuring content moderation.
The Azure Content Moderator service allows for the analysis of images to
detect harmful content, such as hate speech, sexual content, self-harm,
and violence.

The pull request also includes updates to the ChatInput and DocumentsTab
components in the webapp to support the new features, including the use
of the new useContentModerator hook for content moderation and the
useFile hook for image upload handling.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
webapp
- Add image moderation and import handling to useFile hook: Add
loadImage and handleImport functions to useFile hook.
- loadImage function loads an image from a URL and returns a promise.
- handleImport function handles importing a file and returns a promise.
-  Add ContentModerationService for image moderation. 
- analyzeImageAsync method analyzes an image for content moderation.
- getContentModerationStatusAsync method checks the status of the
content moderation service.

webapi
- The `ChatSkill` now includes an instance of `AzureContentModerator`
which can be used to moderate user input when an image is uploaded.
- The `appsettings.json` file has been updated to include configuration
options for the service.
- The `ChatInput` component has been updated to include a new hook
`useContentModerator` which handles document import, including using the
Azure Content Moderator API to analyze image content for offensive or
unwanted elements. - The `handleImport` function in `ChatInput` has been
removed and replaced with a shared function in `useFile` that handles
file imports. Also refactored document import in DocumentsTab component
to use useFile hook.


![image](https://github.com/microsoft/semantic-kernel/assets/125500434/7d9b3ae0-9974-4733-831a-a980bffb43f5)


![image](https://github.com/microsoft/semantic-kernel/assets/125500434/712137cb-72f8-4e8f-ad85-d63fad6ecc87)

![AzureContentSafety](https://github.com/microsoft/semantic-kernel/assets/125500434/3fd59bae-2a9c-4a7f-af04-85367be3631e)

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
~~- [ ] All unit tests pass, and I have added new tests where possible~~
- [x] I didn't break anyone 😄
kb0039 pushed a commit to aaronba/chat-copilot that referenced this pull request Jan 8, 2025
### Motivation and Context

<!-- Thank you for your contribution to the copilot-chat repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
This pull request adds content moderation and image handling features to
the project. Specifically, it adds a new ContentModeratorController with
endpoints for detecting sensitive image content and getting the status
of content moderation. It also adds image text validation to the
DocumentImportController, throwing an exception if an image does not
contain text.

Additionally, this pull request adds an AzureContentModerator service
and a ContentModeratorOptions class for configuring content moderation.
The Azure Content Moderator service allows for the analysis of images to
detect harmful content, such as hate speech, sexual content, self-harm,
and violence.

The pull request also includes updates to the ChatInput and DocumentsTab
components in the webapp to support the new features, including the use
of the new useContentModerator hook for content moderation and the
useFile hook for image upload handling.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
webapp
- Add image moderation and import handling to useFile hook: Add
loadImage and handleImport functions to useFile hook.
- loadImage function loads an image from a URL and returns a promise.
- handleImport function handles importing a file and returns a promise.
-  Add ContentModerationService for image moderation. 
- analyzeImageAsync method analyzes an image for content moderation.
- getContentModerationStatusAsync method checks the status of the
content moderation service.

webapi
- The `ChatSkill` now includes an instance of `AzureContentModerator`
which can be used to moderate user input when an image is uploaded.
- The `appsettings.json` file has been updated to include configuration
options for the service.
- The `ChatInput` component has been updated to include a new hook
`useContentModerator` which handles document import, including using the
Azure Content Moderator API to analyze image content for offensive or
unwanted elements. - The `handleImport` function in `ChatInput` has been
removed and replaced with a shared function in `useFile` that handles
file imports. Also refactored document import in DocumentsTab component
to use useFile hook.


![image](https://github.com/microsoft/semantic-kernel/assets/125500434/7d9b3ae0-9974-4733-831a-a980bffb43f5)


![image](https://github.com/microsoft/semantic-kernel/assets/125500434/712137cb-72f8-4e8f-ad85-d63fad6ecc87)

![AzureContentSafety](https://github.com/microsoft/semantic-kernel/assets/125500434/3fd59bae-2a9c-4a7f-af04-85367be3631e)

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
~~- [ ] All unit tests pass, and I have added new tests where possible~~
- [x] I didn't break anyone 😄
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
webapi Pull requests that update .net code webapp Pull requests that update Typescript code
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

5 participants