Customized category / customized model related spec changes (#45)

* Add ga contract for Content Safety * Update examples for 2023-10-01 * Resolve ModelValidation SpellCheck and Avocado * Fix typo * Fix minor errors in example and readme * Fix example * remove the 0430 content * update readme * re-format markdowm * Fix example * init content safety 2023-10-15-preview * init content safety 2023-10-15-previwe * update * format example * update read me file * update * update * update contract * update * update * update threshold * update new contract by add judgement kind enum * update description * u * update * updat length * update * update * update description for accept severity threshodl * update * merge with acs-ga * Add snippy contract * remove 8 sev levels for image * update contentsafety.json * Update snippy path * add type spec for jailbreak * Add incident response API in 2023-10-30-preview * compiled * Adaptive annotation service contract * Remove operations * update * update * refine * update * make 0,1,2,3,4,5,6,7 to 0, 1, 2, 3, ... and add comman in last enum value * update path * Update NL Snippy path * Resolve conflict * add * add * remove labelName * Fix route and minor errors * Change typo * update * Update protected material API * update project sample * Change path name * Update protect material contract * update jailbreak * Update incident contract * Update contract * Update * scenarios to enumerations (#19) * Remove project related contract (#20) * Remove project related things * Remove accepted * Remove examples * Merge 1015 (#22) * Add ga contract for Content Safety * Update examples for 2023-10-01 * Resolve ModelValidation SpellCheck and Avocado * Fix typo * Fix minor errors in example and readme * Fix example * remove the 0430 content * update readme * re-format markdowm * Fix example * init content safety 2023-10-15-preview * init content safety 2023-10-15-previwe * update * format example * update read me file * update * update * update contract * update * update * update threshold * update new contract by add judgement kind enum * update description * u * update * updat length * update * update * update description for accept severity threshodl * update * merge with acs-ga * Add snippy contract * remove 8 sev levels for image * update contentsafety.json * Update snippy path * add type spec for jailbreak * compiled * update * update * refine * update * make 0,1,2,3,4,5,6,7 to 0, 1, 2, 3, ... and add comman in last enum value * update path * Update NL Snippy path * Resolve conflict * Update protected material API * update project sample * Change path name * Update protect material contract * update jailbreak * Remove project related contract (#20) * Remove project related things * Remove accepted * Remove examples --------- Co-authored-by: FAREAST\suxi <suxi@microsoft.com> Co-authored-by: Lixiang Liu <lixiangliu@microsoft.com> Co-authored-by: Lixiang Liu <liulixiang1988@gmail.com> Co-authored-by: zhaiyutong <mo.huainian@163.com> * Add list API for Annotation Service (#23) * Add ga contract for Content Safety * Update examples for 2023-10-01 * Resolve ModelValidation SpellCheck and Avocado * Fix typo * Fix minor errors in example and readme * Fix example * remove the 0430 content * update readme * re-format markdowm * Fix example * init content safety 2023-10-15-preview * init content safety 2023-10-15-previwe * update * format example * update read me file * update * update * update contract * update * update * update threshold * update new contract by add judgement kind enum * update description * u * update * updat length * update * update * update description for accept severity threshodl * update * merge with acs-ga * Add snippy contract * remove 8 sev levels for image * update contentsafety.json * Update snippy path * add type spec for jailbreak * compiled * update * update * refine * update * make 0,1,2,3,4,5,6,7 to 0, 1, 2, 3, ... and add comman in last enum value * update path * Update NL Snippy path * Resolve conflict * Update protected material API * update project sample * Change path name * Update protect material contract * update jailbreak * Remove project related contract (#20) * Remove project related things * Remove accepted * Remove examples * Add list category APIs --------- Co-authored-by: FAREAST\suxi <suxi@microsoft.com> Co-authored-by: Lixiang Liu <lixiangliu@microsoft.com> Co-authored-by: Lixiang Liu <liulixiang1988@gmail.com> Co-authored-by: zhaiyutong <mo.huainian@163.com> * Update contract for incident response (#24) Co-authored-by: Lixiang Liu <lixiangliu@microsoft.com> * scenarios to enumerations (#25) * [Cognitive Services - Azure AI Content Safety] - 2023-11-30-preview Private Preview * update doc * resolve comments. * Merge main and recompile swagger * add explanation for 2023-11-30-preview * format * generate swagger * Optimize contract (#36) * add 12-30-preview, add explanation spec * update * renamed reason -> harmful chunks * update. * remove new version enum from GA version * Remove examples from 1015 * restore examples * Add examples for jailbreak and protected material * run prettier for example * commit add enum to stable * Add omit-unreachable-types: true to typespec * compile preview * Fix readme * renamed to citation * UPDATE: add reasoning column and update some docs. * update doc. * update api version. * Fix some typo issues. (#40) * Update data plane. * draft * draft * draft * draft * draft * draft * draft * draft * draft * draft * draft * update bd draft * update spec draft * update spec draft * update spec draft --------- Co-authored-by: mengaims <mengai@microsoft.com> Co-authored-by: FAREAST\suxi <suxi@microsoft.com> Co-authored-by: Lixiang Liu <lixiangliu@microsoft.com> Co-authored-by: Lixiang Liu <liulixiang1988@gmail.com> Co-authored-by: zhaiyutong <mo.huainian@163.com> Co-authored-by: mengaims <50131889+mengaims@users.noreply.github.com> Co-authored-by: YUTONG_ZHAI <32332316+zhaiyutong@users.noreply.github.com> Co-authored-by: Jie Zhou <zhojie@microsoft.com> Co-authored-by: Yuxiao Wang <yuxiaowang@microsoft.com> Co-authored-by: Yuxiao Wang <58904990+Sayaka-4987@users.noreply.github.com> Co-authored-by: jiezhou_microsoft <94604058+JieZhou000@users.noreply.github.com> Co-authored-by: David Liang <liangze@microsoft.com>
mengaims · Jan 11, 2024 · f76e444 · f76e444
1 parent 8775b51
commit f76e444
Show file tree

Hide file tree

Showing 4 changed files with 490 additions and 53 deletions.
diff --git a/specification/cognitiveservices/ContentSafety/models.tsp b/specification/cognitiveservices/ContentSafety/models.tsp
@@ -40,6 +40,14 @@ enum AnalyzeImageOutputType {
   FourSeverityLevels,
 }
 
+@added(ContentSafety.Versions.v2024_01_30_Preview)
+enum ModelStatus {
+  created,
+  training,
+  ready,
+  failed,
+}
+
 @doc("The text analysis request.")
 model AnalyzeTextOptions {
   @doc("The text needs to be analyzed. We support a maximum of 10k Unicode characters (Unicode code points) in the text of one request.")
@@ -61,6 +69,20 @@ model AnalyzeTextOptions {
   @added(ContentSafety.Versions.v2023_10_30_Preview)
   @doc("The incidents to detect.")
   incidents?: IncidentOptions;
+
+  @added(ContentSafety.Versions.v2024_01_30_Preview)
+  @doc("The customized categories will be analyzed. If they are not assigned, the analysis results for all available customized categories will be returned.")
+  customizedCategories?: CustomizedCategory[];
+}
+
+@added(ContentSafety.Versions.v2024_01_30_Preview)
+@doc("Customized category for inference.")
+model CustomizedCategory {
+  @doc("Category name")
+  categoryName: string;
+
+  @doc("Model ID for the category. If not provided, the latest available model will be used.")
+  modelId: string;
 }
 
 @added(ContentSafety.Versions.v2023_10_30_Preview)
@@ -88,6 +110,28 @@ model AnalyzeTextResult {
   @added(ContentSafety.Versions.v2024_01_30_Preview)
   @doc("Chunks in the original text detected as harmful content. Analysis result and scores are caused by these.")
   citation?: string[];
+
+  @added(ContentSafety.Versions.v2024_01_30_Preview)
+  @doc("Analysis result for customized categories.")
+  customizedCategoriesAnalysis: TextCustomizedCategoryAnalysis[];
+}
+
+@added(ContentSafety.Versions.v2024_01_30_Preview)
+@doc("Text customized category analysis result.")
+model TextCustomizedCategoryAnalysis {
+  @doc("Customized category name")
+  categoryName: string;
+
+  @doc("customized category analysis")
+  customizedCategoriesAnalysis: TextCustomizedCategoryClassResult[];
+}
+
+@added(ContentSafety.Versions.v2024_01_30_Preview)
+@doc("customized category analysis result")
+model TextCustomizedCategoryClassResult {
+  id: int32;
+  @renamedFrom(Versions.v2024_01_30_Preview, "name")
+  className: string;
 }
 
 @doc("The result of blocklist match.")
@@ -456,9 +500,11 @@ model PreDefinedConcept {
 
 @added(Versions.v2023_10_30_Preview)
 @doc("Label definition.")
-model SubCategory {
+@renamedFrom(Versions.v2024_01_30_Preview, "SubCategory")
+model Class {
   id: int32;
-  name: string;
+  @renamedFrom(Versions.v2024_01_30_Preview, "name")
+  className: string;
   statements: string[];
 }
 
@@ -473,12 +519,39 @@ model TextCustomizedCategory {
   @maxLength(64)
   categoryName: string;
 
+  @doc("A predefined concept with its definition can be used directly in the subcategories.")
+  @added(Versions.v2023_10_30_Preview)
+  @removed(Versions.v2024_01_30_Preview)
   preDefinedConcepts?: PreDefinedConcept[];
-  subCategories: SubCategory[];
+
+  @doc("Subcategories are detailed classes of the category. It is required there is a class 'safe' with id=0.")
+  @renamedFrom(Versions.v2024_01_30_Preview, "subCategories")
+  classes: Class[];
+
+  @added(Versions.v2023_10_30_Preview)
+  @removed(Versions.v2024_01_30_Preview)
   emphases?: string[];
+
+  @doc("Blob that stores the examples.")
   exampleBlobUrl?: url;
 }
 
+@added(ContentSafety.Versions.v2024_01_30_Preview)
+@doc("The response for async training triggering.")
+model SubmitModelTrainingResponse {
+  ...AcceptedResponse;
+
+  @header("operation-location")
+  @doc("The location of the status API for monitoring the triggered model.")
+  modelUrl: string;
+}
+
+@added(ContentSafety.Versions.v2024_01_30_Preview)
+model AsyncJobWarning {
+  code: string;
+  messgae: string;
+}
+
 #suppress "@azure-tools/typespec-azure-core/documentation-required" "MUST fix in next update"
 @added(ContentSafety.Versions.v2023_10_30_Preview)
 @doc("ImageWithText analyze category.")
@@ -489,6 +562,19 @@ enum ImageWithTextCategory {
   Violence,
 }
 
+@added(ContentSafety.Versions.v2024_01_30_Preview)
+@parentResource(TextCustomizedCategory)
+@resource("models")
+model TextTrainedModel {
+  @key
+  modelId: string;
+  status: ModelStatus;
+  createdTime: utcDateTime;
+  lastUpdateTime: utcDateTime;
+  warnings?: AsyncJobWarning[];
+  errors?: string[];
+}
+
 @added(ContentSafety.Versions.v2023_10_30_Preview)
 @doc("The analysis request of the image with text.")
 model AnalyzeImageWithTextOptions {

diff --git a/specification/cognitiveservices/ContentSafety/routes.tsp b/specification/cognitiveservices/ContentSafety/routes.tsp
@@ -290,6 +290,37 @@ interface TextCategoryCustomization {
     TextCustomizedCategory,
     ListQueryParametersTrait<StandardListQueryParameters>
   >;
+
+  @action("trainModel")
+  @added(Versions.v2024_01_30_Preview)
+  trainModel is CategoryOps.ResourceAction<
+    TextCustomizedCategory,
+    {},
+    SubmitModelTrainingResponse
+  >;
+} 
+
+interface TextCustomizedMdoelOps
+  extends Azure.Core.ResourceOperations<NoRepeatableRequests &
+      NoConditionalRequests &
+      NoClientRequestId> {}
+
+@added(Versions.v2024_01_30_Preview)
+interface TextCustomizedModelOperations {
+  @summary("List all existing models")
+  @doc("List existing models.")
+  listTextCustomizedModels is TextCustomizedMdoelOps.ResourceList<
+    TextTrainedModel,
+    ListQueryParametersTrait<StandardListQueryParameters>
+  >;
+
+  @doc("Get a model that is not deleted.")
+  getTextCustomizedModels is TextCustomizedMdoelOps.ResourceRead<TextTrainedModel>;
+
+  @doc("If the model is training, the training process will be cancelled and the model will also be deleted.")
+  deleteCustomizedModel is TextCustomizedMdoelOps.ResourceDelete<
+    TextTrainedModel
+  >;
 }
 
 @added(Versions.v2023_10_30_Preview)

diff --git a/.../cognitiveservices/data-plane/ContentSafety/preview/2023-10-30-preview/contentsafety.json b/.../cognitiveservices/data-plane/ContentSafety/preview/2023-10-30-preview/contentsafety.json
@@ -2931,13 +2931,15 @@
         },
         "preDefinedConcepts": {
           "type": "array",
+          "description": "A predefined concept with its definition can be used directly in the subcategories.",
           "items": {
             "$ref": "#/definitions/PreDefinedConcept"
           },
           "x-ms-identifiers": []
         },
         "subCategories": {
           "type": "array",
+          "description": "Subcategories are detailed classes of the category. It is required there is a class 'safe' with id=0.",
           "items": {
             "$ref": "#/definitions/SubCategory"
           }
@@ -2950,7 +2952,8 @@
         },
         "exampleBlobUrl": {
           "type": "string",
-          "format": "uri"
+          "format": "uri",
+          "description": "Blob that stores the examples."
         }
       },
       "required": [