Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AI assistant #2589

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion code_samples/ai_actions/assets/js/addAudioModule.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { addModule } from '../../vendor/ibexa/connector-ai/src/bundle/Resources/public/js/core/create.ai.module';
import { addModule } from '@ibexa-connector-ai/src/bundle/Resources/public/js/core/create.ai.module';
import TranscribeAudio from './transcribe.audio';

addModule(TranscribeAudio);
44 changes: 23 additions & 21 deletions code_samples/ai_actions/assets/js/transcribe.audio.js
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
import BaseAIComponent from '../../vendor/ibexa/connector-ai/src/bundle/Resources/public/js/core/base.ai.component';
import BaseAIAssistantComponent from '@ibexa-connector-ai/src/bundle/Resources/public/js/core/base.ai.assistant.component';
import Textarea from '@ibexa-connector-ai-modules/ai-assistant/fields/textarea/textarea';

export default class TranscribeAudio extends BaseAIComponent {
constructor(mainElement, config) {
super(mainElement, config);
export default class TranscribeAudio extends BaseAIAssistantComponent {
constructor(mainElement, extraConfig) {
super(mainElement, extraConfig);

this.requestHeaders = {
Accept: 'application/vnd.ibexa.api.ai.AudioText+json',
'Content-Type': 'application/vnd.ibexa.api.ai.TranscribeAudio+json',
};

this.getRequestBody = this.getRequestBody.bind(this);
this.getResponseValue = this.getResponseValue.bind(this);

this.replacedField = Textarea;
}

getAudioInBase64() {
Expand All @@ -18,16 +24,15 @@ export default class TranscribeAudio extends BaseAIComponent {

if (request.status === 200) {
return this.convertToBase64(request.responseText);
} else {
this.processError('Error occured when decoding the file.');
}
}

getRequestBody() {
const inputValue = this.getInputValue();
const body = {
TranscribeAudio: {
Audio: {
base64: this.getAudioInBase64(),
base64: inputValue,
},
RuntimeContext: {},
},
Expand All @@ -40,20 +45,6 @@ export default class TranscribeAudio extends BaseAIComponent {
return JSON.stringify(body);
}

afterFetchData(response) {
super.afterFetchData();

if (response) {
this.outputElement.value = response.AudioText.Text.text[0];
}
}

toggle(forceEnabled) {
super.toggle(forceEnabled);

this.outputElement.disabled = !forceEnabled || !this.outputElement.disabled;
}

convertToBase64(data) {
let binary = '';

Expand All @@ -63,4 +54,15 @@ export default class TranscribeAudio extends BaseAIComponent {

return btoa(binary);
}

getResponseValue(response) {
return response.AudioText.Text.text[0];
}

handleAIDialogConfirm(responseText) {
this.outputElement.value = responseText;
this.outputElement.dispatchEvent(new Event('input'));

super.handleAIDialogClose(responseText);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ ibexa:
admin_ui_forms:
content_edit:
form_templates:
- { template: '@ibexadesign/admin/ui/fieldtype/edit/form_fields_binary_ai.html.twig', priority: -10 } }
- { template: '@ibexadesign/admin/ui/fieldtype/edit/form_fields_binary_ai.html.twig', priority: -10 }
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,27 @@
{% block ezbinaryfile_preview %}
{{ parent() }}

{% import '@ibexadesign/connector_ai/ui/ai_module/macros.html.twig' as ai_macros %}

{% set transcriptFieldIdentifier = 'transcript' %}
{% set fieldTypeIdentifiers = form.parent.parent.vars.value|keys %}

{% if transcriptFieldIdentifier in fieldTypeIdentifiers %}
{% set module_id = 'TranscribeAudio' %}
{% set ai_config_id = 'transcribe_audio' %}
{% set container_selector = '.ibexa-edit-content' %}
{% set input_selector = '.ibexa-field-edit-preview__action--preview' %}
{% set output_selector = '#ezplatform_content_forms_content_edit_fieldsData_transcript_value' %}
{% set cancel_wrapper_selector = '.ibexa-field-edit-preview__media-wrapper' %}

{% embed '@ibexadesign/connector_ai/ui/ai_module/ai_component.html.twig' with {
ai_config_id,
container_selector,
input_selector,
output_selector,
{% set use_ai_btn_attr = {
class: 'btn ibexa-btn ibexa-btn--secondary ibexa-ai-component--custom-btn',
module_id: 'TranscribeAudio',
scroll_selector: '.ibexa-edit-content',
container_selector: '.ibexa-edit-content',
input_selector: '.ibexa-field-edit-preview__action--preview',
output_selector: '#ezplatform_content_forms_content_edit_fieldsData_transcript_value',
ai_config_id: 'transcribe_audio',
} %}
{% endembed %}

<button {{ ai_macros.attributes(use_ai_btn_attr) }}>
<svg class="ibexa-icon ibexa-icon--small ibexa-icon--primary">
<use xlink:href="{{ ibexa_icon_path('explore') }}"></use>
</svg>
{{ 'ibexa_connector_ai.use_ai.label'|trans({}, 'ibexa_connector_ai')|desc('Use AI') }}
</button>
{% endif %}
{% endblock %}
4 changes: 2 additions & 2 deletions docs/ai_actions/ai_actions_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,11 +133,11 @@ Out of the box, after you configure access to the OpenAI service, the [[= produc

Content editors can benefit from using AI capabilities to [enhance or modify text]([[= user_doc =]]/content_management/create_edit_content_items/#ai-assistant).
With a few clicks, they can improve content quality or reduce the workload.
While working on content, editors can select a specific passage and request that AI performs specific actions such as: adjusting the length of the text, changing the tone, or correcting linguistic errors.
While working on content, editors can request that AI performs specific actions such as: adjusting the length of the text, changing the tone, or correcting linguistic errors.

![AI Assistant](img/ai_assistant.png)

This functionality is available in content types that include a RichText field, and certain Page Builder blocks.
This functionality is available in content types that include RichText, Text line, Text Block fields, and certain Page Builder blocks.

### Generating alternative text

Expand Down
Binary file modified docs/ai_actions/img/transcribe_audio.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading