colab-enterprise/Campaign-Assets-Text-to-Video-01.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color='#4285f4'>Overview</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This demo showcases the end-to-end creation of marketing videos using the power of generative AI. Leveraging a combination of Gemini and Vertex AI's, the process begins by brainstorming video concepts and crafting tailored text-to-video prompts. Text-to-Video then generates short video segments based on these prompts, which are seamlessly stitched together. Gemini then analyzes the compiled video to generate engaging voice over scripts. Finally, we utilize text-to-speech technology to create narrations in both English and French, resulting in a complete and captivating marketing video.\n",
    "\n",
    "Process Flow:\n",
    "\n",
    "1. Transform the text-to-video comprehensive documentation into a set of clear instructions that specifically guide the creation of effective text-to-video prompts.\n",
    "2. Create several text-to-video prompts using Gemini (pass in the text-to-video instructions). \n",
    "    * a. A prompt should include \"styles\", \"composition\", \"Ambiance & Emotions\", \"Cinematic effects\" (doc). By having Gemini author our text-to-video prompt we can get new ideas. Also, by passing in the text-to-video instructions our prompt will use the proper techniques for text-to-video.\n",
    "3. Call GenAI to generate videos via text-to-video.\n",
    "    * a. Currently, GenAI creates 6 second videos. So we prompt Gemini for 6 second segments.\n",
    "4. Merge the videos into one video.\n",
    "5. Ask Gemini to watch the merged video and create a voice over script.\n",
    "6. Use Vertex AI text-to-speech to generate the voice over using a British accent and a male voice.\n",
    "7. Use Vertex AI text-to-speech to generate the voice over using a French accent and a male voice.\n",
    "8. Overlay the text-to-speech to merged video.\n",
    "\n",
    "Notes:\n",
    "\n",
    "1. The notebook does merge the videos, but the quality is much better when you merge with a high quality video tool.\n",
    "2. The text-to-speech does not always align perfectly with the video.\n",
    "3. You can also adjust the speed of the text-to-speech to control how fast the words are spoken.\n",
    "\n",
    "Cost:\n",
    "* Unknown: Please check the cost to use text-to-video \n",
    "* Medium: Remember to stop your Colab Enterprise Notebook Runtime\n",
    "\n",
    "Author: \n",
    "* Adam Paternostro"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Architecture Diagram\n",
    "from IPython.display import Image\n",
    "Image(url='https://storage.googleapis.com/data-analytics-golden-demo/chocolate-ai/v1/Artifacts/Campaign-Assets-Text-to-Video-01-Architecture.png', width=1200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color='#4285f4'>Video Walkthrough</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Video](https://storage.googleapis.com/data-analytics-golden-demo/chocolate-ai/v1/Videos/adam-paternostro-video.png)](https://storage.googleapis.com/data-analytics-golden-demo/chocolate-ai/v1/Videos/Campaign-Assets-Text-to-Video-01.mp4)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import HTML\n",
    "\n",
    "HTML(\"\"\"\n",
    "<video width=\"800\" height=\"600\" controls>\n",
    "  <source src=\"https://storage.googleapis.com/data-analytics-golden-demo/chocolate-ai/v1/Videos/Campaign-Assets-Text-to-Video-01.mp4\" type=\"video/mp4\">\n",
    "  Your browser does not support the video tag.\n",
    "</video>\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HMsUvoF4BP7Y"
   },
   "source": [
    "### <font color='#4285f4'>License</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jQgQkbOvj55d"
   },
   "source": [
    "```\n",
    "# Copyright 2024 Google LLC\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m65vp54BUFRi"
   },
   "source": [
    "### <font color='#4285f4'>Pip installs</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "5MaWM6H5i6rX"
   },
   "outputs": [],
   "source": [
    "# PIP Installs\n",
    "import sys\n",
    "\n",
    "# https://pypi.org/project/moviepy/\n",
    "!{sys.executable} -m pip install moviepy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UmyL-Rg4Dr_f"
   },
   "source": [
    "### <font color='#4285f4'>Initialize</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xOYsEVSXp6IP"
   },
   "outputs": [],
   "source": [
    "from PIL import Image\n",
    "from IPython.display import HTML\n",
    "from IPython.display import Audio\n",
    "from functools import reduce\n",
    "import IPython.display\n",
    "import google.auth\n",
    "import requests\n",
    "import json\n",
    "import uuid\n",
    "import base64\n",
    "import os\n",
    "import cv2\n",
    "import random\n",
    "import time\n",
    "import datetime\n",
    "import base64\n",
    "import random\n",
    "import datetime\n",
    "\n",
    "import logging\n",
    "from tenacity import retry, wait_exponential, stop_after_attempt, before_sleep_log, retry_if_exception"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wMlHl3bnkFPZ"
   },
   "outputs": [],
   "source": [
    "# Set these (run this cell to verify the output)\n",
    "\n",
    "bigquery_location = \"${bigquery_location}\"\n",
    "region = \"${region}\"\n",
    "location = \"${location}\"\n",
    "storage_account = \"${chocolate_ai_bucket}\"\n",
    "dataset_name = \"${bigquery_chocolate_ai_dataset}\"\n",
    "public_storage_storage_account = \"data-analytics-golden-demo\"\n",
    "\n",
    "# Get the current date and time\n",
    "now = datetime.datetime.now()\n",
    "\n",
    "# Format the date and time as desired\n",
    "formatted_date = now.strftime(\"%Y-%m-%d-%H-%M\")\n",
    "\n",
    "# Get some values using gcloud\n",
    "project_id = !(gcloud config get-value project)\n",
    "user = !(gcloud auth list --filter=status:ACTIVE --format=\"value(account)\")\n",
    "\n",
    "if len(project_id) != 1:\n",
    "  raise RuntimeError(f\"project_id is not set: {project_id}\")\n",
    "project_id = project_id[0]\n",
    "\n",
    "if len(user) != 1:\n",
    "  raise RuntimeError(f\"user is not set: {user}\")\n",
    "user = user[0]\n",
    "\n",
    "print(f\"project_id = {project_id}\")\n",
    "print(f\"user = {user}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sZ6m_wGrK0YG"
   },
   "source": [
    "### <font color='#4285f4'>Helper Methods</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JbOjdSP1kN9T"
   },
   "source": [
    "#### restAPIHelper\n",
    "Calls the Google Cloud REST API using the current users credentials."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "40wlwnY4kM11"
   },
   "outputs": [],
   "source": [
    "def restAPIHelper(url: str, http_verb: str, request_body: str) -> str:\n",
    "  \"\"\"Calls the Google Cloud REST API passing in the current users credentials\"\"\"\n",
    "\n",
    "  import requests\n",
    "  import google.auth\n",
    "  import json\n",
    "\n",
    "  # Get an access token based upon the current user\n",
    "  creds, project = google.auth.default()\n",
    "  auth_req = google.auth.transport.requests.Request()\n",
    "  creds.refresh(auth_req)\n",
    "  access_token=creds.token\n",
    "\n",
    "  headers = {\n",
    "    \"Content-Type\" : \"application/json\",\n",
    "    \"Authorization\" : \"Bearer \" + access_token\n",
    "  }\n",
    "\n",
    "  if http_verb == \"GET\":\n",
    "    response = requests.get(url, headers=headers)\n",
    "  elif http_verb == \"POST\":\n",
    "    response = requests.post(url, json=request_body, headers=headers)\n",
    "  elif http_verb == \"PUT\":\n",
    "    response = requests.put(url, json=request_body, headers=headers)\n",
    "  elif http_verb == \"PATCH\":\n",
    "    response = requests.patch(url, json=request_body, headers=headers)\n",
    "  elif http_verb == \"DELETE\":\n",
    "    response = requests.delete(url, headers=headers)\n",
    "  else:\n",
    "    raise RuntimeError(f\"Unknown HTTP verb: {http_verb}\")\n",
    "\n",
    "  if response.status_code == 200:\n",
    "    return json.loads(response.content)\n",
    "    #image_data = json.loads(response.content)[\"predictions\"][0][\"bytesBase64Encoded\"]\n",
    "  else:\n",
    "    error = f\"Error restAPIHelper -> ' Status: '{response.status_code}' Text: '{response.text}'\"\n",
    "    raise RuntimeError(error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### RetryCondition (for retrying LLM calls)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def RetryCondition(error):\n",
    "  error_string = str(error)\n",
    "  print(error_string)\n",
    "\n",
    "  retry_errors = [\n",
    "      \"RESOURCE_EXHAUSTED\",\n",
    "      \"No content in candidate\",\n",
    "      # Add more error messages here as needed\n",
    "  ]\n",
    "\n",
    "  for retry_error in retry_errors:\n",
    "    if retry_error in error_string:\n",
    "      print(\"Retrying...\")\n",
    "      return True\n",
    "\n",
    "  return False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Hh6lubX6VTHv"
   },
   "source": [
    "#### Gemini LLM (Pro 1.0 , Pro 1.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "KkX3St_QVVJw"
   },
   "outputs": [],
   "source": [
    "@retry(wait=wait_exponential(multiplier=1, min=1, max=60), stop=stop_after_attempt(10), retry=retry_if_exception(RetryCondition), before_sleep=before_sleep_log(logging.getLogger(), logging.INFO))\n",
    "def GeminiLLM(prompt, model = \"gemini-1.5-pro-001\", response_schema = None,\n",
    "                 temperature = 1, topP = 1, topK = 32):\n",
    "\n",
    "  # https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported_models\n",
    "  # model = \"gemini-1.5-pro-001\"\n",
    "  # model = \"gemini-pro\" # This does support topK (but the function is more generic)\n",
    "  # model = \"gemini-1.0-pro\" # This does not support response_schema\n",
    "\n",
    "  llm_response = None\n",
    "  if temperature < 0:\n",
    "    temperature = 0\n",
    "\n",
    "  creds, project = google.auth.default()\n",
    "  auth_req = google.auth.transport.requests.Request() # required to acess access token\n",
    "  creds.refresh(auth_req)\n",
    "  access_token=creds.token\n",
    "\n",
    "  headers = {\n",
    "      \"Content-Type\" : \"application/json\",\n",
    "      \"Authorization\" : \"Bearer \" + access_token\n",
    "  }\n",
    "\n",
    "  # https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference\n",
    "  url = f\"https://{location}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{location}/publishers/google/models/{model}:generateContent\"\n",
    "\n",
    "  generation_config = {\n",
    "    \"temperature\": temperature,\n",
    "    \"topP\": topP,\n",
    "    \"maxOutputTokens\": 8192,\n",
    "    \"candidateCount\": 1,\n",
    "    \"responseMimeType\": \"application/json\",\n",
    "  }\n",
    "\n",
    "  # Add inthe response schema for when it is provided\n",
    "  if response_schema is not None:\n",
    "    generation_config[\"responseSchema\"] = response_schema\n",
    "\n",
    "  if model == \"gemini-pro\" or model == \"gemini-1.0-pro\" or model == \"gemini-1.0-pro-vision-001\":\n",
    "    generation_config[\"topK\"] = topK\n",
    "\n",
    "  payload = {\n",
    "    \"contents\": {\n",
    "      \"role\": \"user\",\n",
    "      \"parts\": {\n",
    "          \"text\": prompt\n",
    "      },\n",
    "    },\n",
    "    \"generation_config\": {\n",
    "      **generation_config\n",
    "    },\n",
    "    \"safety_settings\": {\n",
    "      \"category\": \"HARM_CATEGORY_SEXUALLY_EXPLICIT\",\n",
    "      \"threshold\": \"BLOCK_LOW_AND_ABOVE\"\n",
    "    }\n",
    "  }\n",
    "\n",
    "  response = requests.post(url, json=payload, headers=headers)\n",
    "\n",
    "  if response.status_code == 200:\n",
    "    try:\n",
    "      json_response = json.loads(response.content)\n",
    "    except Exception as error:\n",
    "      raise RuntimeError(f\"An error occurred parsing the JSON: {error}\")\n",
    "\n",
    "    if \"candidates\" in json_response:\n",
    "      candidates = json_response[\"candidates\"]\n",
    "      if len(candidates) > 0:\n",
    "        candidate = candidates[0]\n",
    "        if \"content\" in candidate:\n",
    "          content = candidate[\"content\"]\n",
    "          if \"parts\" in content:\n",
    "            parts = content[\"parts\"]\n",
    "            if len(parts):\n",
    "              part = parts[0]\n",
    "              if \"text\" in part:\n",
    "                text = part[\"text\"]\n",
    "                llm_response = text\n",
    "              else:\n",
    "                raise RuntimeError(\"No text in part: {response.content}\")\n",
    "            else:\n",
    "              raise RuntimeError(\"No parts in content: {response.content}\")\n",
    "          else:\n",
    "            raise RuntimeError(\"No parts in content: {response.content}\")\n",
    "        else:\n",
    "          raise RuntimeError(\"No content in candidate: {response.content}\")\n",
    "      else:\n",
    "        raise RuntimeError(\"No candidates: {response.content}\")\n",
    "    else:\n",
    "      raise RuntimeError(\"No candidates: {response.content}\")\n",
    "\n",
    "    # Remove some typically response characters (if asking for a JSON reply)\n",
    "    llm_response = llm_response.replace(\"```json\",\"\")\n",
    "    llm_response = llm_response.replace(\"```\",\"\")\n",
    "    llm_response = llm_response.replace(\"\\n\",\"\")\n",
    "\n",
    "    return llm_response\n",
    "\n",
    "  else:\n",
    "    raise RuntimeError(f\"Error with prompt:'{prompt}'  Status:'{response.status_code}' Text:'{response.text}'\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5dHoF0tzHlZ-"
   },
   "source": [
    "#### Gemini LLM - Multimodal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "saJMOmA0HcVO"
   },
   "outputs": [],
   "source": [
    "@retry(wait=wait_exponential(multiplier=1, min=1, max=60), stop=stop_after_attempt(10), retry=retry_if_exception(RetryCondition), before_sleep=before_sleep_log(logging.getLogger(), logging.INFO))\n",
    "def GeminiLLM_Multimodal(multimodal_prompt_list, model = \"gemini-1.5-pro-001\", response_schema = None,\n",
    "                 temperature = 1, topP = 1, topK = 32):\n",
    "\n",
    "  # https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#supported_models\n",
    "  # model = \"gemini-1.5-pro-001\"\n",
    "  # model = \"gemini-pro\" # This does support topK (but the function is more generic)\n",
    "  # model = \"gemini-1.0-pro\" # This does not support response_schema\n",
    "\n",
    "  llm_response = None\n",
    "  if temperature < 0:\n",
    "    temperature = 0\n",
    "\n",
    "  creds, project = google.auth.default()\n",
    "  auth_req = google.auth.transport.requests.Request() # required to acess access token\n",
    "  creds.refresh(auth_req)\n",
    "  access_token=creds.token\n",
    "\n",
    "  headers = {\n",
    "      \"Content-Type\" : \"application/json\",\n",
    "      \"Authorization\" : \"Bearer \" + access_token\n",
    "  }\n",
    "\n",
    "  # https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference\n",
    "  url = f\"https://{location}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{location}/publishers/google/models/{model}:generateContent\"\n",
    "\n",
    "  generation_config = {\n",
    "    \"temperature\": temperature,\n",
    "    \"topP\": topP,\n",
    "    \"maxOutputTokens\": 8192,\n",
    "    \"candidateCount\": 1,\n",
    "    \"responseMimeType\": \"application/json\",\n",
    "  }\n",
    "\n",
    "  # Add inthe response schema for when it is provided\n",
    "  if response_schema is not None:\n",
    "    generation_config[\"responseSchema\"] = response_schema\n",
    "\n",
    "  if model == \"gemini-pro\" or model == \"gemini-1.0-pro\" or model == \"gemini-1.0-pro-vision-001\":\n",
    "    generation_config[\"topK\"] = topK\n",
    "\n",
    "  payload = {\n",
    "    \"contents\": {\n",
    "      \"role\": \"user\",\n",
    "      \"parts\": multimodal_prompt_list\n",
    "    },\n",
    "    \"generation_config\": {\n",
    "      **generation_config\n",
    "    },\n",
    "    \"safety_settings\": {\n",
    "      \"category\": \"HARM_CATEGORY_SEXUALLY_EXPLICIT\",\n",
    "      \"threshold\": \"BLOCK_LOW_AND_ABOVE\"\n",
    "    }\n",
    "  }\n",
    "\n",
    "  response = requests.post(url, json=payload, headers=headers)\n",
    "\n",
    "  if response.status_code == 200:\n",
    "    try:\n",
    "      json_response = json.loads(response.content)\n",
    "    except Exception as error:\n",
    "      raise RuntimeError(f\"An error occurred parsing the JSON: {error}\")\n",
    "\n",
    "    if \"candidates\" in json_response:\n",
    "      candidates = json_response[\"candidates\"]\n",
    "      if len(candidates) > 0:\n",
    "        candidate = candidates[0]\n",
    "        if \"content\" in candidate:\n",
    "          content = candidate[\"content\"]\n",
    "          if \"parts\" in content:\n",
    "            parts = content[\"parts\"]\n",
    "            if len(parts):\n",
    "              part = parts[0]\n",
    "              if \"text\" in part:\n",
    "                text = part[\"text\"]\n",
    "                llm_response = text\n",
    "              else:\n",
    "                raise RuntimeError(\"No text in part: {response.content}\")\n",
    "            else:\n",
    "              raise RuntimeError(\"No parts in content: {response.content}\")\n",
    "          else:\n",
    "            raise RuntimeError(\"No parts in content: {response.content}\")\n",
    "        else:\n",
    "          raise RuntimeError(\"No content in candidate: {response.content}\")\n",
    "      else:\n",
    "        raise RuntimeError(\"No candidates: {response.content}\")\n",
    "    else:\n",
    "      raise RuntimeError(\"No candidates: {response.content}\")\n",
    "\n",
    "    # Remove some typically response characters (if asking for a JSON reply)\n",
    "    llm_response = llm_response.replace(\"```json\",\"\")\n",
    "    llm_response = llm_response.replace(\"```\",\"\")\n",
    "    llm_response = llm_response.replace(\"\\n\",\"\")\n",
    "\n",
    "    return llm_response\n",
    "\n",
    "  else:\n",
    "    raise RuntimeError(f\"Error with prompt:'{multimodal_prompt_list[0]}'  Status:'{response.status_code}' Text:'{response.text}'\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IT_AtoA3gV4q"
   },
   "source": [
    "#### Text to Speech Generation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "dIwlOZ7BgZu7"
   },
   "outputs": [],
   "source": [
    "def TextToSpeechLanguageList(language_code):\n",
    "  creds, project = google.auth.default()\n",
    "  auth_req = google.auth.transport.requests.Request()\n",
    "  creds.refresh(auth_req)\n",
    "  access_token=creds.token\n",
    "\n",
    "  headers = {\n",
    "      \"Content-Type\" : \"application/json\",\n",
    "      \"Authorization\" : \"Bearer \" + access_token,\n",
    "      \"x-goog-user-project\" : project\n",
    "  }\n",
    "\n",
    "  # https://cloud.google.com/text-to-speech/docs/reference/rest/v1/voices/list\n",
    "  url = f\"https://texttospeech.googleapis.com/v1/voices?languageCode={language_code}\"\n",
    "\n",
    "  response = requests.get(url, headers=headers)\n",
    "\n",
    "  if response.status_code == 200:\n",
    "    return response.text\n",
    "  else:\n",
    "    error = f\"Error with language_code:'{language_code}'  Status:'{response.status_code}' Text:'{response.text}'\"\n",
    "    raise RuntimeError(error)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Xh7R560ojLrS"
   },
   "outputs": [],
   "source": [
    "def TextToSpeech(local_filename, text, language_code, language_code_name, ssml_gender, speaking_rate = 1):\n",
    "  creds, project = google.auth.default()\n",
    "  auth_req = google.auth.transport.requests.Request()\n",
    "  creds.refresh(auth_req)\n",
    "  access_token=creds.token\n",
    "\n",
    "  headers = {\n",
    "      \"Content-Type\" : \"application/json\",\n",
    "      \"Authorization\" : \"Bearer \" + access_token,\n",
    "      \"x-goog-user-project\" : project\n",
    "  }\n",
    "\n",
    "  # https://cloud.google.com/text-to-speech/docs/reference/rest/v1/text/synthesize\n",
    "  url = f\"https://texttospeech.googleapis.com/v1/text:synthesize\"\n",
    "\n",
    "  payload = {\n",
    "   \"input\": {\n",
    "      \"text\": text\n",
    "   },\n",
    "   \"voice\": {\n",
    "      \"languageCode\": language_code,\n",
    "      \"name\": language_code_name,\n",
    "      \"ssmlGender\": ssml_gender # FEMALE | MALE\n",
    "   },\n",
    "   \"audioConfig\": {\n",
    "      \"audioEncoding\": \"MP3\",\n",
    "      \"speakingRate\": speaking_rate,\n",
    "   }\n",
    "  }\n",
    "\n",
    "  response = requests.post(url, json=payload, headers=headers)\n",
    "\n",
    "  if response.status_code == 200:\n",
    "    audio_data = json.loads(response.content)[\"audioContent\"]\n",
    "    audio_data = base64.b64decode(audio_data)\n",
    "    with open(local_filename, \"wb\") as f:\n",
    "      f.write(audio_data)\n",
    "    print(f\"Audio generated OK.\")\n",
    "    return local_filename\n",
    "  else:\n",
    "    error = f\"Error with text:'{text}'  Status:'{response.status_code}' Text:'{response.text}'\"\n",
    "    raise RuntimeError(error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CWYQfM8BK5Sn"
   },
   "source": [
    "#### Overlay text to speech to video"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "bq7vG0KBK9XX"
   },
   "outputs": [],
   "source": [
    "from moviepy.editor import VideoFileClip, AudioFileClip\n",
    "\n",
    "def MergeVideoAndAudio(video_filename, audio_filename, output_filename):\n",
    "  # Load the video and audio files\n",
    "  video = VideoFileClip(video_filename)\n",
    "  audio = AudioFileClip(audio_filename)\n",
    "\n",
    "  # Combine the video and audio\n",
    "  final_clip = video.set_audio(audio)\n",
    "\n",
    "  # Save the combined video\n",
    "  final_clip.write_videofile(output_filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AqKylYj_LFqB"
   },
   "source": [
    "#### GetMenuItems"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "myH5rUJdLEl7"
   },
   "outputs": [],
   "source": [
    "def GetMenuItems():\n",
    "  sql = f\"\"\"SELECT menu_id, menu_name, menu_description\n",
    "  FROM `{dataset_name}.menu`\n",
    "  ORDER BY menu_id\"\"\"\n",
    "\n",
    "  result_df = RunQuery(sql)\n",
    "  result_list = []\n",
    "\n",
    "  for index, row in result_df.iterrows():\n",
    "    result_list.append({\n",
    "        \"menu_id\": row['menu_id'],\n",
    "        \"menu_name\": row['menu_name'],\n",
    "        \"menu_description\": row['menu_description']\n",
    "    })\n",
    "\n",
    "  return result_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0ZcHJOCOxVlH"
   },
   "source": [
    "#### Combine Videos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "O0NSV7A9xZQf"
   },
   "outputs": [],
   "source": [
    "from moviepy.editor import VideoFileClip, concatenate_videoclips\n",
    "\n",
    "def merge_videos_sorted(folder_path, output_video_name):\n",
    "  \"\"\"\n",
    "  Merges all MP4 video files in the specified folder into a single video,\n",
    "  sorted by file name.\n",
    "\n",
    "  Args:\n",
    "      folder_path: The path to the folder containing the videos.\n",
    "  \"\"\"\n",
    "\n",
    "  video_files = [f for f in os.listdir(folder_path) if f.endswith('.mp4')]\n",
    "  video_files.sort()  # Sort the files by name\n",
    "\n",
    "  clips = [VideoFileClip(os.path.join(folder_path, video)) for video in video_files]\n",
    "\n",
    "  final_clip = concatenate_videoclips(clips)\n",
    "  final_clip.write_videofile(os.path.join(folder_path, output_video_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bNKaaW5ohCpd"
   },
   "source": [
    "#### Helper Functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "K_Liz2MS2JKy"
   },
   "outputs": [],
   "source": [
    "def RunQuery(sql):\n",
    "  import time\n",
    "  from google.cloud import bigquery\n",
    "  client = bigquery.Client()\n",
    "\n",
    "  if (sql.startswith(\"SELECT\") or sql.startswith(\"WITH\")):\n",
    "      df_result = client.query(sql).to_dataframe()\n",
    "      return df_result\n",
    "  else:\n",
    "    job_config = bigquery.QueryJobConfig(priority=bigquery.QueryPriority.INTERACTIVE)\n",
    "    query_job = client.query(sql, job_config=job_config)\n",
    "\n",
    "    # Check on the progress by getting the job's updated state.\n",
    "    query_job = client.get_job(\n",
    "        query_job.job_id, location=query_job.location\n",
    "    )\n",
    "    print(\"Job {} is currently in state {} with error result of {}\".format(query_job.job_id, query_job.state, query_job.error_result))\n",
    "\n",
    "    while query_job.state != \"DONE\":\n",
    "      time.sleep(2)\n",
    "      query_job = client.get_job(\n",
    "          query_job.job_id, location=query_job.location\n",
    "          )\n",
    "      print(\"Job {} is currently in state {} with error result of {}\".format(query_job.job_id, query_job.state, query_job.error_result))\n",
    "\n",
    "    if query_job.error_result == None:\n",
    "      return True\n",
    "    else:\n",
    "      raise Exception(query_job.error_result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1VjvnfpBzN4s"
   },
   "outputs": [],
   "source": [
    "# This was generated by GenAI\n",
    "\n",
    "def copy_file_to_gcs(local_file_path, bucket_name, destination_blob_name):\n",
    "  \"\"\"Copies a file from a local drive to a GCS bucket.\n",
    "\n",
    "  Args:\n",
    "      local_file_path: The full path to the local file.\n",
    "      bucket_name: The name of the GCS bucket to upload to.\n",
    "      destination_blob_name: The desired name of the uploaded file in the bucket.\n",
    "\n",
    "  Returns:\n",
    "      None\n",
    "  \"\"\"\n",
    "\n",
    "  import os\n",
    "  from google.cloud import storage\n",
    "\n",
    "  # Ensure the file exists locally\n",
    "  if not os.path.exists(local_file_path):\n",
    "      raise FileNotFoundError(f\"Local file '{local_file_path}' not found.\")\n",
    "\n",
    "  # Create a storage client\n",
    "  storage_client = storage.Client()\n",
    "\n",
    "  # Get a reference to the bucket\n",
    "  bucket = storage_client.bucket(bucket_name)\n",
    "\n",
    "  # Create a blob object with the desired destination path\n",
    "  blob = bucket.blob(destination_blob_name)\n",
    "\n",
    "  # Upload the file from the local filesystem\n",
    "  content_type = \"\"\n",
    "  if local_file_path.endswith(\".html\"):\n",
    "    content_type = \"text/html; charset=utf-8\"\n",
    "\n",
    "  if local_file_path.endswith(\".json\"):\n",
    "    content_type = \"application/json; charset=utf-8\"\n",
    "\n",
    "  if content_type == \"\":\n",
    "    blob.upload_from_filename(local_file_path)\n",
    "  else:\n",
    "    blob.upload_from_filename(local_file_path, content_type = content_type)\n",
    "\n",
    "  print(f\"File '{local_file_path}' uploaded to GCS bucket '{bucket_name}' as '{destination_blob_name}.  Content-Type: {content_type}'.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "N7FijsCezJQ6"
   },
   "outputs": [],
   "source": [
    "def download_from_gcs(destination_file_name, gcs_storage_bucket, object_name):\n",
    "  # prompt: Write python code to download a blob from a gcs bucket.  do not use the requests method\n",
    "\n",
    "  from google.cloud import storage\n",
    "  storage_client = storage.Client()\n",
    "  bucket = storage_client.bucket(gcs_storage_bucket)\n",
    "\n",
    "  # Construct a client side representation of a blob.\n",
    "  # Note `Bucket.blob` differs from `Bucket.get_blob` as it doesn't retrieve\n",
    "  # any content from Google Cloud Storage. As we don't need additional data,\n",
    "  # using `Bucket.blob` is preferred here.\n",
    "  blob = bucket.blob(object_name)\n",
    "  blob.download_to_filename(destination_file_name)\n",
    "\n",
    "  print(\n",
    "      \"Downloaded storage object {} from bucket {} to local file {}.\".format(\n",
    "          object_name, gcs_storage_bucket, destination_file_name\n",
    "      )\n",
    "  )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "lmqQixNHZfRj"
   },
   "outputs": [],
   "source": [
    "# prompt: python to delete a file even if it does not exist\n",
    "\n",
    "def delete_file(filename):\n",
    "  try:\n",
    "    os.remove(filename)\n",
    "    print(f\"File '{filename}' deleted successfully.\")\n",
    "  except FileNotFoundError:\n",
    "    print(f\"File '{filename}' not found.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Tq3HEggwhEL1"
   },
   "outputs": [],
   "source": [
    "def PrettyPrintJson(json_string):\n",
    "  json_object = json.loads(json_string)\n",
    "  json_formatted_str = json.dumps(json_object, indent=2)\n",
    "  # print(json_formatted_str)\n",
    "  return json_formatted_str"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rt3ErRFLpsYe"
   },
   "source": [
    "### <font color='#4285f4'>Teach the LLM how to write text-to-video prompts</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "RivD3xaHL1Lk"
   },
   "outputs": [],
   "source": [
    "# We need to tell the LLM how to write text-to-video prompts\n",
    "\n",
    "text_to_video_prompt_guide = \"\"\"\n",
    "Text-to-Video Prompt Writing Help:\n",
    "<text-to-video-prompt-guide>\n",
    "Here are some our best practices for text-to-video prompts:\n",
    "\n",
    "Detailed prompts = better videos:\n",
    "  - More details you add, the more control you’ll have over the video.\n",
    "  - A prompt should look like this: \"Camera dollies to show a close up of a desperate man in a green trench coat is making a call on a rotary style wall-phone, green neon light, movie scene.\"\n",
    "    - Here is a break down of elements need to create a text-to-video prompt using the above prompt as an example:\n",
    "      - \"Camera dollies to show\" = \"Camera Motion\"\n",
    "      - \"A close up of\" = \"Composition\"\n",
    "      - \"A desperate man in a green trench coat\" = \"Subject\"\n",
    "      - \"Is making a call\" = \"Action\"\n",
    "      - \"On a roary style wall-phone\" = \"Scene\"\n",
    "      - \"Green Neon light\" = \"Ambiance\"\n",
    "      - \"Movie Scene\" = \"Style\"\n",
    "\n",
    "Use the right keywords for better control:\n",
    "  - Here is a list of some keywords that work well with text-to-video, use these in your prompts to get the desired camera motion or style.\n",
    "  - Subject: Who or what is the main focus of the shot.  Example: \"happy woman in her 30s\".\n",
    "  - Scene: Where is the location of the shot. Example \"on a busy street, in space\".\n",
    "  - Action: What is the subject doing Examples: \"walking\", \"running\", \"turning head\".\n",
    "  - Camera Motion: What the camera is doing. Example: \"POV shot\", \"Aerial View\", \"Tracking Drone view\", \"Tracking Shot\".\n",
    "\n",
    "Example text-to-video prompt using the above keywords:\n",
    "  - Example text-to-video prompt: \"Tracking drone view of a man driving a red convertible car in Palm Springs, 1970s, warm sunlight, long shadows\"\n",
    "  - Example text-to-video prompt: \"A POV shot from a vintage car driving in the rain, Canada at night, cinematic\"\n",
    "\n",
    "Styles:\n",
    "   - Overall aesthetic. Consider using specific film style keywords.  Examples: \"horror film\", \"film noir, \"animated styles\", \"3D cartoon style render\".\n",
    "  - Example text-to-video prompt: \"Over the shoulder of a young woman in a car, 1970s, film grain, horror film, cinematic he Film noir style, man and woman walk on the street, mystery, cinematic, black and white\"\n",
    "  - Example text-to-video prompt: \"A cute creatures with snow leopard-like fur is walking in winter forest, 3D cartoon style render. An architectural rendering of a white concrete apartment building with flowing organic shapes, seamlessly blending with lush greenery and futuristic elements.\"\n",
    "\n",
    "Composition:\n",
    "  - How the shot is framed. This is often relative to the subject e.g. wide shot, close-up, low angle\n",
    "  - Example text-to-video prompt: \"Extreme close-up of a an eye with city reflected in it. A wide shot of surfer walking on a beach with a surfboard, beautiful sunset, cinematic\"\n",
    "\n",
    "Ambiance & Emotions:\n",
    "  - How the color and light contribute to the scene (blue tones, night)\n",
    "  - Example text-to-video prompt: \"A close-up of a girl holding adorable golden retriever puppy in the park, sunlight Cinematic close-up shot of a sad woman riding a bus in the rain, cool blue tones, sad mood\"\n",
    "\n",
    "Cinematic effects:\n",
    "  - e.g. double exposure, projected, glitch camera effect.\n",
    "  - Example text-to-video prompt: \"A double exposure of silhouetted profile of a woman walking and lake, walking in a forest Close-up shot of a model with blue light with geometric shapes projected on her face\"\n",
    "  - Example text-to-video prompt: \"Silhouette of a man walking in collage of cityscapes Glitch camera effect, close up of woman’s face speaking, neon colors\"\n",
    "</text-to-video-prompt-guide>\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FbQT-vIUi8ab"
   },
   "source": [
    "### <font color='#4285f4'>Generate the Text-to-Video Prompts using Gemini (let Gemini write the prompts for us)</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "lb90PsVL8CMs"
   },
   "outputs": [],
   "source": [
    "text_to_video_prompts = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wF4xTwDM9yvp"
   },
   "source": [
    "##### Text-to-Video: Random Menu item prompts\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "QoYQYg7Ki_MQ"
   },
   "outputs": [],
   "source": [
    "# Write me the json in  OpenAPI 3.0 schema object for the below object.\n",
    "# Make all fields required.\n",
    "#  {\n",
    "#    \"text-to-video-prompt\" : \"text\"\n",
    "#  }\n",
    "response_schema = {\n",
    "  \"type\": \"object\",\n",
    "  \"required\": [\n",
    "    \"text-to-video-prompt\"\n",
    "  ],\n",
    "  \"properties\": {\n",
    "    \"text-to-video-prompt\": {\n",
    "      \"type\": \"string\"\n",
    "    }\n",
    "  }\n",
    "}\n",
    "\n",
    "menu_item_list = GetMenuItems()\n",
    "menu_id = random.randint(1, len(menu_item_list))\n",
    "menu_name = menu_item_list[menu_id-1][\"menu_name\"]\n",
    "menu_description = menu_item_list[menu_id-1][\"menu_description\"]\n",
    "\n",
    "gemini_ad_prompt = f\"\"\"You are a marketing expert and are creating a marketing video for a company that sells premium handcrafted chocolates, handmade desserts and delicious coffee drinks.\n",
    "You need to create a marketing video for the company.\n",
    "The company is based in Paris, France.\n",
    "\n",
    "The video should be about the following menu item:\n",
    "{menu_name}: {menu_description}\n",
    "\n",
    "The video should show the menu item being made.\n",
    "Focus the video on the finishing touches like sprinkling chocolate, adding a layer of cream, adding a layer of sugar, adding a layer of coffee, adding a layer of milk.\n",
    "\n",
    "Output Fields:\n",
    "- \"text-to-video-prompt\":\n",
    " - Read the  \"Text-to-Video Prompt Writing Help\" to learn more about how to create good text-to-video prompts.\n",
    " - Make sure you include all the relevant best practices when creating the text-to-video prompt:\n",
    " - A detailed prompt for generating the video using text-to-video technology.\n",
    " - Focus on creating the menu item with an artistic flair.\n",
    " - Do not include \"text overlays\" in the text-to-video prompt.\n",
    " - The prompt should also reference that we are in a chocolate, dessert, coffee shop in Paris so it knows the context of the video.\n",
    " - Do not include children in the text-to-video prompt.\n",
    "\n",
    "{text_to_video_prompt_guide}\n",
    "\"\"\"\n",
    "\n",
    "#print(gemini_ad_prompt)\n",
    "llm_result = GeminiLLM(gemini_ad_prompt, response_schema=response_schema)\n",
    "gemini_ad_results_dict = json.loads(llm_result)\n",
    "orginal_ad_results_dict = gemini_ad_results_dict # in case we swap it for a pre-canned one\n",
    "print(f\"Menu Name: {menu_name} - {menu_description}\")\n",
    "print()\n",
    "print(PrettyPrintJson(json.dumps(gemini_ad_results_dict)))\n",
    "text_to_video_prompts.append(gemini_ad_results_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uBmWWkEb9cTF"
   },
   "source": [
    "##### Text-to-Video: Swan cake with Friends"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wA1MxCFg5oU3"
   },
   "outputs": [],
   "source": [
    "# Write me the json in  OpenAPI 3.0 schema object for the below object.\n",
    "# Make all fields required.\n",
    "#  {\n",
    "#    \"text-to-video-prompt\" : \"text\"\n",
    "#  }\n",
    "response_schema = {\n",
    "  \"type\": \"object\",\n",
    "  \"required\": [\n",
    "    \"text-to-video-prompt\"\n",
    "  ],\n",
    "  \"properties\": {\n",
    "    \"text-to-video-prompt\": {\n",
    "      \"type\": \"string\"\n",
    "    }\n",
    "  }\n",
    "}\n",
    "\n",
    "menu_item_list = GetMenuItems()\n",
    "menu_id = random.randint(1, len(menu_item_list))\n",
    "menu_id = 8\n",
    "menu_name = menu_item_list[menu_id-1][\"menu_name\"]\n",
    "menu_description = menu_item_list[menu_id-1][\"menu_description\"]\n",
    "\n",
    "gemini_ad_prompt = f\"\"\"You are a marketing expert and are creating a marketing video for a company that sells premium handcrafted chocolates, handmade desserts and delicious coffee drinks.\n",
    "You need to create a marketing video for the company.\n",
    "The company is based in Paris, France.\n",
    "\n",
    "Show people around a table with a Chocolate Swan Cake in the middle.\n",
    "The people should be having fun like it is a party.\n",
    "The party needs to be upscale.\n",
    "\n",
    "The video should be about the following menu item:\n",
    "{menu_name}: {menu_description}\n",
    "\n",
    "Output Fields:\n",
    "- \"text-to-video-prompt\":\n",
    " - Read the  \"Text-to-Video Prompt Writing Help\" to learn more about how to create good text-to-video prompts.\n",
    " - Make sure you include all the relevant best practices when creating the text-to-video prompt:\n",
    " - A detailed prompt for generating the video using text-to-video technology.\n",
    " - Focus on creating the menu item with an artistic flair.\n",
    " - Do not include \"text overlays\" in the text-to-video prompt.\n",
    " - The prompt should also reference that we are in a chocolate, dessert, coffee shop in Paris so it knows the context of the video.\n",
    " - Do not include children in the text-to-video prompt.\n",
    "\n",
    "{text_to_video_prompt_guide}\n",
    "\"\"\"\n",
    "\n",
    "#print(gemini_ad_prompt)\n",
    "llm_result = GeminiLLM(gemini_ad_prompt, response_schema=response_schema)\n",
    "gemini_ad_results_dict = json.loads(llm_result)\n",
    "orginal_ad_results_dict = gemini_ad_results_dict # in case we swap it for a pre-canned one\n",
    "print(f\"Menu Name: {menu_name} - {menu_description}\")\n",
    "print()\n",
    "print(PrettyPrintJson(json.dumps(gemini_ad_results_dict)))\n",
    "text_to_video_prompts.append(gemini_ad_results_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JZ1K2ofwKDnL"
   },
   "source": [
    "##### Text-to-Video: Friends at Picnic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "OCUZa8mw9BAX"
   },
   "outputs": [],
   "source": [
    "# Write me the json in  OpenAPI 3.0 schema object for the below object.\n",
    "# Make all fields required.\n",
    "#  {\n",
    "#    \"text-to-video-prompt\" : \"text\"\n",
    "#  }\n",
    "response_schema = {\n",
    "  \"type\": \"object\",\n",
    "  \"required\": [\n",
    "    \"text-to-video-prompt\"\n",
    "  ],\n",
    "  \"properties\": {\n",
    "    \"text-to-video-prompt\": {\n",
    "      \"type\": \"string\"\n",
    "    }\n",
    "  }\n",
    "}\n",
    "\n",
    "menu_item_list = GetMenuItems()\n",
    "menu_id = random.randint(1, len(menu_item_list))\n",
    "menu_name = menu_item_list[menu_id-1][\"menu_name\"]\n",
    "menu_description = menu_item_list[menu_id-1][\"menu_description\"]\n",
    "\n",
    "gemini_ad_prompt = f\"\"\"You are a marketing expert and are creating a marketing video for a company that sells premium handcrafted chocolates, handmade desserts and delicious coffee drinks.\n",
    "You need to create a marketing video for the company.\n",
    "The company is based in Paris, France.\n",
    "\n",
    "A group of friends eating \"{menu_name}\" at a picnic in a French park.\n",
    "\n",
    "The video should be about the following menu item:\n",
    "{menu_name}: {menu_description}\n",
    "\n",
    "Output Fields:\n",
    "- \"text-to-video-prompt\":\n",
    " - Read the  \"Text-to-Video Prompt Writing Help\" to learn more about how to create good text-to-video prompts.\n",
    " - Make sure you include all the relevant best practices when creating the text-to-video prompt:\n",
    " - A detailed prompt for generating the video using text-to-video technology.\n",
    " - Focus on creating the menu item with an artistic flair.\n",
    " - Do not include \"text overlays\" in the text-to-video prompt.\n",
    " - The prompt should also reference that we are in a chocolate, dessert, coffee shop in Paris so it knows the context of the video.\n",
    " - Do not include children in the text-to-video prompt.\n",
    "\n",
    "{text_to_video_prompt_guide}\n",
    "\"\"\"\n",
    "\n",
    "#print(gemini_ad_prompt)\n",
    "llm_result = GeminiLLM(gemini_ad_prompt, response_schema=response_schema)\n",
    "gemini_ad_results_dict = json.loads(llm_result)\n",
    "orginal_ad_results_dict = gemini_ad_results_dict # in case we swap it for a pre-canned one\n",
    "print(f\"Menu Name: {menu_name} - {menu_description}\")\n",
    "print()\n",
    "print(PrettyPrintJson(json.dumps(gemini_ad_results_dict)))\n",
    "text_to_video_prompts.append(gemini_ad_results_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hgFkSBarIx7H"
   },
   "source": [
    "##### Text-to-Video: Chocolatier in Front of Window"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "oeS8YVWDIzqj"
   },
   "outputs": [],
   "source": [
    "# Write me the json in  OpenAPI 3.0 schema object for the below object.\n",
    "# Make all fields required.\n",
    "#  {\n",
    "#    \"text-to-video-prompt\" : \"text\"\n",
    "#  }\n",
    "response_schema = {\n",
    "  \"type\": \"object\",\n",
    "  \"required\": [\n",
    "    \"text-to-video-prompt\"\n",
    "  ],\n",
    "  \"properties\": {\n",
    "    \"text-to-video-prompt\": {\n",
    "      \"type\": \"string\"\n",
    "    }\n",
    "  }\n",
    "}\n",
    "\n",
    "menu_item_list = GetMenuItems()\n",
    "menu_id = random.randint(1, len(menu_item_list))\n",
    "menu_name = menu_item_list[menu_id-1][\"menu_name\"]\n",
    "menu_description = menu_item_list[menu_id-1][\"menu_description\"]\n",
    "\n",
    "gemini_ad_prompt = f\"\"\"You are a marketing expert and are creating a marketing video for a company that sells premium handcrafted chocolates, handmade desserts and delicious coffee drinks.\n",
    "You need to create a marketing video for the company.\n",
    "The company is based in Paris, France.\n",
    "\n",
    "A chocolatier chef preparing a \"{menu_name}\" and looking out a large picturesque window.\n",
    "The video should pan from the chef to the window.\n",
    "\n",
    "The video should be about the following menu item:\n",
    "{menu_name}: {menu_description}\n",
    "\n",
    "Output Fields:\n",
    "- \"text-to-video-prompt\":\n",
    " - Read the  \"Text-to-Video Prompt Writing Help\" to learn more about how to create good text-to-video prompts.\n",
    " - Make sure you include all the relevant best practices when creating the text-to-video prompt:\n",
    " - A detailed prompt for generating the video using text-to-video technology.\n",
    " - Focus on creating the menu item with an artistic flair.\n",
    " - Do not include \"text overlays\" in the text-to-video prompt.\n",
    " - The prompt should also reference that we are in a chocolate, dessert, coffee shop in Paris so it knows the context of the video.\n",
    " - Do not include children in the text-to-video prompt.\n",
    "\n",
    "{text_to_video_prompt_guide}\n",
    "\"\"\"\n",
    "\n",
    "#print(gemini_ad_prompt)\n",
    "llm_result = GeminiLLM(gemini_ad_prompt, response_schema=response_schema)\n",
    "gemini_ad_results_dict = json.loads(llm_result)\n",
    "orginal_ad_results_dict = gemini_ad_results_dict # in case we swap it for a pre-canned one\n",
    "print(f\"Menu Name: {menu_name} - {menu_description}\")\n",
    "print()\n",
    "print(PrettyPrintJson(json.dumps(gemini_ad_results_dict)))\n",
    "text_to_video_prompts.append(gemini_ad_results_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GNhgCtUdJ6HL"
   },
   "source": [
    "##### Text-to-Video: Friends at an Outdoor Chocolate Buffet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_8JOcAl5J9Xk"
   },
   "outputs": [],
   "source": [
    "# Write me the json in  OpenAPI 3.0 schema object for the below object.\n",
    "# Make all fields required.\n",
    "#  {\n",
    "#    \"text-to-video-prompt\" : \"text\"\n",
    "#  }\n",
    "response_schema = {\n",
    "  \"type\": \"object\",\n",
    "  \"required\": [\n",
    "    \"text-to-video-prompt\"\n",
    "  ],\n",
    "  \"properties\": {\n",
    "    \"text-to-video-prompt\": {\n",
    "      \"type\": \"string\"\n",
    "    }\n",
    "  }\n",
    "}\n",
    "\n",
    "menu_item_list = GetMenuItems()\n",
    "menu_id = random.randint(1, len(menu_item_list))\n",
    "menu_name = menu_item_list[menu_id-1][\"menu_name\"]\n",
    "menu_description = menu_item_list[menu_id-1][\"menu_description\"]\n",
    "\n",
    "gemini_ad_prompt = f\"\"\"You are a marketing expert and are creating a marketing video for a company that sells premium handcrafted chocolates, handmade desserts and delicious coffee drinks.\n",
    "You need to create a marketing video for the company.\n",
    "The company is based in Paris, France.\n",
    "\n",
    "A group of friends at an outdoor buffet that has custom chocolates and desserts on the streets of Paris.\n",
    "They are enjoying the sunshine and eating the below menu items.\n",
    "\n",
    "The video should be about the following menu items:\n",
    "{menu_name}: {menu_description}\n",
    "{menu_item_list[menu_id][\"menu_name\"]}: {menu_item_list[menu_id][\"menu_description\"]}\n",
    "{menu_item_list[menu_id+1][\"menu_name\"]}: {menu_item_list[menu_id+1][\"menu_description\"]}\n",
    "{menu_item_list[menu_id+2][\"menu_name\"]}: {menu_item_list[menu_id+2][\"menu_description\"]}\n",
    "\n",
    "\n",
    "Output Fields:\n",
    "- \"text-to-video-prompt\":\n",
    " - Read the  \"Text-to-Video Prompt Writing Help\" to learn more about how to create good text-to-video prompts.\n",
    " - Make sure you include all the relevant best practices when creating the text-to-video prompt:\n",
    " - A detailed prompt for generating the video using text-to-video technology.\n",
    " - Focus on creating the menu item with an artistic flair.\n",
    " - Do not include \"text overlays\" in the text-to-video prompt.\n",
    " - The prompt should also reference that we are in a chocolate, dessert, coffee shop in Paris so it knows the context of the video.\n",
    " - Do not include children in the text-to-video prompt.\n",
    "\n",
    "{text_to_video_prompt_guide}\n",
    "\"\"\"\n",
    "\n",
    "#print(gemini_ad_prompt)\n",
    "llm_result = GeminiLLM(gemini_ad_prompt, response_schema=response_schema)\n",
    "gemini_ad_results_dict = json.loads(llm_result)\n",
    "orginal_ad_results_dict = gemini_ad_results_dict # in case we swap it for a pre-canned one\n",
    "print(f\"Menu Name: {menu_name} - {menu_description}\")\n",
    "print()\n",
    "print(PrettyPrintJson(json.dumps(gemini_ad_results_dict)))\n",
    "text_to_video_prompts.append(gemini_ad_results_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ju3lRHquN0bL"
   },
   "source": [
    "##### Text-to-Video: Discovering the Chocolate Shop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cSOmdsI5N1J6"
   },
   "outputs": [],
   "source": [
    "# Write me the json in  OpenAPI 3.0 schema object for the below object.\n",
    "# Make all fields required.\n",
    "#  {\n",
    "#    \"text-to-video-prompt\" : \"text\"\n",
    "#  }\n",
    "response_schema = {\n",
    "  \"type\": \"object\",\n",
    "  \"required\": [\n",
    "    \"text-to-video-prompt\"\n",
    "  ],\n",
    "  \"properties\": {\n",
    "    \"text-to-video-prompt\": {\n",
    "      \"type\": \"string\"\n",
    "    }\n",
    "  }\n",
    "}\n",
    "\n",
    "menu_item_list = GetMenuItems()\n",
    "menu_id = random.randint(1, len(menu_item_list))\n",
    "menu_name = menu_item_list[menu_id-1][\"menu_name\"]\n",
    "menu_description = menu_item_list[menu_id-1][\"menu_description\"]\n",
    "\n",
    "gemini_ad_prompt = f\"\"\"You are a marketing expert and are creating a marketing video for a company that sells premium handcrafted chocolates, handmade desserts and delicious coffee drinks.\n",
    "You need to create a marketing video for the company.\n",
    "The company is based in Paris, France.\n",
    "\n",
    "Hidden in the winding streets of Paris there is a small but beautiful shop that sells chocolates and desserts.\n",
    "Two friends discover it while walking down the street.\n",
    "They look in the store window and see beautiful chocolates and desserts.\n",
    "\n",
    "\n",
    "Output Fields:\n",
    "- \"text-to-video-prompt\":\n",
    " - Read the  \"Text-to-Video Prompt Writing Help\" to learn more about how to create good text-to-video prompts.\n",
    " - Make sure you include all the relevant best practices when creating the text-to-video prompt:\n",
    " - A detailed prompt for generating the video using text-to-video technology.\n",
    " - Focus on creating the menu item with an artistic flair.\n",
    " - Do not include \"text overlays\" in the text-to-video prompt.\n",
    " - The prompt should also reference that we are in a chocolate, dessert, coffee shop in Paris so it knows the context of the video.\n",
    " - Do not include children in the text-to-video prompt.\n",
    "\n",
    "{text_to_video_prompt_guide}\n",
    "\"\"\"\n",
    "\n",
    "#print(gemini_ad_prompt)\n",
    "llm_result = GeminiLLM(gemini_ad_prompt, response_schema=response_schema)\n",
    "gemini_ad_results_dict = json.loads(llm_result)\n",
    "orginal_ad_results_dict = gemini_ad_results_dict # in case we swap it for a pre-canned one\n",
    "print(f\"Menu Name: {menu_name} - {menu_description}\")\n",
    "print()\n",
    "print(PrettyPrintJson(json.dumps(gemini_ad_results_dict)))\n",
    "text_to_video_prompts.append(gemini_ad_results_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xCdbpWr_U9G5"
   },
   "source": [
    "##### Text-to-Video: The inside of a display of chocolates and desserts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "WsZazQmDU8IC"
   },
   "outputs": [],
   "source": [
    "# Write me the json in  OpenAPI 3.0 schema object for the below object.\n",
    "# Make all fields required.\n",
    "#  {\n",
    "#    \"text-to-video-prompt\" : \"text\"\n",
    "#  }\n",
    "response_schema = {\n",
    "  \"type\": \"object\",\n",
    "  \"required\": [\n",
    "    \"text-to-video-prompt\"\n",
    "  ],\n",
    "  \"properties\": {\n",
    "    \"text-to-video-prompt\": {\n",
    "      \"type\": \"string\"\n",
    "    }\n",
    "  }\n",
    "}\n",
    "\n",
    "menu_item_list = GetMenuItems()\n",
    "menu_id = random.randint(1, len(menu_item_list))\n",
    "menu_name = menu_item_list[menu_id-1][\"menu_name\"]\n",
    "menu_description = menu_item_list[menu_id-1][\"menu_description\"]\n",
    "\n",
    "gemini_ad_prompt = f\"\"\"You are a marketing expert and are creating a marketing video for a company that sells premium handcrafted chocolates, handmade desserts and delicious coffee drinks.\n",
    "You need to create a marketing video for the company.\n",
    "The company is based in Paris, France.\n",
    "\n",
    "The inside of a display of chocolates and desserts.\n",
    "\n",
    "Output Fields:\n",
    "- \"text-to-video-prompt\":\n",
    " - Read the  \"Text-to-Video Prompt Writing Help\" to learn more about how to create good text-to-video prompts.\n",
    " - Make sure you include all the relevant best practices when creating the text-to-video prompt:\n",
    " - A detailed prompt for generating the video using text-to-video technology.\n",
    " - Focus on creating the menu item with an artistic flair.\n",
    " - Do not include \"text overlays\" in the text-to-video prompt.\n",
    " - The prompt should also reference that we are in a chocolate, dessert, coffee shop in Paris so it knows the context of the video.\n",
    " - Do not include children in the text-to-video prompt.\n",
    "\n",
    "{text_to_video_prompt_guide}\n",
    "\"\"\"\n",
    "\n",
    "#print(gemini_ad_prompt)\n",
    "llm_result = GeminiLLM(gemini_ad_prompt, response_schema=response_schema)\n",
    "gemini_ad_results_dict = json.loads(llm_result)\n",
    "orginal_ad_results_dict = gemini_ad_results_dict # in case we swap it for a pre-canned one\n",
    "print(f\"Menu Name: {menu_name} - {menu_description}\")\n",
    "print()\n",
    "print(PrettyPrintJson(json.dumps(gemini_ad_results_dict)))\n",
    "text_to_video_prompts.append(gemini_ad_results_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6sAG4tKmsxMm"
   },
   "source": [
    "##### Text-to-Video: Show inside of a beautiful Paris chocolate shop with chocolate truffles being made."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "AqZ9NWEkszwE"
   },
   "outputs": [],
   "source": [
    "# Write me the json in  OpenAPI 3.0 schema object for the below object.\n",
    "# Make all fields required.\n",
    "#  {\n",
    "#    \"text-to-video-prompt\" : \"text\"\n",
    "#  }\n",
    "response_schema = {\n",
    "  \"type\": \"object\",\n",
    "  \"required\": [\n",
    "    \"text-to-video-prompt\"\n",
    "  ],\n",
    "  \"properties\": {\n",
    "    \"text-to-video-prompt\": {\n",
    "      \"type\": \"string\"\n",
    "    }\n",
    "  }\n",
    "}\n",
    "\n",
    "menu_item_list = GetMenuItems()\n",
    "menu_id = random.randint(1, len(menu_item_list))\n",
    "menu_name = menu_item_list[menu_id-1][\"menu_name\"]\n",
    "menu_description = menu_item_list[menu_id-1][\"menu_description\"]\n",
    "\n",
    "gemini_ad_prompt = f\"\"\"You are a marketing expert and are creating a marketing video for a company that sells premium handcrafted chocolates, handmade desserts and delicious coffee drinks.\n",
    "You need to create a marketing video for the company.\n",
    "The company is based in Paris, France.\n",
    "\n",
    "Show inside of a beautiful Paris chocolate shop with chocolate truffles being made.\n",
    "Dark chocolate decorated with a white drizzle.\n",
    "The item should be like a piece of art work and be very detailed.\n",
    "Show the finishing touches of the item.\n",
    "\n",
    "\n",
    "Output Fields:\n",
    "- \"text-to-video-prompt\":\n",
    " - Read the  \"Text-to-Video Prompt Writing Help\" to learn more about how to create good text-to-video prompts.\n",
    " - Make sure you include all the relevant best practices when creating the text-to-video prompt:\n",
    " - A detailed prompt for generating the video using text-to-video technology.\n",
    " - Focus on creating the menu item with an artistic flair.\n",
    " - Do not include \"text overlays\" in the text-to-video prompt.\n",
    " - The prompt should also reference that we are in a chocolate, dessert, coffee shop in Paris so it knows the context of the video.\n",
    " - Do not include children in the text-to-video prompt.\n",
    "\n",
    "{text_to_video_prompt_guide}\n",
    "\"\"\"\n",
    "\n",
    "#print(gemini_ad_prompt)\n",
    "llm_result = GeminiLLM(gemini_ad_prompt, response_schema=response_schema)\n",
    "gemini_ad_results_dict = json.loads(llm_result)\n",
    "orginal_ad_results_dict = gemini_ad_results_dict # in case we swap it for a pre-canned one\n",
    "print(f\"Menu Name: {menu_name} - {menu_description}\")\n",
    "print()\n",
    "print(PrettyPrintJson(json.dumps(gemini_ad_results_dict)))\n",
    "text_to_video_prompts.append(gemini_ad_results_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ON8euLEW99ap"
   },
   "source": [
    "### <font color='#4285f4'>View Generate Text-to-Video Prompts</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "n1wOpk6J-Aqn"
   },
   "outputs": [],
   "source": [
    "for item in text_to_video_prompts:\n",
    "  print(\"------------------------------------------------------------------------------------------------------------------------------------\")\n",
    "  print(item[\"text-to-video-prompt\"])\n",
    "  print(\"------------------------------------------------------------------------------------------------------------------------------------\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e0xMMVppTzJ3"
   },
   "source": [
    "### <font color='#4285f4'>Create local working directories</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "o6q7b54ZT1iH"
   },
   "outputs": [],
   "source": [
    "# Create a directory to download the videos from GCS and a directory to combine the text to speech and videos\n",
    "directory_text_to_video = f\"text-to-video-{formatted_date}\"\n",
    "directory_text_to_speech = f\"text-to-speech-{formatted_date}\"\n",
    "directory_video_and_audio = f\"video-and-audio-{formatted_date}\"\n",
    "directory_full_video = f\"full-video-{formatted_date}\"\n",
    "\n",
    "# Holds the local video files (no audio)\n",
    "os.makedirs(directory_text_to_video, exist_ok=True)\n",
    "directory_text_to_video_path = os.getcwd() + f\"/{directory_text_to_video}/\"\n",
    "print(f\"directory_text_to_video_path: {directory_text_to_video_path}\")\n",
    "\n",
    "# Holds the audio file(s)\n",
    "os.makedirs(directory_text_to_speech, exist_ok=True)\n",
    "directory_text_to_speech_path = os.getcwd() + f\"/{directory_text_to_speech}/\"\n",
    "print(f\"directory_text_to_speech_path: {directory_text_to_speech_path}\")\n",
    "\n",
    "# Full video\n",
    "os.makedirs(directory_full_video, exist_ok=True)\n",
    "directory_full_video_path = os.getcwd() + f\"/{directory_full_video}/\"\n",
    "print(f\"directory_full_video_path: {directory_full_video_path}\")\n",
    "\n",
    "full_video_filename_with_audio_en_GB = directory_full_video_path + \"full-video-with-audio-en-GB.mp4\"\n",
    "print(f\"full_video_filename_with_audio_en_GB: {full_video_filename_with_audio_en_GB}\")\n",
    "\n",
    "full_video_filename_with_audio_fr_FR = directory_full_video_path + \"full-video-with-audio-fr-FR.mp4\"\n",
    "print(f\"full_video_filename_with_audio_fr_FR: {full_video_filename_with_audio_fr_FR}\")\n",
    "\n",
    "full_video_filename_no_audio = directory_full_video_path + \"full-video-no-audio.mp4\"\n",
    "print(f\"full_video_filename_no_audio: {full_video_filename_no_audio}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6GTNHglPkDV7"
   },
   "source": [
    "### <font color='#4285f4'>Video Generation</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5XxZTZVCOEjv"
   },
   "source": [
    "##### Function generateVideo (Call Text-to-Video REST API)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "dkyDpT8SOE2k"
   },
   "outputs": [],
   "source": [
    "# Preview Access Required.  Please contact your Google Account Team.\n",
    "\n",
    "def generateVideo(prompt, full_output_gcs_path_and_filename):\n",
    "  return True\n",
    "\n",
    "  \"\"\"Calls text-to-video to create the video and waits for the output (which can be several minutes)\"\"\"\n",
    "\n",
    "  model = \"text-to-video\"\n",
    "  url = f\"https://text-to-video\"\n",
    "\n",
    "  request_body = { }\n",
    "\n",
    "  json_result = restAPIHelper(url, \"POST\", request_body)\n",
    "  operation_name = json_result[\"operation_name\"]\n",
    "\n",
    "  url = f\"https://text-to-video\"\n",
    "\n",
    "  request_body = { }\n",
    "\n",
    "  json_result = restAPIHelper(url, \"POST\", request_body)\n",
    "  status = json_result[\"status\"]\n",
    "\n",
    "  while status == \"RUNNING\":\n",
    "    time.sleep(10)\n",
    "    json_result = restAPIHelper(url, \"POST\", request_body)\n",
    "    status = json_result[\"status\"]\n",
    "\n",
    "  return True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "k66j7GbaA_Lx"
   },
   "source": [
    "#### Use pre-geneated videos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SBreCwTcKnlb"
   },
   "source": [
    "##### Story 1 (Friends)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "HvFfmr89BFHu"
   },
   "outputs": [],
   "source": [
    "# Pre-Generated Videos (text-to-video)\n",
    "# This download the files from a public bucket and then upload them as though text-to-video generated them\n",
    "\"\"\"\n",
    "video_and_audio_processing = []\n",
    "\n",
    "# If you want to use the pre-generate video this will use data from the public storage account\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-01.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-02.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-03.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-04.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-05.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-06.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-07.mp4\"\n",
    "})\n",
    "\n",
    "# Download the videos to the notebook computer\n",
    "for item in video_and_audio_processing:\n",
    "  download_from_gcs(f\"{directory_text_to_video_path}{item['video-filename']}\", \"data-analytics-golden-demo\", f\"chocolate-ai/v1/Campaign-Assets-Text-to-Video-01/story-01/{item['video-filename']}\")\n",
    "\n",
    "  # Simulate that text-to-video generated the videos (pretend that we have the files outputed from text-to-video on storage)\n",
    "  copy_file_to_gcs(f\"{directory_text_to_video_path}{item['video-filename']}\", storage_account, f\"chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/{item['video-filename']}\")\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gJgDqDb0KrYC"
   },
   "source": [
    "##### Story 2 (Swan Cake)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "DjBFOkJ5Ktv8"
   },
   "outputs": [],
   "source": [
    "# Pre-Generated Videos (text-to-video)\n",
    "# This download the files from a public bucket and then upload them as though text-to-video generated them\n",
    "\"\"\"\n",
    "video_and_audio_processing = []\n",
    "\n",
    "# If you want to use the pre-generate video this will use data from the public storage account\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-01.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-02.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-03.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-04.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-05.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-06.mp4\"\n",
    "})\n",
    "\n",
    "\n",
    "# Download the videos to the notebook computer\n",
    "for item in video_and_audio_processing:\n",
    "  download_from_gcs(f\"{directory_text_to_video_path}{item['video-filename']}\", \"data-analytics-golden-demo\", f\"chocolate-ai/v1/Campaign-Assets-Text-to-Video-01/story-02/{item['video-filename']}\")\n",
    "\n",
    "  # Simulate that text-to-video generated the videos (pretend that we have the files outputed from text-to-video on storage)\n",
    "  copy_file_to_gcs(f\"{directory_text_to_video_path}{item['video-filename']}\", storage_account, f\"chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/{item['video-filename']}\")\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n7A8XEzrKuoS"
   },
   "source": [
    "##### Story 3 (Chocolatier)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gAK-aUqSKxFm"
   },
   "outputs": [],
   "source": [
    "# Pre-Generated Videos (text-to-video)\n",
    "# This download the files from a public bucket and then upload them as though text-to-video generated them\n",
    "\n",
    "video_and_audio_processing = []\n",
    "\n",
    "# If you want to use the pre-generate video this will use data from the public storage account\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-01.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-02.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-03.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-04.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-05.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-06.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-07.mp4\"\n",
    "})\n",
    "video_and_audio_processing.append({\n",
    "    \"video-filename\" : \"text-to-video-08.mp4\"\n",
    "})\n",
    "\n",
    "# Download the videos to the notebook computer\n",
    "for item in video_and_audio_processing:\n",
    "  download_from_gcs(f\"{directory_text_to_video_path}{item['video-filename']}\", \"data-analytics-golden-demo\", f\"chocolate-ai/v1/Campaign-Assets-Text-to-Video-01/story-03/{item['video-filename']}\")\n",
    "\n",
    "  # Simulate that text-to-video generated the videos (pretend that we have the files outputed from text-to-video on storage)\n",
    "  copy_file_to_gcs(f\"{directory_text_to_video_path}{item['video-filename']}\", storage_account, f\"chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/{item['video-filename']}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4vwwZIC-MLPJ"
   },
   "source": [
    "#### Create the Videos (using text-to-video)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "K9c_G_2J6FZD"
   },
   "outputs": [],
   "source": [
    "# We need to swap our prompts to our pre-generated prompts (this is Story 1)\n",
    "text_to_video_prompts = []\n",
    "text_to_video_prompts.append({\n",
    "    \"text-to-video-prompt\" : \"Couple going down the streets and seeing a chocolate shop, like window shopping and seeing the display of chocolate and desserts.\"\n",
    "    })\n",
    "\n",
    "text_to_video_prompts.append({\n",
    "    \"text-to-video-prompt\" : \"Camera slowly moves forward, showcasing the exterior of a Parisian cafe.  The storefront is painted a light blue with gold trim. Red and white striped awnings protect small round tables, each adorned with a single red rose in a vase. The camera pans to the left to show two friends, a man and a woman, walking down the street. They stop at the storefront and look through the window at beautiful pastries on display. The camera zooms in on their faces as their eyes widen in amazement as they gaze upon rows of colorful macarons in pastel pink, green, and yellow, gleaming eclairs, and artfully crafted chocolate sculptures. Slow zoom, warm sunlight, ambiance of wonder, Paris street scene.\"\n",
    "    })\n",
    "\n",
    "text_to_video_prompts.append({\n",
    "    \"text-to-video-prompt\" : \"Camera close-up of a chocolatier chef delicately placing chocolate curls onto a \\\"Parisian Chocolate Symphony\\\" dessert to a wide shot of the picturesque Parisian street scene outside the window.  The Parisian chocolate shop scene is softly lit and warm. The camera pans to the window.\"\n",
    "    })\n",
    "\n",
    "text_to_video_prompts.append({\n",
    "    \"text-to-video-prompt\" : \"Medium range shot of three friends sitting at a bistro table, laughing together, Parisian cafe in the background, warm afternoon sunlight.  The camera slowly pushes in on the table as we hear the friends talking about the delicious premium handcrafted chocolates, handmade desserts, and coffee drinks in front of them. Close up of the Parisian Surprise, a playful dome of rich milk chocolate, hiding a secret center of marshmallow fluff and a sprinkle of sea salt.  Pull back to reveal the Doughnut Delight: Five fluffy, mini chocolate chip brioche donuts are served skewered and standing upright in a bed of creamy vanilla bean ice cream.  Topped with a drizzle of homemade salted caramel sauce and a sprinkle of cocoa powder. Camera slowly rotates around to the Rich Indulgence: Three layers of our darkest chocolate mousse, separated by thin layers of hazelnut dacquoise and topped with a delicate chocolate cage.  Finally, a slow zoom in on the Midnight in Provence: A whimsical dance of dark chocolate mousse infused with lavender, nestled atop a buttery almond biscuit. Experience the surprise of black currant jelly pockets and a sprinkle of delicate lavender buds. Warm, inviting, indulgent.\"\n",
    "    })\n",
    "\n",
    "# Same prompt a previous one\n",
    "text_to_video_prompts.append({\n",
    "    \"text-to-video-prompt\" : \"Medium range shot of three friends sitting at a bistro table, laughing together, Parisian cafe in the background, warm afternoon sunlight.  The camera slowly pushes in on the table as we hear the friends talking about the delicious premium handcrafted chocolates, handmade desserts, and coffee drinks in front of them. Close up of the Parisian Surprise, a playful dome of rich milk chocolate, hiding a secret center of marshmallow fluff and a sprinkle of sea salt.  Pull back to reveal the Doughnut Delight: Five fluffy, mini chocolate chip brioche donuts are served skewered and standing upright in a bed of creamy vanilla bean ice cream.  Topped with a drizzle of homemade salted caramel sauce and a sprinkle of cocoa powder. Camera slowly rotates around to the Rich Indulgence: Three layers of our darkest chocolate mousse, separated by thin layers of hazelnut dacquoise and topped with a delicate chocolate cage.  Finally, a slow zoom in on the Midnight in Provence: A whimsical dance of dark chocolate mousse infused with lavender, nestled atop a buttery almond biscuit. Experience the surprise of black currant jelly pockets and a sprinkle of delicate lavender buds. Warm, inviting, indulgent.\"\n",
    "    })\n",
    "\n",
    "text_to_video_prompts.append({\n",
    "    \"text-to-video-prompt\" : \"Close-up shot of a pastry chef dusting  eclair with powdered sugar in a Parisian cafe, warm lighting, focus on the pastry.\"\n",
    "    })\n",
    "\n",
    "text_to_video_prompts.append({\n",
    "    \"text-to-video-prompt\" : \"Medium range shot of three friends sitting at a bistro table, laughing together, Parisian cafe in the background, warm afternoon sunlight.  The camera slowly pushes in on the table as we hear the friends talking about the delicious premium handcrafted chocolates, handmade desserts, and coffee drinks in front of them. Close up of the Parisian Surprise, a playful dome of rich milk chocolate, hiding a secret center of marshmallow fluff and a sprinkle of sea salt.  Pull back to reveal the Doughnut Delight: Five fluffy, mini chocolate chip brioche donuts are served skewered and standing upright in a bed of creamy vanilla bean ice cream.  Topped with a drizzle of homemade salted caramel sauce and a sprinkle of cocoa powder. Camera slowly rotates around to the Rich Indulgence: Three layers of our darkest chocolate mousse, separated by thin layers of hazelnut dacquoise and topped with a delicate chocolate cage.  Finally, a slow zoom in on the Midnight in Provence: A whimsical dance of dark chocolate mousse infused with lavender, nestled atop a buttery almond biscuit. Experience the surprise of black currant jelly pockets and a sprinkle of delicate lavender buds. Warm, inviting, indulgent.\"\n",
    "    })\n",
    "\n",
    "# if using Story 2, you need to remove a prompt (we are not actually using the prompt text)\n",
    "#text_to_video_prompts.pop()\n",
    "\n",
    "# if using Story 3, you need an additional prompt\n",
    "text_to_video_prompts.append({\"text-to-video-prompt\" : \"Text does not matter\" })\n",
    "\n",
    "i = 1\n",
    "for item in text_to_video_prompts:\n",
    "  filename = f\"text-to-video-0{i}.mp4\"\n",
    "  text_to_video_output_path_for_generated_video = f\"gs://{storage_account}/chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/{filename}\"\n",
    "\n",
    "  # No need to run this for the pre-generated videos\n",
    "  # generateVideo(item[\"text-to-video-prompt\"], text_to_video_output_path_for_generated_video)\n",
    "\n",
    "  # Download the generate video from GCS\n",
    "  download_from_gcs(f\"{directory_text_to_video_path}{filename}\", storage_account, f\"chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/{filename}\")\n",
    "\n",
    "  i = i + 1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "whH3KKEGWJ-t"
   },
   "outputs": [],
   "source": [
    "# See the downloaded files\n",
    "!ls {directory_text_to_video_path}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QVsdBsl_Kjs-"
   },
   "source": [
    "### <font color='#4285f4'>Merge the videos into one video</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "02ubD5duJGIQ"
   },
   "outputs": [],
   "source": [
    "# Merge the videos (the file names are when sorted match the placement of each video in the overall video)\n",
    "\n",
    "print(\"Merging videos (without audio)\")\n",
    "print(f\"directory_text_to_video_path: {directory_text_to_video_path}\")\n",
    "print(f\"full_video_filename_no_audio: {full_video_filename_no_audio}\")\n",
    "\n",
    "merge_videos_sorted(directory_text_to_video_path, full_video_filename_no_audio)\n",
    "copy_file_to_gcs(f\"{full_video_filename_no_audio}\", storage_account, f\"chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/full-video-no-audio.mp4\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "WYXhNh1XYHVD"
   },
   "outputs": [],
   "source": [
    "# View the directory of the merge text-to-video (no audio)\n",
    "!ls {full_video_filename_no_audio}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Wgq5iz30CfwN"
   },
   "outputs": [],
   "source": [
    "# prompt: python to play a mp4 in a jupyter notebook\n",
    "video_mp4 = open(full_video_filename_no_audio, 'rb').read()\n",
    "video_url = \"data:video/mp4;base64,\" + base64.b64encode(video_mp4).decode()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "zOf7J4OaDmSo"
   },
   "outputs": [],
   "source": [
    "# 16:9 aspect ratio\n",
    "HTML(f\"\"\"\n",
    "<p>Combined text-to-video using moviepy.</p>\n",
    "<video width=600 height=337 controls>\n",
    "      <source src=\"{video_url}\" type=\"video/mp4\">\n",
    "</video>\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nOWzEh-WkGfQ"
   },
   "source": [
    "### <font color='#4285f4'>Text to Speech Generation - Use Gemini to write a script (let it watch the video and come up with the voice over)</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "LIgDjiAHYl2y"
   },
   "outputs": [],
   "source": [
    "fileUri = f\"gs://{storage_account}/chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/full-video-no-audio.mp4\"\n",
    "\n",
    "# Write me the json in  OpenAPI 3.0 schema object for the below object.\n",
    "# Make all fields required.\n",
    "#{\n",
    "#  \"voiceover\": [\n",
    "#    {\n",
    "#      \"timestamp\": \"0:00-0:05\",\n",
    "#      \"visuals\": \"Show storefront and pastries in the window\",\n",
    "#      \"script\": \"Craving something delicious?  Something decadent?  At AI Chocolates, we hand-craft elegant chocolates, pastries, and desserts that are as delightful to look at as they are to taste.\"\n",
    "#   }\n",
    "# ]\n",
    "\n",
    "response_schema = {\n",
    "  \"type\": \"object\",\n",
    "  \"required\": [\"voiceover\"],\n",
    "  \"properties\": {\n",
    "    \"voiceover\": {\n",
    "      \"type\": \"array\",\n",
    "      \"items\": {\n",
    "        \"type\": \"object\",\n",
    "        \"required\": [\"timestamp\", \"visuals\", \"script\"],\n",
    "        \"properties\": {\n",
    "          \"timestamp\": {\n",
    "            \"type\": \"string\"\n",
    "          },\n",
    "          \"visuals\": {\n",
    "            \"type\": \"string\"\n",
    "          },\n",
    "          \"script\": {\n",
    "            \"type\": \"string\"\n",
    "          },\n",
    "          \"explanation\": {\n",
    "            \"type\": \"string\"\n",
    "          }\n",
    "        }\n",
    "      }\n",
    "    }\n",
    "  }\n",
    "}\n",
    "\n",
    "prompt = \"\"\"You are a marketing expert, watch the attached video for a chocolate, dessert, coffee shop named Chocolate A.I. and create a voice over script.\n",
    "\n",
    "Sample Json Output:\n",
    "{\n",
    "  \"voiceover\": [\n",
    "    {\n",
    "      \"timestamp\": \"0:00-0:05\",\n",
    "      \"visuals\": \"Show storefront and pastries in the window\",\n",
    "      \"script\": \"Craving something delicious?  Something decadent?  At AI Chocolates, we hand-craft elegant chocolates, pastries, and desserts that are as delightful to look at as they are to taste.\",\n",
    "      \"explanation\": \"Opens with enticing questions to capture attention and introduces the brand with a focus on craftsmanship and visual appeal.\"\n",
    "    },\n",
    "    {\n",
    "      \"timestamp\": \"0:06-0:11\",\n",
    "      \"visuals\": \"Continue to show storefront and various pastries\",\n",
    "      \"script\": \"Our light and airy shop, with its charming Parisian flair, offers the perfect setting to relax, indulge, and escape the everyday.\",\n",
    "      \"explanation\": \"Highlights the shop's ambiance and atmosphere, emphasizing a sense of indulgence and escape.\"\n",
    "    },\n",
    "    {\n",
    "      \"timestamp\": \"0:12-0:17\",\n",
    "      \"visuals\": \"Show the chef preparing a chocolate assortment and then friends enjoying treats\",\n",
    "      \"script\": \"Using only the finest ingredients, our expert chocolatiers create a symphony of flavors that will tantalize your taste buds.\",\n",
    "      \"explanation\": \"Emphasizes the quality of ingredients and the expertise of the chocolatiers, appealing to discerning taste buds.\"\n",
    "    },\n",
    "    {\n",
    "      \"timestamp\": \"0:18-0:29\",\n",
    "      \"visuals\": \"Focus on the joy and connection happening over the food\",\n",
    "      \"script\": \"Whether you're looking for a special treat for yourself or a loved one, AI Chocolates offers a range of options to satisfy every sweet tooth.  Come experience the magic of AI Chocolates.\",\n",
    "      \"explanation\": \"Positions AI Chocolates as a destination for both personal indulgence and gifting, while emphasizing the variety and ability to satisfy any chocolate craving. Ends with a call to action.\"\n",
    "    },\n",
    "    {\n",
    "      \"timestamp\": \"Optional closing shot\",\n",
    "      \"visuals\": \"Storefront, logo, or website address\",\n",
    "      \"script\": \"AI Chocolates. Where happiness is always on the menu.\",\n",
    "      \"explanation\": \"Reinforces the brand name and leaves a lasting impression by connecting AI Chocolates with happiness.\"\n",
    "    }\n",
    "  ]\n",
    "}\n",
    "\"\"\"\n",
    "\n",
    "multimodal_prompt_list = [\n",
    "    { \"text\": prompt },\n",
    "    { \"fileData\": {  \"mimeType\": \"video/mp4\", \"fileUri\": fileUri } },\n",
    "  ]\n",
    "\n",
    "voice_llm_response = GeminiLLM_Multimodal(multimodal_prompt_list, response_schema=response_schema)\n",
    "voice_llm_dict = json.loads(voice_llm_response)\n",
    "\n",
    "print(PrettyPrintJson(voice_llm_response))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "HclEUE25z6ce"
   },
   "outputs": [],
   "source": [
    "# Save the prompt so we know how we got this data\n",
    "with open(\"full-text-to-speech.txt\", \"w\") as f:\n",
    "  f.write(PrettyPrintJson(voice_llm_response))\n",
    "\n",
    "# upload it to GCS\n",
    "copy_file_to_gcs(\"full-text-to-speech.txt\", storage_account, f\"chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/full-text-to-speech.txt\")\n",
    "delete_file(\"full-text-to-speech.txt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cTn39dKEzxCX"
   },
   "outputs": [],
   "source": [
    "# If you want the script from the orginally generated video, this is the text to speech\n",
    "text = \"\"\"\n",
    "Craving something delicious? Something decadent? At Chocolate A.I., we hand-craft elegant chocolates, pastries, and desserts that are as delightful to look at as they are to taste.\n",
    "Our light and airy shop, with its charming Parisian flair, offers the perfect setting to relax, indulge, and escape the everyday.\n",
    "Using only the finest ingredients, our expert chocolatiers create a symphony of flavors that will tantalize your taste buds.\n",
    "Whether you're looking for a special treat for yourself or a loved one,Chocolate A.I. offers a range of options to satisfy every sweet tooth.\n",
    "Chocolate A.I. Where happiness is always on the menu.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "unra5_W9zQ4r"
   },
   "outputs": [],
   "source": [
    "# British\n",
    "language_code = \"en-GB\"\n",
    "language_code_name = \"en-GB-Neural2-B\"\n",
    "ssml_gender = \"MALE\"\n",
    "\n",
    "text = \"\"\n",
    "for item in voice_llm_dict[\"voiceover\"]:\n",
    "  text += item[\"script\"] + \"  \"\n",
    "\n",
    "text = text.strip()\n",
    "\n",
    "# NOTE: You might need to change \"Chocolate A.I. -> Chocolate A.I.. (put a second period at the end of a sentance).\"\n",
    "# Story 2\n",
    "# text = \"Indulge in the artistry of chocolate at Chocolate A.I.. ----  Each creation is a testament to our passion for crafting edible masterpieces.  From the finest cocoa beans to the most delicate decorations, we use only the highest quality ingredients.  Whether you're celebrating a special occasion or simply treating yourself, Chocolate A.I. offers a symphony of flavors to satisfy your sweetest cravings.  Come experience the magic of  Chocolate A.I., where every bite is a work of art.  Chocolate A.I.. - Indulge  your senses.\"\n",
    "# Story 3\n",
    "# text = \"Indulge in the artistry of chocolate at Chocolate A.I., where each piece is handcrafted to perfection.  Our master chocolatiers use only the finest, ethically sourced cocoa beans and the freshest ingredients to create a symphony of flavors that will tantalize your taste buds.  From decadent truffles to exquisite bonbons and luxurious chocolate bars, each creation is a testament to our unwavering commitment to quality and craftsmanship.  More than just a chocolate shop, Chocolate A.I. is an experience. A place where you can escape the everyday and savor moments of pure indulgence.  Visit Chocolate A.I. and discover a world of chocolate beyond your wildest imagination.  Chocolate A.I.. - Where artistry meets indulgence.\"\n",
    "\n",
    "audio_file_name = f\"{directory_text_to_speech_path}full-text-to-speech-en-GB.mp3\"\n",
    "\n",
    "print (f\"Text: {text}\")\n",
    "TextToSpeech(audio_file_name,text, language_code, language_code_name, ssml_gender, .90)\n",
    "\n",
    "display(Audio(audio_file_name, autoplay=True,rate=16000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "LTNXdid4dcoc"
   },
   "outputs": [],
   "source": [
    "!ls {directory_text_to_speech_path}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "i-bZChhB15un"
   },
   "outputs": [],
   "source": [
    "# upload it to GCS\n",
    "copy_file_to_gcs(audio_file_name, storage_account, f\"chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/full-text-to-speech-en-GB.mp3\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "kGDVd-xEzu1H"
   },
   "outputs": [],
   "source": [
    "# French\n",
    "language_code = \"fr-FR\"\n",
    "language_code_name = \"fr-FR-Neural2-B\"\n",
    "ssml_gender = \"MALE\"\n",
    "\n",
    "text = \"\"\n",
    "for item in voice_llm_dict[\"voiceover\"]:\n",
    "  text += item[\"script\"] + \"  \"\n",
    "\n",
    "text = text.strip()\n",
    "\n",
    "# NOTE: You might need to change \"Chocolate A.I. -> Chocolate A.I.. (put a second period at the end of a sentance).\"\n",
    "# Story 2\n",
    "# text = \"Indulge in the artistry of chocolate at Chocolate A.I.. ----  Each creation is a testament to our passion for crafting edible masterpieces.  From the finest cocoa beans to the most delicate decorations, we use only the highest quality ingredients.  Whether you're celebrating a special occasion or simply treating yourself, Chocolate A.I. offers a symphony of flavors to satisfy your sweetest cravings.  Come experience the magic of  Chocolate A.I., where every bite is a work of art.  Chocolate A.I.. - Indulge  your senses.\"\n",
    "# Story 3\n",
    "text = \"Indulge in the artistry of chocolate at Chocolate A.I., where each piece is handcrafted to perfection.  Our master chocolatiers use only the finest, ethically sourced cocoa beans and the freshest ingredients to create a symphony of flavors that will tantalize your taste buds.  From decadent truffles to exquisite bonbons and luxurious chocolate bars, each creation is a testament to our unwavering commitment to quality and craftsmanship.  More than just a chocolate shop, Chocolate A.I. is an experience. A place where you can escape the everyday and savor moments of pure indulgence.  Visit Chocolate A.I. and discover a world of chocolate beyond your wildest imagination.  Chocolate A.I.. - Where artistry meets indulgence.\"\n",
    "\n",
    "audio_file_name = f\"{directory_text_to_speech_path}full-text-to-speech-fr-FR.mp3\"\n",
    "\n",
    "print (f\"Text: {text}\")\n",
    "TextToSpeech(audio_file_name,text, language_code, language_code_name, ssml_gender, .95) # 1.01)\n",
    "\n",
    "display(Audio(audio_file_name, autoplay=True,rate=16000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PYT3d0Lcf5Er"
   },
   "outputs": [],
   "source": [
    "!ls {directory_text_to_speech_path}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "kSisdXg7f6rz"
   },
   "outputs": [],
   "source": [
    "# upload it to GCS\n",
    "copy_file_to_gcs(audio_file_name, storage_account, f\"chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/full-text-to-speech-fr-FR.mp3\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "94init3ix9kE"
   },
   "source": [
    "### <font color='#4285f4'>Combine Video with Audio (text-to-video with text-to-audio)</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "igc6PaZcH7Ti"
   },
   "source": [
    "#### English"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "a38ZuOz1cer1"
   },
   "outputs": [],
   "source": [
    "audio_file_name = f\"{directory_text_to_speech_path}full-text-to-speech-en-GB.mp3\"\n",
    "\n",
    "print(f\"full_video_filename_no_audio: {full_video_filename_no_audio}\")\n",
    "print(f\"audio_file_name: {audio_file_name}\")\n",
    "print(f\"full_video_filename_with_audio_en_GB: {full_video_filename_with_audio_en_GB}\")\n",
    "\n",
    "MergeVideoAndAudio(full_video_filename_no_audio, audio_file_name, full_video_filename_with_audio_en_GB)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "fKSXJQCRiGAC"
   },
   "outputs": [],
   "source": [
    "! ls {directory_full_video_path}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MWPygOjFKITJ"
   },
   "outputs": [],
   "source": [
    "# Show the first merged file\n",
    "# prompt: python to play a mp4 in a jupyter notebook\n",
    "video_mp4 = open(full_video_filename_with_audio_en_GB, 'rb').read()\n",
    "video_url = \"data:video/mp4;base64,\" + base64.b64encode(video_mp4).decode()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "svGgFRm6KLT2"
   },
   "outputs": [],
   "source": [
    "# Play the video\n",
    "# 16:9 aspect ratio\n",
    "HTML(f\"\"\"\n",
    "<p>You might need to speed up or slow down the text-to-speech to match the overall video length.</p>\n",
    "<video width=600 height=337 controls>\n",
    "      <source src=\"{video_url}\" type=\"video/mp4\">\n",
    "</video>\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MRuxxscCiyxB"
   },
   "outputs": [],
   "source": [
    "# Up load the completed video to GCS\n",
    "copy_file_to_gcs(full_video_filename_with_audio_en_GB, storage_account, f\"chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/full-video-with-audio-en-GB.mp4\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Be99BlCDIAG0"
   },
   "source": [
    "#### French"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NxkCeteJIAG0"
   },
   "outputs": [],
   "source": [
    "audio_file_name = f\"{directory_text_to_speech_path}full-text-to-speech-fr-FR.mp3\"\n",
    "\n",
    "print(f\"full_video_filename_no_audio: {full_video_filename_no_audio}\")\n",
    "print(f\"audio_file_name: {audio_file_name}\")\n",
    "print(f\"full_video_filename_with_audio_efull_video_filename_with_audio_fr_FRn_GB: {full_video_filename_with_audio_fr_FR}\")\n",
    "\n",
    "MergeVideoAndAudio(full_video_filename_no_audio, audio_file_name, full_video_filename_with_audio_fr_FR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7X5NtXbYIAG0"
   },
   "outputs": [],
   "source": [
    "! ls {directory_full_video_path}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "LV8l9bvBIAG0"
   },
   "outputs": [],
   "source": [
    "# Show the first merged file\n",
    "# prompt: python to play a mp4 in a jupyter notebook\n",
    "video_mp4 = open(full_video_filename_with_audio_fr_FR, 'rb').read()\n",
    "video_url = \"data:video/mp4;base64,\" + base64.b64encode(video_mp4).decode()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gZXR9J1_IAG0"
   },
   "outputs": [],
   "source": [
    "# Play the video\n",
    "# 16:9 aspect ratio\n",
    "HTML(f\"\"\"\n",
    "<p>You might need to speed up or slow down the text-to-speech to match the overall video length.</p>\n",
    "<video width=600 height=337 controls>\n",
    "      <source src=\"{video_url}\" type=\"video/mp4\">\n",
    "</video>\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "UusJ_ZGSIAG1"
   },
   "outputs": [],
   "source": [
    "# Up load the completed video to GCS\n",
    "copy_file_to_gcs(full_video_filename_with_audio_fr_FR, storage_account, f\"chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}/full-video-with-audio-fr-FR.mp4\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "x4zDFw8d1hXb"
   },
   "source": [
    "### <font color='#4285f4'>View the Video in Cloud Storage</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wt-mRH-n07EW"
   },
   "outputs": [],
   "source": [
    "print(f\"View the GCS bucket: https://console.cloud.google.com/storage/browser/{storage_account}/chocolate-ai/Campaign-Assets-Text-to-Video-01/text-to-video-{formatted_date}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "42IxhtRRrvR-"
   },
   "source": [
    "### <font color='#4285f4'>Clean Up</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "6lF2Z7skFbvf"
   },
   "outputs": [],
   "source": [
    "user_input = input(\"Do you want to the files on this notebook machine (Y/n)?\")\n",
    "if user_input == \"Y\":\n",
    "  import shutil\n",
    "  print(f\"Removing directory: {directory_text_to_video_path}\")\n",
    "  shutil.rmtree(directory_text_to_video_path)\n",
    "\n",
    "  print(f\"Removing directory: {directory_text_to_speech_path}\")\n",
    "  shutil.rmtree(directory_text_to_speech_path)\n",
    "\n",
    "  print(f\"Removing directory: {directory_full_video_path}\")\n",
    "  shutil.rmtree(directory_full_video_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ASQ2BPisXDA0"
   },
   "source": [
    "### <font color='#4285f4'>Reference Links</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pRQZxRKA6FZE"
   },
   "source": [
    "- [Google.com](https://www.google.com)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "HMsUvoF4BP7Y",
    "m65vp54BUFRi",
    "UmyL-Rg4Dr_f",
    "JbOjdSP1kN9T",
    "Hh6lubX6VTHv",
    "wySHopHNei3E",
    "CWYQfM8BK5Sn",
    "AqKylYj_LFqB",
    "FbQT-vIUi8ab",
    "wF4xTwDM9yvp",
    "uBmWWkEb9cTF",
    "JZ1K2ofwKDnL",
    "hgFkSBarIx7H",
    "GNhgCtUdJ6HL",
    "ju3lRHquN0bL",
    "xCdbpWr_U9G5",
    "6sAG4tKmsxMm",
    "SBreCwTcKnlb",
    "gJgDqDb0KrYC",
    "n7A8XEzrKuoS",
    "94init3ix9kE",
    "42IxhtRRrvR-",
    "ASQ2BPisXDA0"
   ],
   "name": "Campaign-Assets-Text-to-Video-01",
   "private_outputs": true,
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}