Skip to content

Commit

Permalink
Adding video understanding example (#367)
Browse files Browse the repository at this point in the history
* Video understanding example

* Adding link to the video understanding notebook

* Adding links to the video understanding notebook

* Typos and licence

* Typos and some noew prompts

* Colab link

* Colab link
  • Loading branch information
Giom-V authored Dec 18, 2024
1 parent d79abc5 commit 01170a9
Show file tree
Hide file tree
Showing 5 changed files with 721 additions and 5 deletions.
3 changes: 2 additions & 1 deletion gemini-2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@ Explore Gemini 2.0’s capabilities through the following notebooks using Google
* [Search tool](./search_tool.ipynb) \- Quick start using the Search tool with the unary and Live APIs in the GenAI SDK
* [Spatial understanding](./spatial_understanding.ipynb) \- Comprehensive overview of 2D spatial understanding capabilities with the GenAI SDK
* [Spatial understanding (3D)](./spatial_understanding_3d.ipynb) \- Comprehensive overview of 3D spatial understanding capabilities with the GenAI SDK
* [Video understanding](./video_understanding.ipynb) \- Comprehensive overview Gemini 2.0 video understanding capabilities with the GenAI SDK

Or explore on your own local machine.

* [Live API starter script](./live_api_starter.py) \- A locally runnable Python script using GenAI SDK that supports streaming audio in and audio + video out from your machine
* [Live API starter script](./live_api_starter.py) \- A locally runnable Python script using GenAI SDK that supports streaming audio + video (camera or screen) in and audio from your machine

Also find websocket-specific examples in the [`websockets`](./websockets/) directory.
2 changes: 1 addition & 1 deletion gemini-2/live_api_starter.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -656,7 +656,7 @@
"- If you're interested in the low level details of using the websockets directly, see the [websocket version of this tutorial](../gemini-2/websockets/live_api_starter.ipynb).\n",
"- Try the [Tool use in the live API tutorial](../gemini-2/live_api_tool_use.ipynb) for an walkthrough of Gemini-2's new tool use capabilities.\n",
"- There is a [Streaming audio in Colab example](../gemini-2/websockets/live_api_streaming_in_colab.ipynb), but this is more of a **demo**, it's **not optimized for readability**.\n",
"- Other nice Gemini 2.0 examples can also be found in the [Cookbook](https://github.com/google-gemini/cookbook/blob/main/gemini-2/).\n"
"- Other nice Gemini 2.0 examples can also be found in the [Cookbook](https://github.com/google-gemini/cookbook/blob/main/gemini-2/), in particular the [video understanding](./video_understanding.ipynb) and the [spatial understanding](./spatial_understanding.ipynb) ones.\n"
]
}
],
Expand Down
4 changes: 2 additions & 2 deletions gemini-2/spatial_understanding.ipynb

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion gemini-2/spatial_understanding_3d.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4710,7 +4710,7 @@
"\n",
"Related to image recognition and reasoning, other than the [2D spatial understanding](spatial_understanding.ipynb) notebook, [Market a jet backpack](../examples/Market_a_Jet_Backpack.ipynb) and [Guess the shape](../examples/Guess_the_shape.ipynb) examples are worth checking to continue exploring the capabilities of Gemini (Note: These use the old SDK).\n",
"\n",
"Other Gemini 2.0 examples are available in the [Gemini 2.0 cookbook](../gemini-2/). The [audio streaming (right from Colab)](../gemini-2/live_api_streaming_in_colab.ipynb) and [multiple tools](../gemini-2/plotting_and_mapping.ipynb) examples are in particular worth checking if you are interested in advanced capabilities of the model."
"Other Gemini 2.0 examples are available in the [Gemini 2.0 cookbook](../gemini-2/). The [video understanding](./video_understanding.ipynb), [audio streaming (right from Colab)](../gemini-2/live_api_streaming_in_colab.ipynb) and [multiple tools](../gemini-2/plotting_and_mapping.ipynb) examples are in particular worth checking if you are interested in advanced capabilities of the model."
]
}
],
Expand Down
Loading

0 comments on commit 01170a9

Please sign in to comment.