Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Annotator overlays in ml_annotators #86

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Annotator overlays in ml_annotators #86

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e52ad93
Tentatively updated environment for next gv/hv dev releases
jbednar Jan 15, 2020
be63cc2
Added separate point-drawing examples and simplified overlays
jbednar Jan 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ml_annotators/anaconda-project.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ packages: &pkgs
- ipykernel=5.1.0
- pandas=0.24
- bokeh=1.4.0
- holoviews=1.13.0a21
- geoviews=1.7.0a6
- holoviews=1.13.0a22
- geoviews=1.7.0a7
- panel=0.7.1a2

dependencies: *pkgs
Expand Down
70 changes: 55 additions & 15 deletions ml_annotators/ml_annotators.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"metadata": {},
"outputs": [],
"source": [
"import holoviews as hv\n",
"import holoviews as hv, holoviews.streams as hvs\n",
"import geoviews as gv\n",
"\n",
"hv.extension('bokeh')"
Expand All @@ -20,11 +20,11 @@
"\n",
"The [Bokeh](https://bokeh.org) Python plotting library lets users build interactive apps and plots in a web browser for a very wide variety of data types. The high-level library [HoloViews](https://holoviews.org) builds on Bokeh, making it easier to use for common data-processing tasks, and the corresponding [GeoViews](https://geoviews.org) library adds support for plotting in geographic coordinate systems. \n",
"\n",
"These tools now (as of development releases in 1/2020) all support interactively collecting data _from_ the user, not just interacting with existing data, with components provided by HoloViews (and by GeoViews for data on maps) that make it simple to get data into Python ready to process and use for tagging data for machine-learning pipelines (or any other purpose!). These \"annotation\" and \"drawing\" tools can be used to annotate existing data sets or geographic locations, to classify each example or regions into categories or with numeric or other labels.\n",
"These tools now (as of development releases in 1/2020) all support interactively collecting data _from_ the user, not just interacting with existing data, with components provided by HoloViews (and by GeoViews for data on maps) that make it simple to get data into Python ready to process and use for tagging data for machine-learning pipelines (or any other purpose!). These \"drawing\" and \"annotation\" tools can be used to annotate existing data sets or geographic locations, to classify each example or regions into categories or with numeric or other labels.\n",
"\n",
"These tools make it possible to work directly with data in its native values (*as data*) and then immediately use it for further processing in Python. Other tools like [labelImg](https://github.com/tzutalin/labelImg) will usually be faster and easier to use for the specific things they do, so if one of those meets your need, use it! Meanwhile, use the Bokeh/HoloViews annotation tools if you want to quickly create a fully custom app for special purposes, especially if you want to stay working with data you are already using in Python, in its native coordinates.\n",
"\n",
"Here, we will focus only on the easy-to-use, high-level [\"annotator\" components from HoloViews](http://build.holoviews.org/user_guide/Annotators.html); fully custom control is always available by using [Bokeh's drawing tools](https://docs.bokeh.org/en/latest/docs/reference/models/tools.html#bokeh.models.tools.PointDrawTool) directly."
"Here, we will focus primarily on the easy-to-use, high-level [\"annotator\" components from HoloViews](http://build.holoviews.org/user_guide/Annotators.html); fully custom control is always available by using [Bokeh's drawing tools](https://docs.bokeh.org/en/latest/docs/reference/models/tools.html#bokeh.models.tools.PointDrawTool) directly."
]
},
{
Expand Down Expand Up @@ -89,7 +89,50 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The Bokeh tools in the tool bar let you pan and zoom on this plot interactively, but the data in it is fixed. What if we wanted to label all the trees that we can see here, i.e. add more data points? That's where the HoloViews Annotators come in."
"## HoloViews PointDraw tool\n",
"\n",
"The Bokeh tools in the tool bar above let you pan and zoom on that plot interactively, but the data in it is fixed. What if we wanted to label all the trees that we can see there, i.e. add more data points? Bokeh and HoloViews support \"drawing tools\" that can edit data on plots, and it's easy to add a drawing tool to add more points wherever you like:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"point_stream = hvs.PointDraw(data=points.columns(), source=points, empty_value=None)\n",
"points.opts(active_tools=['point_draw'], tools=['hover'])\n",
"tiles * points"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You'll see that there is now a [PointDraw tool](../reference/streams/bokeh/PointDraw.ipynb) in the toolbar:<img src=\"https://bokeh.pydata.org/en/latest/_images/PointDraw.png\">\n",
"\n",
"Once you select that tool, you should be able to click and drag any of the existing points to a new location, as well as click to add a new point anywhere there is not already one. You can delete points by selecting them in the plot and pressing Backspace or Delete (depending on operating system).\n",
"\n",
"Try clicking on some of the trees that haven't been labeled. Once you are happy, you can get the data back into Python to do whatever you want with it, whether that be saving to a file or immediately performing some computations:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"pd.DataFrame(point_stream.data)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The other drawing tools in HoloViews work much the same way, allowing you to add [boxes](http://holoviews.org/reference/streams/bokeh/BoxEdit.html#bokeh-gallery-boxedit),\n",
"[freehand curves](http://holoviews.org/reference/streams/bokeh/FreehandDraw.html#bokeh-gallery-freehanddraw),\n",
"or [polygons](http://holoviews.org/reference/streams/bokeh/PolyDraw.html#bokeh-gallery-polydraw) ([edited separately](http://holoviews.org/reference/streams/bokeh/PolyEdit.html#bokeh-gallery-polyedit))."
]
},
{
Expand All @@ -98,6 +141,8 @@
"source": [
"## HoloViews Point Annotator\n",
"\n",
"What if you also want to associate some numeric, textual, or category label or value with each new data item or datapoint you add? You can do that manually using the drawing tools by adding a separate table and linking it to your element (e.g. Points). To make that easy, HoloViews provides \"Annotator\" objects that package the element, drawing tool, and editable table together into a single object.\n",
"\n",
"A HoloViews (or GeoViews) annotator lets you add, change, or add information to data in a Bokeh plot, then get the data back into Python easily. Here, let's make an annotator for the points, then overlay the annotated points on the map tiles like we had before:"
]
},
Expand All @@ -108,18 +153,16 @@
"outputs": [],
"source": [
"points_annotator = hv.annotate.instance()\n",
"hv.annotate.compose(tiles, points_annotator(points, annotations=dict(Size=int, Type=str)))"
"points_annotator(tiles * points, annotations=dict(Size=int, Type=str))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You'll see that there is now a table of coordinates and also that there is now a [PointDraw tool](../reference/streams/bokeh/PointDraw.ipynb) in the toolbar:<img src=\"https://bokeh.pydata.org/en/latest/_images/PointDraw.png\">\n",
"You'll see that there is now a table of coordinates along side your plot, and that you still have a drawing tool in the toolbar. As you click on an existing point, you should be able to click and drag to see the location update in the table. Whether you click on the table or the points, the same object should be selected in each, so that you can see how the graphical and tabular representations relate.\n",
"\n",
"Once you select that tool, you should be able to click and drag any of the existing points and see the location update in the table. Whether you click on the table or the points, the same object should be selected in each, so that you can see how the graphical and tabular representations relate.\n",
"\n",
"The PointDraw tool also allows us to add completely new points; once the tool is selected, just click on the plot above in locations not already containing a point and you can see a new point and a new table row appear ready for editing. You can also delete points by selecting them in the plot or the table then moving back to the plot (if needed) and pressing Backspace or Delete (depending on operating system).\n",
"When you add new points, a new table row should appear ready for editing. For deleting points, you can select them either on the plot or as a row in the table, but the mouse pointer needs to be in the plot area for Backspace/Delete to take effect.\n",
"\n",
"Whether for existing or newly added points, you can use the table to edit the latitude and longitude values numerically or add an optional \"Size\" estimate or \"Type\" description for each point.\n",
"\n",
Expand Down Expand Up @@ -187,9 +230,8 @@
"rectangles = gv.Rectangles([(0, 0, 3, 3), (12, 12, 15, 15)]).opts(fill_alpha=0.2)\n",
"box_annotator = hv.annotate.instance()\n",
"labels = gv.tile_sources.StamenLabels()\n",
"layout = box_annotator(rectangles, name=\"Rectangles\")\n",
"\n",
"hv.annotate.compose(tiles, layout, labels)"
"box_annotator(tiles * rectangles * labels, name=\"Rectangles\")"
]
},
{
Expand Down Expand Up @@ -243,9 +285,7 @@
" [55.864370, 55.863135, 55.861888, 55.862793, 55.864370])])\n",
"\n",
"path_annotator = hv.annotate.instance()\n",
"layout = path_annotator(path, annotations=['Label'], vertex_annotations=['Value'])\n",
"\n",
"hv.annotate.compose(tiles, layout)"
"path_annotator(tiles * path, annotations=['Label'], vertex_annotations=['Value'])"
]
},
{
Expand Down Expand Up @@ -295,7 +335,7 @@
"source": [
"# Integrating Annotators into your workflows\n",
"\n",
"As you can see above, it's fairly straightforward to build an annotator to collect a specific type of data. To collect data at a large scale, you'll want to focus on usability, which will often mean creating a special-purpose app to collect data across multiple images, multiple datasets, by multiple raters, etc. Doing so is beyond the scope of this introduction, but can be straightforward using the separate [Panel](https://panel.holoviz.org) library for building apps, also based on Bokeh and having full support for HoloViews. The annotator objects can be included directly in a Panel layout and connected to other Panel objects for seamless updating and integration into a larger workflow.\n",
"As you can see above, it's fairly straightforward to use a drawing tool or build an annotator to collect a specific type of data. To collect data at a large scale, you'll want to focus on usability, which will often mean creating a special-purpose app to collect data across multiple images, multiple datasets, by multiple raters, etc. Doing so is beyond the scope of this introduction, but can be straightforward using the separate [Panel](https://panel.holoviz.org) library for building apps, which is also based on Bokeh and has full support for HoloViews. Annotator objects can be included directly in a Panel layout and connected to other Panel objects for seamless updating and integration into a larger workflow.\n",
"\n",
"For more details, see:\n",
"\n",
Expand Down