Use with AUVs

[factoidforrest]

How adapted is Vizanti to use with AUV mission planning? I see some limited functionality for depth control, however it seems fair to say that most of the path planning is adapted to 2d missions. As you know, AUVs operate in 3 dimensions, and do much of their mission out of contact with the base station. Even so, there does exist a need for surface manuevering control, as well as mission planning, and this project looks like it has that part totally figured out.

I'm a senior javascript dev, so I'm happy to help make adaptions for my project, I was just wondering what work has already been done and how major the project developers would judge the work to be of extending vizanti for AUV use. Vizanti has clearly been written with extensibility in mind, so I judge it is at least possible. Id also be open to hearing about other projects that better meet the mission, if you know of any.

Really all i would say is necessary as a bare minimum for AUV use would be:

• Waypoint planning that includes desired depth on each waypoint and in mission grids

There are also plenty of nice-to-haves for AUVs

• Nautical chart display (it looks like vizanti is already super dynamic with what maps it can render so this may already be present)
• 3d map visualization (This is probably a massive change for vizanti and maybe not worth the effort)

Anyway this is an amazing project and looks very well thought out, I look forward to trying it.

[MoffKalast]

Hey, thanks for the interest! I think you've come to the right place 😃

I've actually been using it with the BlueROV2 in DAVE/Gazebo quite a bit in recent months, and the altimeter/depth widget was added specifically for sending a target depth to the AUV depth controller as a sort of first step towards supporting some sort of Z axis control.

Waypoint planning that includes desired depth on each waypoint and in mission grids

Yeah that would be a great addition, but I've never quite had a super clear idea on what would be the best way to do it UX-wise. So far my best idea was to add a sort of graph box to the modal window that would show a "sideways" profile map that has:

• Z on the vertical axis (probably a dropdown to select +,-, or centered on zero with both)
• total path distance on the horizontal axis (segmented into numbered waypoints, with spacing between them adjusted so it matches the weighed distances between them, scaled to total box width)

Then one could just pick and slide each point up and down to adjust the Z values. There's some caveat in terms of what max/min Z we display on the graph and what happens when points bunch too close together that needs some further thought. Maybe some sort of scrolling for the box I guess?

In any case, it's definitely planned and would be super useful for both AUVs and aircraft but I haven't really properly had the time or urgent need to really dive (ha) into it. If you want to take a stab at it as a PR in the meantime, it would be most welcome. Contributing.md has a short rundown of the project layout, and I can give you some pointers if you have any questions.

do much of their mission out of contact with the base station

At least for surface vessels the current setup actually works pretty well. Connect to the boat wifi, load up the map, set and publish waypoints, the onboard planner takes the route and does its thing outside wifi range, returns back (hopefully), wifi reconnects and I can teleop the boat back for collection and stop bag recording, all from just a smartphone.

What I've seen people do for AUVs is to give them a short tether with an antenna buoy at the end which gives you some leeway during testing if say, your nav stack crashes when it's submerged...

Nautical chart display

That would be great to have, and in theory if there was a tile server that complied with the web standard for map tiles it would be plug and play with just the URL change. I've tried this in the past with a few offerings that sort of had some routes mapped but it didn't really work properly for some reason.

A funny workaround I've been using for displaying a small local map was to publish a TF link at a known origin for it, and then set a robotmodel widget image to display there. Having a more proper way of rendering custom images on specific locations would be a great addition for many use cases regardless (maybe also even a browser-uploaded image overlay that you can scale and rotate in the fixed frame). Again haven't really gotten around to it yet.

3d map visualization (This is probably a massive change for vizanti and maybe not worth the effort)

Ha yes, well it would be quite a rework and I'm still not entirely convinced that it would be all that usable on mobile. I think porting it to something that actually uses better GPU acceleration than canvas with a more workable abstraction than WebGL like Pixi.js or Three.js would be a good move eventually once there's better WebGPU support, and I guess adding some kind of 3D might be an option at that point. TBD

Id also be open to hearing about other projects that better meet the mission, if you know of any.

Afaik most people use MOOS-IvP instead of ROS for AUVs and that comes with pMarineViewer as the mission planner, but given that it's ~20 years old it's not exactly the most user friendly thing. There's also Ardupilot MP but I haven't heard of anyone using it underwater. Both of these are also top down 2D, for 3D I think only Foxlglove Studio really has full 3D rendering, but afaik it doesn't really have any mission planning capabilities and it was mainly designed around self driving cars.

[factoidforrest]

Awesome, thanks for the great and thorough response. I will take a look at the code and try to cook up a PR.

UX: For display, I was thinking it would be fairly straightforward to display a small number by waypoints when z != 0. As for creation, a text box for "altitude" which controls the Z of any waypoints created would probably do it, as an MVP.

Anyway, for now, I need to just get the tool running. I'll take a look at your code and the "contributing" document and see what I can figure out in the way of devex for just running the tool locally on my machine. Thanks for your encouragement! I really like what you've done here and I agree that I'm in the right place.

[MoffKalast]

@factoidforrest So I've been thinking about this problem from the UX perspective a bit, and I might have found an interesting approach:

This is the waypoints widget as idle (with a new a configurable distance safety zone shadow):

Then when you click it once, it goes into the current X and Y setting mode, letting you add/remove points and move them around:

Click it again though and it would go into Z setting mode, where you click on each point and drag up and down to set the Z value on a slider (mostly a mockup so far):

Then if you let go and drag again it moves the scale to that value, so you can fine tune:

In theory having it logarithmic like this would let you make a fairly exact adjustment with just two drags (especially handy on touchscreens), but I've yet to test how it would work in practice.

Then as you add more points in XY mode it would set the Z value of the new point depending on where you add it. If a new point gets appended to the end, it copies it from the last point, if it gets inserted between two points, it takes the percentage of distance it is to each point and calculates a weighed average Z, so you can e.g. split up a long climb without any Z changes.

It would still be good to have a way to set the exact value for all three XYZ coords regardless (if we're adding that anyway) and applying a Z value to all points. Maybe single click/tap to input value and click and hold/drag to show the slider?

There is a sort of secondary "polygon" mode I'm vaguely planning (again a mockup), that would use the points to define an area instead and the path would be generated by running parallel lines and finding intersections with that polygon, giving you a survey pattern of any area shape:

Not sure if this would make sense to add as a different widget altogether or just a different mode you can set, but it it's something to keep in mind in terms of compatibility (how do we let the user set the Z value of generated points if they change entirely every time the polygon boundaries are moved?)

Another thing that would be pretty cool is indicating the depth with gradients (since canvas supports this easily):

Basically having a light and dark color, then mapping that colour range to max and min Z, showing in this mockup case a surface waypoint, dive down, and then finally surface again for an AUV.

Or for airplanes: liftoff, cruise, landing:

I've set up a branch with some of this implemented: #100

[MoffKalast]

Nautical chart display (it looks like vizanti is already super dynamic with what maps it can render so this may already be present)

So I've been looking at openseamap for an unrelated thing today and noticed they list some tile servers for nautical overlay tiles, out of which the https://t1.openseamap.org/seamark/{z}/{x}/{y}.png seems to actually work with the current rendering setup:

The obvious caveat is that it's an empty overlay so it needs the OSM tiles too. I think it shouldn't be too difficult to modify it so if fetches the base + an optional overlay layer, combines both images and saves that. Something for the TODO list.

There's also a "sport" tile server, but those seem to be entirely empty.