Support rendering a map with GPS tracks with a video

[mathstuf]

The fewer issues I have to read, the more new features I will have time to implement, so I ask that you please try these things first

• Check out all the menus and keyboard shortcuts. Maybe the feature already exists?
• Try with the newest version from GitHub
• Read the documentation and FAQ, Known issues, Troubleshooting to see if the feature already exists
• See if someone already suggested this under Issues or Discussions

Description

Data from devices such as dashcams also have GPS tracks associated with them. It'd be nice to be able to associate the video editing while trimming the GPS tracks with them at the same time. Might be related to #1027 if I could get the GPS data into a metadata stream.

[mifi]

that's a cool feature! I've added experimental map (can be accessed under the Tracks screen). it's very basic and obviously it should be improved so that it will follow the player's time. It only supports DJI embedded subtitle (inside .mp4 file) because that's what I had to test. Not sure which other formats to support, and how to best support all of them.

[mathstuf]

GPX is the other obvious one. My dashcam makes some CSV-based format, but I wrote a script to convert it to GPX. The main issue would be aligning the timestamps with the frames (and knowing when the video starts…embedded in the filename at least for me).

[mifi]

Is there any software (preferrably javascript) that can convert other formats into GPX? Like DJI subtitle, Gopro GPS meta, iPhone MOV gps, etc (embedded into video files as a separate track).

[mathstuf]

https://www.npmjs.com/package/convert-google-directions seems like a project to chase dependencies for how it does so.

[mifi]

that project seems to parse Google maps URLs into coordinates

[mathstuf]

Sorry, should have been clearer; I linked it for the set of formats it targets (and the libraries it uses to do so) more than its code itself.

[mifi]

Oh actually I think exiftool can extract GPS from most files:

https://www.youtube.com/watch?v=Uje5si23lBk

I tested it on an iPhone video and it is able to extract gps locations + timestamps!

exiftool -ee IMG_5288.MOV

the problem is that exiftool is perl. see also #1027

[EgorKin]

May be this can be useful for you - mp4 to GPX

[howff]

This is really interesting, thank you :-)

I believe the check is done here:

lossless-cut/src/renderer/src/util/streams.ts

Line 258 in 87bb9df

export const isGpsStream = <T extends Pick<FFprobeStream, 'codec_type' | 'tags'>>(stream: T) => stream.codec_type === 'subtitle' && stream.tags?.['handler_name'] === '\u0010DJI.Subtitle';

looking for DJI.subtitle
but my track has handler_name: '\x10DJI.Meta'
so can that be supported also?
It's from a Mini 2.
Thanks!

[mifi]

i could add it but i'm not sure if the format of the data in a '\x10DJI.Meta' stream is the same as for '\u0010DJI.Subtitle'?

[howff]

Sorry, I was using the wrongly! When I try to plot the DJI.Subtitle track it says no GPS coordinates found.
Here's the dump where you can see the GPS coordinates are present:

00000000  00 00 00 1c 66 74 79 70  69 73 6f 6d 00 00 02 00  |....ftypisom....|
00000010  69 73 6f 6d 69 73 6f 32  6d 70 34 31 00 00 00 08  |isomiso2mp41....|
00000020  66 72 65 65 00 01 48 0a  6d 64 61 74 00 fe 46 2f  |free..H.mdat..F/|
00000030  32 2e 38 2c 20 53 53 20  36 37 38 2e 35 32 2c 20  |2.8, SS 678.52, |
00000040  49 53 4f 20 31 30 30 2c  20 45 56 20 30 2c 20 44  |ISO 100, EV 0, D|
00000050  5a 4f 4f 4d 20 31 2e 30  30 30 2c 20 47 50 53 20  |ZOOM 1.000, GPS |
00000060  28 2d 33 2e 39 31 33 30  2c 20 35 36 2e 35 30 31  |(-3.9130, 56.501|
00000070  39 2c 20 32 36 29 2c 20  44 20 30 2e 35 37 6d 2c  |9, 26), D 0.57m,|
00000080  20 48 20 31 30 32 2e 34  30 6d 2c 20 48 2e 53 20  | H 102.40m, H.S |
00000090  30 2e 30 30 6d 2f 73 2c  20 56 2e 53 20 2d 30 2e  |0.00m/s, V.S -0.|
000000a0  30 30 6d 2f 73 20 0a 00  00 00 00 00 00 00 00 00  |00m/s ..........|
000000b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000120  00 00 00 00 00 00 00 00  00 00 00 00 00 fe 46 2f  |..............F/|
00000130  32 2e 38 2c 20 53 53 20  36 37 38 2e 35 32 2c 20  |2.8, SS 678.52, |
00000140  49 53 4f 20 31 30 30 2c  20 45 56 20 30 2c 20 44  |ISO 100, EV 0, D|
00000150  5a 4f 4f 4d 20 31 2e 30  30 30 2c 20 47 50 53 20  |ZOOM 1.000, GPS |
00000160  28 2d 33 2e 39 31 33 30  2c 20 35 36 2e 35 30 31  |(-3.9130, 56.501|
00000170  39 2c 20 32 36 29 2c 20  44 20 30 2e 36 31 6d 2c  |9, 26), D 0.61m,|
00000180  20 48 20 31 30 32 2e 35  30 6d 2c 20 48 2e 53 20  | H 102.50m, H.S |
00000190  30 2e 30 30 6d 2f 73 2c  20 56 2e 53 20 2d 30 2e  |0.00m/s, V.S -0.|
000001a0  30 30 6d 2f 73 20 0a 00  00 00 00 00 00 00 00 00  |00m/s ..........|
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

[mifi]

maybe you can run this command on your file and share the result: (replace STREAM_INDEX with the index of the subtitle stream, e.g. 0, 1 or 2 etc)

ffmpeg -hide_banner -i VIDEO.mp4 -map 0:STREAM_INDEX -f srt -

[howff]

1
00:00:00,000 --> 00:00:01,000
F/2.8, SS 678.52, ISO 100, EV 0, DZOOM 1.000, GPS (-3.9130, 56.5019, 26), D 0.57m, H 102.40m, H.S 0.00m/s, V.S -0.00m/s ^M


2
00:00:01,000 --> 00:00:02,000
F/2.8, SS 678.52, ISO 100, EV 0, DZOOM 1.000, GPS (-3.9130, 56.5019, 26), D 0.61m, H 102.50m, H.S 0.00m/s, V.S -0.00m/s ^M

[mifi]

ok thanks.it seems that your drone has an additional field DZOOM which mine does not. i'll add support for it

[wolrah]

I would love it if support could be added for GoPro GPS data. I currently use https://github.com/time4tea/gopro-dashboard-overlay on the individual segment files and then cut/merge in LosslessCut, so if LC had this feature built in it'd save me a fair bit of effort every time I record a bike ride.

Here are a couple of other projects that work with this data which might be useful for understanding the format:

https://github.com/JuanIrache/gopro-telemetry
https://github.com/progweb/gpx2video

I can provide all sorts of example content from a Hero 5 Black plus a few clips from a Hero 10, and will probably be adding a Hero 13 to the mix soon.