Can't show the same video in multiple space views #7420
Comments
7 tasks
emilk
changed the title
Wumpf
added a commit
that referenced
this issue
Sep 17, 2024
### What * Fixes #7368 * creating new issues for the previously remaining items * Fixes #6532 Had to patch this through to all 3 sdks which naturally takes quite a bit of extra machinery. Decided to go with raw ns here since it's easiest to pass through FFI and is the most versatile (hopefully our `VideoFrameReference` typing will be solved with tags instead of per component enums in the future, making this more equivalent). Changed snippets around. There's now: * a sample with two single frozen frames, showing next to each other. Demonstrating that video frames can be used individually and without any extra utilities * this doesn't quite work yet due to #7420, so this can be regarded a bit of a work in progress * sample using the new frame extraction utility, essentially a "send full video" which we may encapsulate into a higher level utility at some point. It's quite short though in Python, so I'm not too worried about this! Asset drag & drop got updated to also use this utility. All this churn also led me to changing how `re_video` is used a bit, making the entry point more highlevel. Media type got a bit annoying there because we can't afford `re_types` dependencies in the C++ & Rust SDKs. The solution here is to have independent media type parsing for videos in `re_video`. ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7421?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7421?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [x] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide * [x] pass `main` ci - [PR Build Summary](https://build.rerun.io/pr/7421) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
6 tasks
Wumpf
added a commit
that referenced
this issue
Sep 19, 2024
### What * Fixes #7373 * I'm sure there's more to do here and surely more cases to try out whether they display the correct errors, but this PR should handle everything that we detect as an error case, laying the foundation to pass everything through As part of this work it was necessary to make clearer difference between video loading and video decoding. You'll notice changes in re_renderer's video and the video cache accordingly. (there's some loose ends here like "don't create a decoder when we're just inspecting a blob" which will be part of #7420) Generally, decode errors can only occur when showing (!) a video frame, therefore they are shown on screen as part of a the 2D/3D view:  (trying to show frames on native) Note that you can interact with this 2D view just as if error was logged data: https://github.com/user-attachments/assets/ee609074-6fa6-4b59-ba33-ea6026c0f0d0 On the other hand when we can't read the _blob_ itself (potentially without trying to decode it) the errors show up on it like this:  Since this error happens already on loading right now we can't generate frames automatically if you drag&drop in a video like this. But you can still log your frames manually, resulting in this:  In the shown example the blob isn't part of the entity since we're referring to a different entity for the blob. Also, this is actually a native screenshot - the avc parsing error comes _before_ decoding, which is why this comes up prior to the lack of decoder. If we instead have the failing blob on the entity that also has the frame, we get this view:  When there's no error, we show now a little bit of information for blobs:  (blob selected)  (data result with this blob selected) Some shortcomings: * some decode errors like "we can't decode this at all because it's not supported", could be shown was warnings the property grid on blobs. This is quite unfavorable from a program structure perspective, but as user I'd be wondering why not? * dragging in a video format with an unknown codec won't generate frames and therefore no big "can't read this video" which is rather unfortunate. As we progress in separating video container reading & decoding this is going to get better! * blob errors aren't terribly well formatted / a bit ugly * error states in streams panel as suggested on the ticket would be nice, but that's a more general ~rabbit hole~ feature I don't want to tackle in the near term ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7433?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7433?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [x] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide - [PR Build Summary](https://build.rerun.io/pr/7433) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
|
This should also be able to unlock playing back a video in the selection panel (preview selected video) |
|
I don't think that's a good idea universally, at least not the simple solution of just looping a video in there: High resolution videos take a serious amount of processing & memory that we don't want to waste on a small preview. Also if it just keeps playing in there that might be highly distractive. So at least some controls would be needed. |
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We added the ability to reference videos, but our video decoder goes crazy if it has to show frames from different locations at the same time (that's expected the way it's written). Therefore:
I think we can implement that track mechanism easily enough. But if it turns out to be a hassle we can cut back!
The text was updated successfully, but these errors were encountered: