Issues with batch job logs

[soxofaan]

Note: I'm not sure if I should report this issue under this repo, feel free to move it appropriately.

I had a failed batch job on openeo.eodc.eu and noticed the batch job logs look like this (Jupyter notebook on the left, web editor on the right):

a couple of issues:

• all log items have level INFO, while there are clearly items that should have level ERROR
• There is a stack trace, but apparently each line of the stack trace is a separate log item/message. I think that is a bad format to provide the stack trace to the user, as you lose the inherent line oriented structure and context of stack traces. E.g., now you have log messages with just ^^^^^, which makes no sense. You also lose the indentation, making it hard to parse visually.
• In some messages there is a timestamp, e.g. 2024-09-24 13:03:25,976 - openeo_execut.... Note that there is a dedicated "time" field for openeo batch job log items

[soxofaan]

about the stack traces in the logs, the spec mentions that:

It is NOT RECOMMENDED to add stacktraces to the message.

This was added by Open-EO/openeo-api#457 triggered from Open-EO/openeo-api#455 . However, I'm not sure what the recommended alternative is. @m-mohr ?

FYI: I think in the VITO implementation we just put the exception message in the message, without the full stack trace. We can look up the full stack trace at our end, but the openeo batch job logs don't show that to the user (not in the least because the stack trace is usually way too cryptic for a user to be useful)

[m-mohr]

For the average user the stack traces will just be confusing or at least unhelpful, that's why it's "not recommended". There's not really an alternative right now. If you think they are useful for your users, you can still put them in the message, it's just a recommendation. We never defined an API field for stack traces yet (although there's an issue for it).

[ValentinaHutter]

Thanks for pointing this out, I will move this issue to our internal issue tracking system. Feel free to reopen this issue, in case anything related comes up.