UDF debugging and logging

[soxofaan]

Debugging/logging in and around UDF usage is not straightforward at the moment.

some things to tackle:

• functionality in client to allow logging from UDF code (e.g. function for (debug) logging from UDFs openeo-python-client#302)
• extra logging on driver side where UDFs are invoked
• make sure logging from UDF is visible in user logs

[bossie]

Invoking this UDF will add inspect log entries:

# Load initial data cube.
s2_cube = connection.load_collection(
    "SENTINEL2_L2A",
    spatial_extent={"west": 4.00, "south": 51.04, "east": 4.10, "north": 51.1},
    temporal_extent=["2022-03-01", "2022-03-31"],
    bands=["B02", "B03", "B04"]
)

# UDF code (as inline string)
udf_code = """
from openeo.udf import XarrayDataCube
from openeo.udf.debug import inspect

def apply_datacube(cube: XarrayDataCube, context: dict) -> XarrayDataCube:
    array = cube.get_array()
    inspect(data=len(array), message="len(array)", code="test_udf_logging", level="warning")
    array.values = 0.0001 * array.values
    return cube
"""

# Create UDF helper object encapsulating the UDF code.
udf = openeo.UDF(code=udf_code, runtime="Python", data={"from_parameter": "x"})

# Pass UDF object as child process to `apply`.
rescaled = s2_cube.apply(process=udf)

rescaled.execute_batch(out_format="GTiff")

Their log level, message, data and code are properly returned by the batch jobs logs API: