Clip endpoint for MosaicJSON

[dchirst]

Hi!

I am trying to create a custom endpoint for my company's DEM data, which exists in many different COGs. I've been using the MosaicJSON endpoint to combine all these cogs into one cohesive view.

I'd now like to create an endpoint where you can give a bbox and get back a single image clipped to that bbox - much like the clip endpoint for COGs, but for MosaicJSON. Do you have any advice on how to go about it?

Some options include:

1. Merge all smaller COGs into one huge COG that covers the entirety of Great Britain. I haven't tried this because it sounds like a terrible idea for performance, but I don't know enough about COGs to say this with any certainty
2. Create a custom endpoint for titiler that takes a bbox, finds all tiles that overlap with the bbox, clip them individually, then merge them

Thank you in advance - Titiler is an incredible product

[vincentsarago]

ahhhhhh I deleted my original answer 😭 😭 😭

First thanks for the question.

Create a custom endpoint for titiler that takes a bbox, finds all tiles that overlap with the bbox, clip them individually, then merge them

We don't have /crop endpoint because for MosaicJSON it could result in opening a LOT of files and thus the requests might take too much time.

We added a /statistics endpoint in titiler-pgstac because by using pgstac we can reduce the number of files needed to be open (/statistics use a custom Backend().feature method which is close to a part() method you would need)

https://github.com/stac-utils/titiler-pgstac/blob/master/titiler/pgstac/mosaic.py#L321-L384
https://github.com/stac-utils/titiler-pgstac/blob/master/titiler/pgstac/factory.py#L438-L475

Merge all smaller COGs into one huge COG that covers the entirety of Great Britain. I haven't tried this because it sounds like a terrible idea for performance, but I don't know enough about COGs to say this with any certainty

Well to be honest, COGs scale pretty well, we've work with huge COGs before and it's ofter faster than Mosaics. if you don't need to update this COG often, if might be a good solution.

3. Another solution is to use the WMTS driver in GDAL, pointing to your titiler deployment. This is a bit weird because you could have an endpoint that is pointing to it's own deployment.

wmts_dst = "WMTS:http://titiler.xyz/cog/WMTSCapabilities.xml?url=https://noaa-eri-pds.s3.amazonaws.com/2022_Pre_Event/EC2201a_OB_N_RGB/EC2201a_OB_N_C0700000w415145n.tif&bidx=1&bidx=2&bidx=3&tile_format=png"

httpx.get(
    "https://titiler.xyz/cog/crop/-69.99510242702613,41.85313673098173,-69.99038118408833,41.857529724573766.png",
    params={
        "url": wmts_dst,
        "max_size": 512,
    }
)

url: https://titiler.xyz/cog/crop/-69.99510242702613,41.85313673098173,-69.99038118408833,41.857529724573766.png?url=WMTS%3Ahttp%3A%2F%2Ftitiler.xyz%2Fcog%2FWMTSCapabilities.xml%3Furl%3Dhttps%3A%2F%2Fnoaa-eri-pds.s3.amazonaws.com%2F2022_Pre_Event%2FEC2201a_OB_N_RGB%2FEC2201a_OB_N_C0700000w415145n.tif%26bidx%3D1%26bidx%3D2%26bidx%3D3%26tile_format%3Dpng&unscale=false&resampling=nearest&max_size=128&return_mask=true

[vincentsarago]

🥳 with such a BIG COG, the header might be quite big. It might be wise to make sure to set GDAL_INGESTED_BYTES_AT_OPEN environment to a large value to avoid doing to many requests

https://developmentseed.org/titiler/advanced/performance_tuning/#gdal_ingested_bytes_at_open

You can validate this with https://github.com/developmentseed/tilebench

[dvd3v]

As an addition to this discussion; it is also possible to clip a MosaicJSON and return as a Numpy Array using the WMTS driver with the /cog/crop endpoint:

wmts_dst = "WMTS:http://titiler.xyz/mosaicjson/WMTSCapabilities.xml?url=https://some-mosaic.json&tile_format=png"

response = httpx.get(
    "https://titiler.xyz/cog/crop/-69.99510242702613,41.85313673098173,-69.99038118408833,41.857529724573766.npy",
    params={
        "url": wmts_dst
    }
)

arr = np.load(BytesIO(response.content))

However this only makes sense for categorical data, as datasets are rescaled from 0-255. Is there a way to extract the raw values from a MosaicJSON using the WMTS solution?

[vincentsarago]

sadly I don't think the WMTS driver accept tif image 😭

[dvd3v]

It accepts the tile_format=tif parameter, but the values are still rescaled. So unfortunately this solution will not work if you need the actual raw values. Still good for clipping MosaicJSONs to PNGs!
The solution for getting the raw values would be using TiTiler-PgSTAC with this custom endpoint?

[vincentsarago]

The solution for getting the raw values would be using TiTiler-PgSTAC with this custom endpoint?

Yes