Query params to filter/promote multiple series instances

[ladeirarodolfo]

✋ Many people requests features. Tell us why yours is important to the
community. How does it add value? Why this feature?

Is your request very specific to your needs? Consider
contributing it yourself! Or reach
out to a community member that offers
consulting services.

Request

What feature or change would you like to see made?
Today, at viewer page, there is the possibility to filter or promote series based on seriesInstanceUID query param. But this works only for only one series UIDs.
Would be useful if user could filter/promote one or any.
...

Why should we prioritize this feature?

...

Acceptance criteria

• User must be capable to filter/promote series based on query param seriesInstanceUID
• If many seriesInstanceUID it must be separated by comma. I.e
  viewer/1.3.6.1.4.1.25403.345050719074.3824.20170126085406.1/?seriesInstanceUID=1.3.6.1.4.1.25403.345050719074.3824.20170126085406.2,2.25.374532628115939034791691984300357809891
• If one or many seriesInstanceUID were not filtered/promoted a user message must pop up

Technical details

• App can filter or promote series for a given series into query param
• To control between filtering or promoting there is a flag filterQueryParam on app config file. False means promote is ON and true means filter is ON.
• It will filter/promote based on url query param named as seriesInstanceUID and only if there is a given series into that study, otherwise a message will be prompt to user.

PR

There is already a PR going on which might help for this possible new feature
#1533

[fedorov]

I was testing this feature with this URL:

https://idc-sandbox-000.firebaseapp.com/projects/canceridc-data/locations/us/datasets/idc/dicomStores/idc/study/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/?seriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150

I still see all of the series in the study loaded (I tried both upper and lower case for the parameter).

Did I misunderstand the syntax, or is it not yet implemented?

[swederik]

Since app-config has 'filterQueryParam' set to false, it should be 'promoting' those series to the top of the side panel list. @JamesAPetts may have broken that with his sorting changes. I get a 403 permission denied for that link so I can't see exactly what's happening at the moment though. Could you add me and James to the list?

[JamesAPetts]

I just checked and this is still working correctly on master after the series sorting changes.

So yes you just need to change the app config filterQueryParam to true, if you want the app to filter instead of just promoting a series to the top.

[pieper]

I changed filterQueryParam to true for the idc sandbox and it is deploying now. Should be done in a few minutes.

[fedorov]

Great, it works with one series, but if I try to also include SEG series UID, I get error:

https://idc-sandbox-000.firebaseapp.com/projects/canceridc-data/locations/us/datasets/idc/dicomStores/idc/study/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/?seriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150,1.2.276.0.7230010.3.1.3.0.3098.1553287530.406853

I loaded the complete study using OHIF download functionality, and copy-pasted SeriesInstanceUID from the DICOM dump.

[JamesAPetts]

Aha, sorry I missed that bit, was just linked to the last column.

I'll check if I can reproduce the issue.

[JamesAPetts]

https://idc-sandbox-000.firebaseapp.com/projects/canceridc-data/locations/us/datasets/idc/dicomStores/idc/study/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/?seriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150,1.2.276.0.7230010.3.1.3.0.3098.1553287530.406853

Erik and I still do not have access to the idc sandbox btw, we've been getting 403s as Erik described above.

However, I'm using a proxy and checking the same study, and can reproduce your issue. For either of the two series, using one of them works, and with both it doesn't.

This is interesting as this works fine on server.dcmjs.org, incorrect code logic is probably being applied to the google adapter. I'll investigate.

[pieper]

I don't have access either. I emailed Bill L to see if he can change that. I guess this is the dicom store he will use behind the proxy but it would be logical if we could access it directly.

[JamesAPetts]

I've played around for an hour, dug through and can't find anything too strange on the client side, but it appears dicomweb-client is falling back to no queries when requests are made to the server.

This is the first call being made, which hits a 404:

https://proxy-dot-idc-dev.appspot.com/v1beta1/projects/idc-dev-etl/locations/us/datasets/pre-mvp-temp/dicomStores/cross-collection-temp/dicomWeb/studies/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/series?SeriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150%2C1.2.276.0.7230010.3.1.3.0.3098.1553287530.406853

Then the following call with no query params is made as backup, which suceeds:

https://proxy-dot-idc-dev.appspot.com/v1beta1/projects/idc-dev-etl/locations/us/datasets/pre-mvp-temp/dicomStores/cross-collection-temp/dicomWeb/studies/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/series

And then the app compares the query with the result and you get your error.

It seems the backend does not accept request correctly, in a more human readable way:

https://proxy-dot-idc-dev.appspot.com/v1beta1/projects/idc-dev-etl/locations/us/datasets/pre-mvp-temp/dicomStores/cross-collection-temp/dicomWeb/studies/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/series?SeriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150

returns a result and:

https://proxy-dot-idc-dev.appspot.com/v1beta1/projects/idc-dev-etl/locations/us/datasets/pre-mvp-temp/dicomStores/cross-collection-temp/dicomWeb/studies/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/series?SeriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150,1.2.276.0.7230010.3.1.3.0.3098.1553287530.406853

does not.

It appears that the proxy cannot deal with multiple series filters?

[fedorov]

It appears that the proxy cannot deal with multiple series filters?

Adding @wlongabaugh

I don't have access either. I emailed Bill L to see if he can change that. I guess this is the dicom store he will use behind the proxy but it would be logical if we could access it directly.

While waiting for Bill to respond, the same study is also available under mvp wave0 here: https://idc-sandbox-000.firebaseapp.com/projects/idc-dev-etl/locations/us-central1/datasets/idc_tcia_mvp_wave0/dicomStores/idc_tcia/study/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/?seriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150,1.2.276.0.7230010.3.1.3.0.3098.1553287530.406853

[JamesAPetts]

I also get a 403 on that one.
FWIW the above fails for me to, still get a 403, I think our permissions have been removed entirely.

I'm hiting the same data using https://proxy-dot-idc-dev.appspot.com/v1beta1 as my healthcareApiEndpoint using the google cloud adapter. Which led me to the investigation above, its an open proxy.

But its difficult to say if which proxy is the issue, as I can't test the above endpoints through the IDC sandbox at the current time.

[fedorov]

I think our permissions have been removed entirely

Indeed - I checked IAM, and all of Radical Imaging IDC team members have been removed entirely under idc-dev-etl, although you are all still listed under idc-sandbox-000. @wlongabaugh can you explain this?

[wlongabaugh]

I have not removed anybody from idc-dev-etl. Radical Imaging team has pretty much only been on the sandbox IIRC. Check the project spreadsheet in private team folder "GCP project management" to see the complete history of all my permission changes.

[wlongabaugh]

I will look at the proxy logs. It should be gluing all parameters on to send to Google. I note that the URL that gives the blue box that Query parameters were not totally applied has a URL containing ".../?seriesInstanceUID=". Is that "/" really supposed to be there?

[fedorov]

@JamesAPetts Bill added you and Erik to idc-dev-etl, so hopefully you should be able to access this URL now: https://idc-sandbox-000.firebaseapp.com/projects/idc-dev-etl/locations/us-central1/datasets/idc_tcia_mvp_wave0/dicomStores/idc_tcia/study/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/?seriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150,1.2.276.0.7230010.3.1.3.0.3098.1553287530.406853

[pieper]

Perhaps it belongs in a different ticket, but for me, the info dialog fades away far to fast for me to understand what it's trying to tell me.

[wlongabaugh]

The problem seems to exist with Google, not the proxy. Going directly to Google provides the exact same results as the proxy:

Direct to GCHC:

curl -H "Authorization: Bearer OAUTHTOKEN" "https://healthcare.googleapis.com//v1beta1/projects/idc-dev-etl/locations/us/datasets/pre-mvp-temp/dicomStores/cross-collection-temp/dicomWeb/studies/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/series?SeriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150%2C1.2.276.0.7230010.3.1.3.0.3098.1553287530.406853"

[]

curl -H "Authorization: Bearer OAUTHTOKEN" "https://healthcare.googleapis.com//v1beta1/projects/idc-dev-etl/locations/us/datasets/pre-mvp-temp/dicomStores/cross-collection-temp/dicomWeb/studies/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/series?SeriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150"

[{"00080005":{"vr":"CS","Value":["ISO_IR 100"]},"00080060":{"vr":"CS","Value":["CT"]},"0020000E":{"vr":"UI","Value":["1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150"]},"00400244":{"vr":"DA","Value":["20000101"]}}]

Via the proxy:

https://proxy-dot-idc-dev.appspot.com/v1beta1/projects/idc-dev-etl/locations/us/datasets/pre-mvp-temp/dicomStores/cross-collection-temp/dicomWeb/studies/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/series?SeriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150%2C1.2.276.0.7230010.3.1.3.0.3098.1553287530.406853

[]

https://proxy-dot-idc-dev.appspot.com/v1beta1/projects/idc-dev-etl/locations/us/datasets/pre-mvp-temp/dicomStores/cross-collection-temp/dicomWeb/studies/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/series?SeriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150

[{"00080005":{"vr":"CS","Value":["ISO_IR 100"]},"00080060":{"vr":"CS","Value":["CT"]},"0020000E":{"vr":"UI","Value":["1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150"]},"00400244":{"vr":"DA","Value":["20000101"]}}]

[wlongabaugh]

The proxy just takes the entire string after the ? and glues it on to what is sent to Google.

And note I get my OAuth token via "gcloud auth application-default print-access-token"

[fedorov]

I would just wait to hear from James now that he has access.

[JamesAPetts]

I do have access now, thanks.

If I'm interpretting Googles DICOM Conformance Statement correctly, it appears google just doesn't support this:

https://cloud.google.com/healthcare/docs/dicom

Only single value, exact matching is supported, except for StudyDate which supports range queries and PatientName which supports fuzzy matching.

With this knowledge in mind, if enableGoogleCloudAdapter is true in the appconfig, we can get around this by making multiple requests with one SeriesInstanceUID query param in each, and agregating them.

[JamesAPetts]

Built a fix for when using google cloud, PR inc.

[wlongabaugh]

Unfortunately the IDC viewer does not set enableGoogleCloudAdapter to true, since we have the proxy that does not use the Google project/store selection mechanism. The app-config.js file we use is here: https://github.com/ImagingDataCommons/IDC-Viewer-Support/blob/master/static_files/app-config-template.js

[JamesAPetts]

Interesting, not a problem, I suppose a bunch of platfroms could not support this. I'll add another tag to the config, and check if enableGoogleCloudAdapter (as we know google has this issue) or that new field are true.

[JamesAPetts]

I've fixed this here: #2035

I'll close this unless we encounter further issues.

[fedorov]

@JamesAPetts I tried the same URL as above:

https://idc-sandbox-000.firebaseapp.com/projects/idc-dev-etl/locations/us-central1/datasets/idc_tcia_mvp_wave0/dicomStores/idc_tcia/study/1.3.6.1.4.1.14519.5.2.1.6279.6001.131383203689189807643685075952/?seriesInstanceUID=1.3.6.1.4.1.14519.5.2.1.6279.6001.214252223927572015414741039150,1.2.276.0.7230010.3.1.3.0.3098.1553287530.406853

and I get the same error as before

This is on Sandbox, which @pieper updated with the latest and greatest stuff an hour ago. Am I missing something?

[pieper]

I checked and the sandbox does have splitQueryParameterCalls: true, and it is the latest ohif master. Hard to parse the discussion here but I thought that was what's needed to fix this.

[fedorov]

There's also this comment from @JamesAPetts in #2037:

Add example config for splitting up multiple query parameters into multiple calls.

[JamesAPetts]

This now works on the IDC fork.