Provide Optional Static Request Format

[eironside]

Is your feature request related to a problem? Please describe.

Some consuming applications are unable to dynamically update the request URL. For those systems it would be useful to provide a static request URL that doesn't need to change on every request.

In several requests in the API, a timestamp is required (either a start time, an end time, or both). But some applications are not able to dynamically update their URL request to accomodate this feature. It would be good to have an alternative 'static' URL format that applications can use to request data.

Describe the solution you'd like

Ideally, I would like to see 2 things:

1. Not specifying the timestamps makes some sort of assumption (for example end time is always the current reqeust time if it is not provided in the request.
2. Allow data requests to include a time span (for example 3600 seconds) that is applied to the request.

An example of these two requests combined would be the 'Realtime Data' requst. Instead of having to supply both a start time and an end time before anything will be returned, please provide an option where endtime defaults to the request time (current time) if it is not in the request and a new URL parameter 'duration=3600' [in seconds] specifies the start time to be Now()-1 hour. Thus, the request could be a static request asking for the last hour's worth of data.

Is this a breaking change

• No, not breaking.

Impacted Spec

For which spec is this feature being requested?

• provider

Describe alternatives you've considered

In my case, a second python script that is executed hourly is needed to update the primary application's configuration. This is not ideal and adds another layer of complexity to our solution.

Additional context

Our application is a 3rd party commercially available application that cannot be modified. The application is a black box and does not allow for URLs to be dynamically adjusted for timestamp requests. Having to implement another layer of configuration management to accomplish something that seems rather trivial to implement on the other side of the API is not ideal.

[thekaveman]

Hello @eironside - for some more context, see the discussions over in #268 and a related implementation in #357. The general consensus was that allowing some type of parameter inference is a bad idea, and we wanted to make time-bounding parameters explicit to reduce ambiguity as much as possible.

Other stakeholders please weigh in here especially if that thinking has evolved @johnpena @bhandzo @babldev @hunterowens @hyperknot @ascherkus

[hyperknot]

As a software developer working with both a client and server side APIs, I believe this limitation is unrealistic in your external "3rd party commercially available application". Using query parameters was included in HTTP 0.9, in 1991, so we are talking about an almost 30 year old standard.

In computers and software everything changes insanely fast, almost nothing is as mature and reliable as query strings in HTTP specs.

For your specific problem, I recommend adding a proxy endpoint on your own server to achieve this.

Also, supplying MDS authorization credentials to client apps is not a good idea, it's trivial to steal them out of requests, and then all the private data could be downloaded from the provider! I believe MDS endpoints should be used only for server-server connections, to make sure the credentials cannot be stolen.

[ascherkus]

@eironside I'd like to understand a bit more about the constraints of your application, as well as the specific data your application requires.

My understanding is (please correct / clarify as needed):

1. The URL in your client application is hardcoded / fixed and is making MDS HTTP requests directly to provider endpoints
2. You're interested in only having the last hour's worth of data

For (1), does this imply there is no programmability / scripting available in this client? If that's the case, can an HTTP proxy be used similar to what @hyperknot mentioned? E.g., you could prototype replacing your python script with a python HTTP server that maps the request from your black box into the data your application requires (not saying this is the most secure / ideal setup, but it is something you can explore).

For (2), could you expand on the type of data (trips, status changes) as well as time ranges you're interested in? For example if it's just displaying recent data of the fleet, I'm curious whether something like #376 would suit your needs.

[eironside]

@ascherkus Our COTS software allows users to configure an end point URL they would like to poll for data. This input is designed to be flexible and can poll any end point for any data. Because of the vast array of different APIs we may encounter, it is not possible to decide exactly how to handle changing URL prameters such as time windows.

For (1) you are correct. There is no ability to script or dynamically calculate parts of the URL request OOTB. We do provide an API that allows users to run periodic scripts to update the URL, that is how our customers work around these issues now. A proxy would also work in this case to inject the correct time window parameters into the request. So we do have a workaround.

But most of our end users are not developers who are "script" or "proxy" savvy. They are most likely not part of an IT department either. So creating and deploying such workarouds are not easy for them. I'm not arguing they can't or shouldn't be able to do it, but I am asking for an alternative [enhancement] that would be significantly easier for them.

With respect to @hyperknot 's comments about URL parameters, the issue is not whether we allow parameters (we certainly do). The issue is that we do not know anything about the APIs our customers will encounter. So while making a simple modification to our software to handle MDS's time window parameter mechanisms seems easy, there is no guarantee those changes will work on the next API we encounter. Providing mechanisms for our users to easily update URL parameters that we don't know anything about is a rabbit hole we are not traversing yet.

In response to @thekaveman 's statement on 'not allowing parameter inference' I'm good with that. Avoiding ambiguity is usually a good thing and forcing time bounds makes sense (if an end bound is not given, but the start time is in 1970, inferring they want all data between 1970 and 2020 is probably a bad thing). So in this case, instead of inferring the start/end timestamsp, just provide an alternative [enhancement to the api] that supplies data for the last X seconds (users must supply either start/end or range parameters in their request).

For (2) our users will be primarily interested in 'real-time' data updates and will be polling for changes to that data relatively frequently. I believe the request in #376 is exactly the type of thing our users would want.

[thekaveman]

Thanks for the additional context @eironside.

On the biweekly Provider call today we confirmed that the /vehicles endpoint proposed in #376 will be going out with our next release 0.4.1 currently scheduled for 01 March 2020. Can we tag this issue for close once that PR is released?