Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filters to Platform Usage report #687

Open
4 tasks done
Tracked by #586
labradford opened this issue Aug 14, 2023 · 9 comments
Open
4 tasks done
Tracked by #586

Add filters to Platform Usage report #687

labradford opened this issue Aug 14, 2023 · 9 comments
Assignees
Labels

Comments

@labradford
Copy link
Collaborator

labradford commented Aug 14, 2023

Story

Platform Usage Reports should be able to receive filter params for the following:

  • Begin_Date* required
  • End_Date* required
  • Metric_Type (Searches_Platform, Total_Item_Requests, Unique_Item_Requests, Unique_Title_Requests)

not currently supported in this implementation because we do not have the historical data to support

  • Platform(?)
  • Access_Method (Regular or TDM - only regular users for now until we know if there are know data miners)

Acceptance Criteria

  • Platform Usage report can be filtered on the above items.

Screenshots / Video

Testing Instructions and Sample Files

Notes

@labradford labradford moved this to Ready for Development in palni-palci Aug 14, 2023
@labradford labradford changed the title Add filters to platform usage report Add filters to Platform Usage report Aug 14, 2023
@summer-cook summer-cook moved this from Ready for Development to In Development in palni-palci Aug 16, 2023
@summer-cook summer-cook self-assigned this Aug 16, 2023
@summer-cook summer-cook moved this from In Development to Code Review in palni-palci Aug 17, 2023
@summer-cook summer-cook moved this from Code Review to Deploy to Staging in palni-palci Aug 21, 2023
@summer-cook summer-cook moved this from Deploy to Staging to SoftServ QA in palni-palci Aug 23, 2023
@labradford
Copy link
Collaborator Author

QA - Platform report is properly filtering

@labradford labradford moved this from SoftServ QA to Pitt QA in palni-palci Aug 23, 2023
@ctgraham
Copy link
Collaborator

ctgraham commented Sep 5, 2023

Pitt QA concerns:

begin_date / end_date limits

When limiting the begin_date and end_date, metrics from outside of the range are still reported:
https://pittir.commons-archive.org/api/sushi/r51/reports/pr?begin_date=2023-06&end_date=2023-07&metric_type=Total_Item_Requests
Observed output:

"Total_Item_Requests":{"2023-05":98,"2023-06":700,"2023-07":740,"2023-08":521}

Expected output is:

"Total_Item_Requests":{"2023-06":700,"2023-07":740}

begin_date after end_date

When the begin_date filter is after the end_date filter, a 3020 Exception is expected:
https://pittir.commons-archive.org/api/sushi/r51/reports/pr?begin_date=2023-04&end_date=2022-08&metric_type=Total_Item_Investigations
Observed output:

{"Report_Header":{"Release":"5.1","Report_ID":"PR","Report_Name":"Platform Report","Created":"2023-09-05T18:59:56Z","Created_By":"pittir.commons-archive.org","Institution_ID":{},"Institution_Name":"pittir.commons-archive.org","Registry_Record":"","Report_Attributes":{"Attributes_To_Show":["Access_Method"]},"Report_Filters":{"Begin_Date":"2023-04-01","End_Date":"2022-08-31","Metric_Type":["Total_Item_Investigations"]}},"Report_Items":{"Platform":"pittir.commons-archive.org","Attribute_Performance":[{"Data_Type":"Platform","Access_Method":"Regular","Performance":{"Searches_Platform":{}}}]}}

Expected output:

{
  "Code": 3020,
  "Message": "Invalid Date Arguments",
  "Data": "end_date cannot be less than begin_date"
}

Are general Exception responses to be considered for another issue? I see that various invalid inputs return JSON with errors, but not formatted as exceptions. E.g.:
https://pittir.commons-archive.org/api/sushi/r51/reports/pr

{"error":"param is missing or the value is empty: begin_date"}

@ctgraham
Copy link
Collaborator

ctgraham commented Sep 5, 2023

Pitt QA concerns

Report filters in the header

See also:
#720 (comment)

Here, the metric_type report filter (or others) should be reflected in the Report_Header.

jeremyf added a commit that referenced this issue Sep 5, 2023
Prior to this commit, when given a YOP we were not providing that value
as part of the response document.

With this commit, when we're provided a YOP, we include that as part of
the "Report_Header > Report_Filters".

Related to:

- #720
- #687
@labradford labradford moved this from Pitt QA to Ready for Development in palni-palci Sep 19, 2023
@jeremyf jeremyf self-assigned this Sep 19, 2023
jeremyf added a commit that referenced this issue Sep 19, 2023
Prior to this commit, the inner logic that built the array of data was
not using the begin and end date filter.

With this commit, those sub-queries are all limiting the queries to
those that are within the given date range.

Related to:

- #687
@jeremyf jeremyf moved this from Ready for Development to Deploy to Staging in palni-palci Sep 20, 2023
@jeremyf jeremyf moved this from Deploy to Staging to SoftServ QA in palni-palci Sep 20, 2023
@alishaevn
Copy link
Contributor

@ctgraham both of your comments above have been addressed. ready for review.

@alishaevn alishaevn moved this from SoftServ QA to Pitt QA in palni-palci Sep 26, 2023
@ctgraham
Copy link
Collaborator

ctgraham commented Sep 26, 2023

Confirming that the date range bounding and report filters look good.

I did try a query placing the month before the year as:
https://pittir.commons-archive.org/api/sushi/r51/reports/pr?begin_date=04-2023&end_date=08-2022&metric_type=Total_Item_Investigations

This returned an Exception 3031, but probably should have been an Exception 3020.

I also tried overspecifying the date as year-month-day:
https://pittir.commons-archive.org/api/sushi/r51/reports/pr?begin_date=2023-05-30&end_date=2023-08-15&metric_type=Total_Item_Investigations

This returned the non-intuitive message of:
"The requested begin_date of 2023-05-30 is before the earliest metric date of 2023-05-27."

A query for start-of-month-July and end-of-month-July return identical data, so I presume the day of month is currently ignored.
https://pittir.commons-archive.org/api/sushi/r51/reports/pr?begin_date=2023-06-01&end_date=2023-07-31&metric_type=Total_Item_Investigations
https://pittir.commons-archive.org/api/sushi/r51/reports/pr?begin_date=2023-06-01&end_date=2023-07-01&metric_type=Total_Item_Investigations

Finally, will the inclusion of Search_Investigations be removed from this report when filtered to Total_Item_Investigations per:
#686 (comment) ?

@alishaevn
Copy link
Contributor

alishaevn commented Sep 27, 2023

@ctgraham

I also tried overspecifying the date as year-month-day:
https://pittir.commons-archive.org/api/sushi/r51/reports/pr?begin_date=2023-05-30&end_date=2023-08-15&metric_type=Total_Item_Investigations

This returned the non-intuitive message of:
"The requested begin_date of 2023-05-30 is before the earliest metric date of 2023-05-27."

what messaging are you imagining instead?

A query for start-of-month-July and end-of-month-July return identical data, so I presume the day of month is currently ignored.

correct.

Finally, will the inclusion of Search_Investigations be removed from this report when filtered to Total_Item_Investigations per:
#686 (comment) ?

yes! (except that Total_Item_Investigations isn't a valid filter value. I believe you mean Total_Item_Requests. ref: this link)

@alishaevn alishaevn self-assigned this Sep 27, 2023
@ctgraham
Copy link
Collaborator

If the user's input is valid, but discarded/amended for a successful response (like substituting a begin_date of "2023-05-01" for "2023-05-30"), an informational Exception 0 should be included in the Report_Header which describes the successful substitution.

If the user's input is valid, but discarded/amended for a failed response, the resulting Exception should describe that, e.g.:
"The requested begin_date of 2023-05-30 is evaluated as 2023-05-01 and is before the earliest metric date of 2023-05-27."

alishaevn added a commit that referenced this issue Sep 27, 2023
`Searches_Platform` is a valid metric_type filter according to
https://countermetrics.stoplight.io/docs/counter-sushi-api/xggfvq5by3gx0-pr-p1-report-filters.

before this commit we were always returning `Searches_Platform`, despite
whether a `metric_type` filter was being used.

with this commit, we only return `Searches_Platform` if the `metric_type`
param includes it, or is not being passed at all.

ref: #687 (comment)
alishaevn added a commit that referenced this issue Sep 27, 2023
This is necessary because calling `to_date` on a string in 'MM-YYYY'
format will assume that 'MM' is the day, and will set the current month
as 'MM' instead.

params.fetch(:begin_date) => "06-2023"
params.fetch(:begin_date).to_date => Wed, 06 Sep 2023

We are also still allowing for dates to use MM-DD-YYYY format.

'2023-06-10'.match(/(^\d{4}-\d{2}$)|(^\d{4}-\d{2}-\d{2}$)/)
	=> #<MatchData "2023-06-10" 1:nil 2:"2023-06-10">
'2023-06'.match(/(^\d{4}-\d{2}$)|(^\d{4}-\d{2}-\d{2}$)/)
	=> #<MatchData "2023-06" 1:"2023-06" 2:nil>
'06-2023'.match(/(^\d{4}-\d{2}$)|(^\d{4}-\d{2}-\d{2}$)/)
	=> nil

ref: #687 (comment)
alishaevn added a commit that referenced this issue Sep 28, 2023
exception 0 is a special case, as it represents info or debugging messages.
when the query is successful, despite the exception, the exception is
to be returned as part of the report header.

ref: https://countermetrics.stoplight.io/docs/counter-sushi-api/jmelferytrixm-exception-0
ref: #687 (comment)
@alishaevn alishaevn moved this from Pitt QA to Code Review in palni-palci Sep 28, 2023
@labradford labradford moved this from Code Review to Deploy to Staging in palni-palci Sep 28, 2023
@labradford labradford moved this from Deploy to Staging to SoftServ QA in palni-palci Sep 28, 2023
@alishaevn
Copy link
Contributor

@ctgraham

Finally, will the inclusion of Search_Investigations be removed from this report when filtered to Total_Item_Investigations per: #686 (comment) ?

this is ready for review.
https://pittir.commons-archive.org/api/sushi/r51/reports/pr_p1?begin_date=2023-06&end_date=2023-07&metric_type=Total_Item_Requests

If the user's input is valid, but discarded/amended for a successful response (like substituting a begin_date of "2023-05-01" for "2023-05-30"), an informational Exception 0 should be included in the Report_Header which describes the successful substitution.

this is ready for review.
https://pittir.commons-archive.org/api/sushi/r51/reports/pr_p1?begin_date=2023-06-06&end_date=2023-07

If the user's input is valid, but discarded/amended for a failed response, the resulting Exception should describe that, e.g.:
"The requested begin_date of 2023-05-30 is evaluated as 2023-05-01 and is before the earliest metric date of 2023-05-27."

giving a date in YYYY-MM-DD format and giving an end date that is before the begin date, are two separate issues. as they are now both being represented by their respective exceptions, is that sufficient?

@alishaevn alishaevn moved this from SoftServ QA to Pitt QA in palni-palci Sep 29, 2023
@ctgraham
Copy link
Collaborator

Looks good.

@ctgraham ctgraham moved this from Pitt QA to PALs QA in palni-palci Sep 29, 2023
@ndroark ndroark moved this from PALs QA to Deploy to Production in palni-palci Oct 17, 2023
@jillpe jillpe moved this from Deploy to Production to Client Verification in palni-palci Jan 22, 2024
@jeremyf jeremyf removed their assignment May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Client Verification
Development

No branches or pull requests

5 participants