Skip to content

Commit

Permalink
Fix dataset export conflict with paginated results (#484)
Browse files Browse the repository at this point in the history
Our front-end uses `from` and `search_after` query params to handle paginated
search results. When a user exports a subset of our complaint database, the
aforementioned pagination params are sent to the API causing it to throw an
error because `from mod size != 0`.

Instead of reworking the API logic it's better to just purge pagination
params from export URLs because they're irrelevant. The whole point of exporting
is to download all the search results and not just a single page of search results.

See https://github.com/cfpb/ccdb5-api/blob/a63efc345e1a652f9c8ec4875e9247aef6f56128/complaint_search/serializer.py#L185-L189
See https://jira/browse/DATAP-33
  • Loading branch information
contolini authored Jan 25, 2024
1 parent c47e1fc commit 9bedfa7
Show file tree
Hide file tree
Showing 4 changed files with 42 additions and 4 deletions.
34 changes: 34 additions & 0 deletions cypress/e2e/export/export.cy.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
describe('Complaint export', () => {
const currentPage = '.m-pagination_label';
const nextButton = '.m-pagination .m-pagination_btn-next';

const exportButton = '.export-btn';
const filteredDataset = 'label[for=dataset_filtered]';
const exportUriInput = '.heres-the-url input';

beforeEach(() => {
let request = '?**&field=all**sort=created_date_desc';
let fixture = { fixture: 'list/get-complaints.json' };
cy.intercept(request, fixture).as('getComplaints');

request = '?**&size=0';
fixture = { fixture: 'list/get-aggs.json' };
cy.intercept('GET', request, fixture).as('getAggs');

cy.visit('?size=10&searchText=debt%20recovery&tab=List');
cy.wait('@getComplaints');
cy.wait('@getAggs');
});

it('sends user to an export link without pagination params', () => {
cy.get(nextButton).click();
cy.url().should('include', 'page=2');
cy.get(currentPage).should('have.text', 'Page 2');

cy.get(exportButton).click();
cy.get(filteredDataset).click();

cy.get(exportUriInput).should('not.include.value', 'frm=');
cy.get(exportUriInput).should('not.include.value', 'search_after=');
});
});
6 changes: 3 additions & 3 deletions dist/ccdb5.js

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion dist/ccdb5.js.map

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions src/components/Dialogs/DataExport/dataExportUtils.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,9 @@ export function buildSomeResultsUri(format, size, queryState) {
// eslint-disable-next-line camelcase
params.no_aggs = true;

// Remove unnecessary pagination query params
delete params.from;
delete params.searchAfter;

return API_PLACEHOLDER + stateToQS(params);
}

0 comments on commit 9bedfa7

Please sign in to comment.