Skip to content

Commit

Permalink
[Reporting] Formatting fixes for CSV export in Discover, CSV download…
Browse files Browse the repository at this point in the history 
… from Dashboard panel (#67027)

* [Reporting] Data formatting fixes for CSV export in Discover, CSV download from Dashboard panel

commit e195964deaa3e7e8d94704d6514e01498c913a81
Author: Timothy Sullivan <tsullivan@elastic.co>
Date:   Mon Jul 13 10:17:36 2020 -0700

    Squashed commit of the following:

    commit 87c9c496a6cccaf7a60a44b496f7c0c0423cd2ea
    Merge: d531101ab3 ed749eb
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Mon Jul 13 10:17:02 2020 -0700

        Merge branch 'data/allow-custom-formatting' into reporting/csv-date-format-consistency

    commit d531101ab3c2f12628287bd5ad4a02bbf8b5c990
    Merge: 400e2ffba4 17dc043
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Mon Jul 13 10:15:38 2020 -0700

        Merge branch 'master' into reporting/csv-date-format-consistency

    commit ed749eb
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Mon Jul 13 10:12:28 2020 -0700

        move shared code to common

    commit 4e5eebd
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Mon Jul 13 09:07:32 2020 -0700

        3td time api doc chagens

    commit 34df331
    Merge: 54fa2fe 17dc043
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Mon Jul 13 08:50:21 2020 -0700

        Merge branch 'master' into data/allow-custom-formatting

    commit 400e2ffba4546cf78c53ce96b45a59878f0df076
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Sun Jul 12 21:29:34 2020 -0700

        [Reporting] Data formatting fixes for CSV export in Discover, CSV download from Dashboard panel

    commit 54fa2fe
    Merge: 1b6e9e8 e1253ed
    Author: Elastic Machine <elasticmachine@users.noreply.github.com>
    Date:   Sun Jul 12 22:18:38 2020 -0600

        Merge branch 'master' into data/allow-custom-formatting

    commit 1b6e9e8
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Fri Jul 10 15:03:08 2020 -0700

        weird api change needed but no real diff

    commit fc9ff7b
    Merge: 736e9ee 66c531d
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Fri Jul 10 14:51:51 2020 -0700

        Merge branch 'master' into data/allow-custom-formatting

    commit 736e9ee
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Thu Jul 9 17:43:10 2020 -0700

        fix path for tests

    commit 1bebcc8
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Thu Jul 9 17:25:09 2020 -0700

        re-use public code in server, add test

    commit 1e1d3c5
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Thu Jul 9 16:35:30 2020 -0700

        rerun api changes

    commit 231f793
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Thu Jul 9 16:31:55 2020 -0700

        fix src/plugins/data/public/field_formats/constants.ts

    commit d42275c
    Merge: 206aed6 8e2277a
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Thu Jul 9 16:01:40 2020 -0700

        Merge branch 'master' into data/allow-custom-formatting

    commit 206aed6
    Merge: 5aa2d80 09da110
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Thu Jul 9 15:03:12 2020 -0700

        Merge branch 'master' into data/allow-custom-formatting

    commit 5aa2d80
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Wed Jul 8 12:12:31 2020 -0700

        api doc changes

    commit 76e2c30
    Merge: 1789afc 595e9c2
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Wed Jul 8 12:04:12 2020 -0700

        Merge branch 'master' into data/allow-custom-formatting

    commit 1789afc
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Fri Jul 3 11:23:03 2020 -0700

        simplify changes

    commit 6428455
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Thu Jul 2 16:05:57 2020 -0700

        add more to tests - need help though

    commit 6aacfbd
    Author: Timothy Sullivan <tsullivan@elastic.co>
    Date:   Thu Jul 2 12:04:28 2020 -0700

        [Data Plugin] Allow server-side date formatters to accept custom timezone

        When Advanced Settings shows the date format timezone to be "Browser,"
        this means nothing to field formatters in the server-side context. The
        field formatters need a way to accept custom format parameters. This
        allows a server-side module that creates a FieldFormatMap to set a
        timezone as a custom parameter. When custom formatting parameters exist,
        they get combined with the defaults.

* comments
  • Loading branch information
@tsullivan
tsullivan committed Jul 14, 2020
1 parent ed0d26f commit e7a4394
Show file tree
Hide file tree
Showing 52 changed files with 5,700 additions and 1,507 deletions.
2 changes: 2 additions & 0 deletions x-pack/plugins/reporting/common/types.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

// eslint-disable-next-line @kbn/eslint/no-restricted-paths
export { ReportingConfigType } from '../server/config';
// eslint-disable-next-line @kbn/eslint/no-restricted-paths
export { LayoutInstance } from '../server/export_types/common/layouts';

export type JobId = string;
export type JobStatus =
Expand Down
4 changes: 2 additions & 2 deletions x-pack/plugins/reporting/public/plugin.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import {
import { ManagementSectionId, ManagementSetup } from '../../../../src/plugins/management/public';
import { SharePluginSetup } from '../../../../src/plugins/share/public';
import { LicensingPluginSetup } from '../../licensing/public';
import { ReportingConfigType, JobId, JobStatusBuckets } from '../common/types';
import { JobId, JobStatusBuckets, ReportingConfigType } from '../common/types';
import { JOB_COMPLETION_NOTIFICATIONS_SESSION_KEY } from '../constants';
import { getGeneralErrorToast } from './components';
import { ReportListing } from './components/report_listing';
Expand Down Expand Up @@ -144,7 +144,7 @@ export class ReportingPublicPlugin implements Plugin<void, void> {

uiActions.addTriggerAction(CONTEXT_MENU_TRIGGER, action);

share.register(csvReportingProvider({ apiClient, toasts, license$ }));
share.register(csvReportingProvider({ apiClient, toasts, license$, uiSettings }));
share.register(
reportingPDFPNGProvider({
apiClient,
Expand Down
43 changes: 32 additions & 11 deletions x-pack/plugins/reporting/public/share_context_menu/register_csv_reporting.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,29 @@
*/

import { i18n } from '@kbn/i18n';
import moment from 'moment-timezone';
import React from 'react';

import { ToastsSetup } from 'src/core/public';
import { IUiSettingsClient, ToastsSetup } from 'src/core/public';
import { ShareContext } from '../../../../../src/plugins/share/public';
import { LicensingPluginSetup } from '../../../licensing/public';
import { JobParamsDiscoverCsv, SearchRequest } from '../../server/export_types/csv/types';
import { ReportingPanelContent } from '../components/reporting_panel_content';
import { ReportingAPIClient } from '../lib/reporting_api_client';
import { checkLicense } from '../lib/license_check';
import { LicensingPluginSetup } from '../../../licensing/public';
import { ShareContext } from '../../../../../src/plugins/share/public';
import { ReportingAPIClient } from '../lib/reporting_api_client';

interface ReportingProvider {
apiClient: ReportingAPIClient;
toasts: ToastsSetup;
license$: LicensingPluginSetup['license$'];
uiSettings: IUiSettingsClient;
}

export const csvReportingProvider = ({ apiClient, toasts, license$ }: ReportingProvider) => {
export const csvReportingProvider = ({
apiClient,
toasts,
license$,
uiSettings,
}: ReportingProvider) => {
let toolTipContent = '';
let disabled = true;
let hasCSVReporting = false;
Expand All @@ -33,6 +40,14 @@ export const csvReportingProvider = ({ apiClient, toasts, license$ }: ReportingP
disabled = !enableLinks;
});

// If the TZ is set to the default "Browser", it will not be useful for
// server-side export. We need to derive the timezone and pass it as a param
// to the export API.
const browserTimezone =
uiSettings.get('dateFormat:tz') === 'Browser'
? moment.tz.guess()
: uiSettings.get('dateFormat:tz');

const getShareMenuItems = ({
objectType,
objectId,
Expand All @@ -44,13 +59,19 @@ export const csvReportingProvider = ({ apiClient, toasts, license$ }: ReportingP
return [];
}

const getJobParams = () => {
return {
...sharingData,
type: objectType,
};
const jobParams: JobParamsDiscoverCsv = {
browserTimezone,
objectType,
title: sharingData.title as string,
indexPatternId: sharingData.indexPatternId as string,
searchRequest: sharingData.searchRequest as SearchRequest,
fields: sharingData.fields as string[],
metaFields: sharingData.metaFields as string[],
conflictedTypesFields: sharingData.conflictedTypesFields as string[],
};

const getJobParams = () => jobParams;

const shareActions = [];

if (hasCSVReporting) {
Expand Down
47 changes: 25 additions & 22 deletions x-pack/plugins/reporting/public/share_context_menu/register_pdf_png_reporting.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,15 @@
import { i18n } from '@kbn/i18n';
import moment from 'moment-timezone';
import React from 'react';
import { ToastsSetup, IUiSettingsClient } from 'src/core/public';
import { ReportingAPIClient } from '../lib/reporting_api_client';
import { checkLicense } from '../lib/license_check';
import { ScreenCapturePanelContent } from '../components/screen_capture_panel_content';
import { LicensingPluginSetup } from '../../../licensing/public';
import { IUiSettingsClient, ToastsSetup } from 'src/core/public';
import { ShareContext } from '../../../../../src/plugins/share/public';
import { LicensingPluginSetup } from '../../../licensing/public';
import { LayoutInstance } from '../../common/types';
import { JobParamsPNG } from '../../server/export_types/png/types';
import { JobParamsPDF } from '../../server/export_types/printable_pdf/types';
import { ScreenCapturePanelContent } from '../components/screen_capture_panel_content';
import { checkLicense } from '../lib/license_check';
import { ReportingAPIClient } from '../lib/reporting_api_client';

interface ReportingPDFPNGProvider {
apiClient: ReportingAPIClient;
Expand All @@ -39,6 +42,14 @@ export const reportingPDFPNGProvider = ({
disabled = !enableLinks;
});

// If the TZ is set to the default "Browser", it will not be useful for
// server-side export. We need to derive the timezone and pass it as a param
// to the export API.
const browserTimezone =
uiSettings.get('dateFormat:tz') === 'Browser'
? moment.tz.guess()
: uiSettings.get('dateFormat:tz');

const getShareMenuItems = ({
objectType,
objectId,
Expand All @@ -57,44 +68,36 @@ export const reportingPDFPNGProvider = ({
return [];
}

const getReportingJobParams = () => {
const getPdfJobParams = (): JobParamsPDF => {
// Relative URL must have URL prefix (Spaces ID prefix), but not server basePath
// Replace hashes with original RISON values.
const relativeUrl = shareableUrl.replace(
window.location.origin + apiClient.getServerBasePath(),
''
);

const browserTimezone =
uiSettings.get('dateFormat:tz') === 'Browser'
? moment.tz.guess()
: uiSettings.get('dateFormat:tz');

return {
...sharingData,
objectType,
browserTimezone,
relativeUrls: [relativeUrl],
relativeUrls: [relativeUrl], // multi URL for PDF
layout: sharingData.layout as LayoutInstance,
title: sharingData.title as string,
};
};

const getPngJobParams = () => {
const getPngJobParams = (): JobParamsPNG => {
// Replace hashes with original RISON values.
const relativeUrl = shareableUrl.replace(
window.location.origin + apiClient.getServerBasePath(),
''
);

const browserTimezone =
uiSettings.get('dateFormat:tz') === 'Browser'
? moment.tz.guess()
: uiSettings.get('dateFormat:tz');

return {
...sharingData,
objectType,
browserTimezone,
relativeUrl,
relativeUrl, // single URL for PNG
layout: sharingData.layout as LayoutInstance,
title: sharingData.title as string,
};
};

Expand Down Expand Up @@ -161,7 +164,7 @@ export const reportingPDFPNGProvider = ({
reportType="printablePdf"
objectType={objectType}
objectId={objectId}
getJobParams={getReportingJobParams}
getJobParams={getPdfJobParams}
isDirty={isDirty}
onClose={onClose}
/>
Expand Down
4 changes: 1 addition & 3 deletions x-pack/plugins/reporting/server/export_types/csv/server/create_job.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,19 @@ export const scheduleTaskFnFactory: ScheduleTaskFnFactory<ESQueueCreateJobFn<
>> = function createJobFactoryFn(reporting) {
const config = reporting.getConfig();
const crypto = cryptoFactory(config.get('encryptionKey'));
const setupDeps = reporting.getPluginSetupDeps();

return async function scheduleTask(jobParams, context, request) {
const serializedEncryptedHeaders = await crypto.encrypt(request.headers);

const savedObjectsClient = context.core.savedObjects.client;
const indexPatternSavedObject = await savedObjectsClient.get(
'index-pattern',
jobParams.indexPatternId!
jobParams.indexPatternId
);

return {
headers: serializedEncryptedHeaders,
indexPatternSavedObject,
basePath: setupDeps.basePath(request),
...jobParams,
};
};
Expand Down
Loading

0 comments on commit e7a4394

Please sign in to comment.