Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export code cleanup - Use getComponentPaymentFields from processorClass #13082

Merged
merged 4 commits into from
Nov 16, 2018

Conversation

eileenmcnaughton
Copy link
Contributor

@eileenmcnaughton eileenmcnaughton commented Nov 12, 2018

Overview

Minor code refactor as part of ongoing cleanup on export processor

Before

Less functions on new processor class

After

More functions on new processor class

Technical Details

We have been cleaning up this class preliminary to address bugs, including moving code to use the new ExportProcessor class

Comments

@twomice I've added another small refactor - this affects the compilation of payment fields being exported (eg from a participant export or member export) - it seems they should be included in 'primary fields' for the entity or be selectable for export - but the way this functionality was added is pretty bad.

Next step is to move getTransformedFieldValue off to the ExportProcessor class & extracting a buildRow function. The goal would be to merge the household in that function (maybe via array merge) and completely eliminate the mergeSameHousehold function

@civibot
Copy link

civibot bot commented Nov 12, 2018

(Standard links)

// get payment related in for event and members
$paymentDetails = CRM_Contribute_BAO_Contribution::getContributionDetails($exportMode, $ids);
//get all payment headers.
// If we haven't selected specific payment fields, load in all the
// payment headers.
if (!$processor->isExportSpecifiedPaymentFields()) {
$paymentHeaders = self::componentPaymentFields();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced by $processor->getPaymentHeaders(

else {
$paymentHeaders = array();
}
$nullContributionDetails = array_fill_keys(array_keys($paymentHeaders), NULL);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calculated when it is used

@@ -468,6 +459,16 @@ public static function exportComponents(

list($outputColumns, $headerRows, $sqlColumns, $metadata) = self::getExportStructureArrays($returnProperties, $processor);

// add payment headers if required
if ($addPaymentHeader && $processor->isExportPaymentFields()) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved to being before the row-by-row loop

@eileenmcnaughton eileenmcnaughton changed the title Use getComponentPaymentFields from processorClass Export code cleanup - Use getComponentPaymentFields from processorClass Nov 12, 2018
@twomice
Copy link
Contributor

twomice commented Nov 15, 2018

@eileenmcnaughton The code looks reasonable to me; I don't have any improvements to suggest. I also don't expect that manual testing would turn up anything useful, but can do it if you think it's helpful.

@eileenmcnaughton
Copy link
Contributor Author

@twomice so in general we do try to test each change - I'm OK with merging this & then doing the next one if you commit to do a thorough round of testing towards the end of the month (ie. test all at once rather than individually)

@twomice
Copy link
Contributor

twomice commented Nov 15, 2018

@eileenmcnaughton If you're saying manual testing would be appropriate here, I can do that. I suppose it would be good to try a few different exports with primary fields, custom fields, memberships, and contributions?

@eileenmcnaughton
Copy link
Contributor Author

@twomice In this case the focus would be on exporting entities with contributions - ie. some entities support (or at least according to the code :-) exporting related contribution data

@twomice
Copy link
Contributor

twomice commented Nov 15, 2018

Great, will do. Thanks!

The saving of not creating an empty file occassionally doesn't warrant the code complexity
@mattwire
Copy link
Contributor

mattwire commented Nov 16, 2018

@eileenmcnaughton I've done some basic testing with this PR:

  • Participant export, standard fields ~151contacts: Exports ok, export has contact, participant, contribution details where appropriate.
  • Membership export, standard fields ~150contacts: Exports ok, export has contact, membership, contribution details where appropriate.
  • Contact export, standard fields ~200contacts: Exports ok, export has contact details. Merge same address works results in about 150 results, Merge households crashes as before (but this PR is not fixing that).

I think this is ok to merge

@eileenmcnaughton
Copy link
Contributor Author

thanks @mattwire - merging

@eileenmcnaughton eileenmcnaughton merged commit 9ab0b9f into civicrm:master Nov 16, 2018
@eileenmcnaughton eileenmcnaughton deleted the export branch November 16, 2018 20:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants