Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRM_Utils_SQL - Add "onDuplicate()" and "syncInto()" helpers #17780

Merged
merged 2 commits into from
Jul 13, 2020

Conversation

totten
Copy link
Member

@totten totten commented Jul 9, 2020

Overview

This adds helper functions, onDuplicate(...) and syncInto(...), for reading data from one table and writing to another.

This is an offshoot of #17724 which only adds the helper function and the unit-test.

Before

It is possible to construct a SELECT query and insert the results to another table:

CRM_Utils_SQL_Select::from('foo_table')
  ->select(['foo_name', 'foo_value1', 'foo_value2'])
  ->insertInto('bar_table', ['bar_name', 'bar_output1', 'bar_output2']);

However, this will always insert new records. It cannot be used to UPDATE existing records (i.e. INSERT INTO ... SELECT ... ON DUPLICATE UPDATE...).

The operation REPLACE INTO (replaceInto()) is somewhat similar to ON DUPLICATE UPDATE. Both operations will insert a new record. But if there's a conflict/pre-existing record, the REPLACE INTO will remove and replace the record, whereas ON DUPLICATE UPDATE only modifies fields. One benchmark reported a ~35x performance difference between REPLACE INTO and ON DUPLICATE UPDATE.

After

This first commit makes it possible to construct an ON DUPLICATE UPDATE query, eg.

CRM_Utils_SQL_Select::from('foo_table')
  ->select(['foo_name', 'foo_value1', 'foo_value2'])
  ->insertInto('bar_table', ['bar_name', 'bar_output1', 'bar_output2'])
  ->onDuplicate(['bar_output1 = foo_value1', 'bar_output2 = foo_value2']);

This parallels the underlying SQL structure. However, it is a bit verbose -- observe that every field on every side of the data-transfer (foo_value1, bar_value1, foo_value2, etc) has to be mentioned multiple times.

The second commit provides a convenience function which does the same thing - but with cleaner notation:

CRM_Utils_SQL_Select::from('foo_table')
  ->syncInto('bar_table', 'bar_name', [
    'bar_name' => 'foo_name',
    'bar_output1' => 'foo_value1',
    'bar_output2' => 'foo_value1',
  ])

totten added 2 commits July 8, 2020 23:19
…UPDATE..."

This is a query structure in which you want to build a SELECT query - and,
using the results, perform a mix of INSERTs and UPDATEs.
This is slightly more sugary variant of "INSERT INTO...SELECT...ON DUPLICATE UPDATE..." which requires
less boilerplate/duplication.

Before
------

```php
CRM_Utils_SQL_Select::from('foo_table')
  ->select(['foo_name', 'foo_value1', 'foo_value2'])
  ->insertInto('bar_table', ['bar_name', 'bar_output1', 'bar_output2'])
  ->onDuplicate(['bar_output1 = foo_value1', 'bar_output2 = foo_value2'])
```

After
------

```php
CRM_Utils_SQL_Select::from('foo_table')
  ->syncInto('bar_table', 'bar_name', [
    'bar_name' => 'foo_name',
    'bar_output1' => 'foo_value1',
    'bar_output2' => 'foo_value1',
  ])
```
@civibot
Copy link

civibot bot commented Jul 9, 2020

(Standard links)

@eileenmcnaughton
Copy link
Contributor

Seems like a straight forward helper & a logical sql function

@eileenmcnaughton eileenmcnaughton merged commit be7cf00 into civicrm:master Jul 13, 2020
@totten totten deleted the master-sync-into branch July 15, 2020 06:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants