Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SQLite compatibility and fix SQL Full Group By issues (> MySQL 5.5 and MariaDB) #413

Merged
merged 60 commits into from
Jan 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
87a9f47
Tests: Add npm scripts to test sql query on different sql servers
bonny Nov 18, 2023
379b1b9
Tests: Remove commented out docker compose things
bonny Nov 18, 2023
c2af052
Add some comments to log query class
bonny Nov 20, 2023
95401d6
Tests: Add more log query tests
bonny Nov 20, 2023
2f10689
Tests: Add more log query occasions tests
bonny Nov 21, 2023
4d8598d
Remove mysql_get_server_info() because unknown function says editor
bonny Nov 22, 2023
927d815
Merge branch 'main' into bonny/issue409-db-issues
bonny Dec 9, 2023
9b3529d
Misc log query cleanup
bonny Dec 9, 2023
44351a5
Log query: Use arrays to store where clauses + misc cleanup
bonny Dec 10, 2023
80eef6f
Log query: Add more arguments to prepare function and misc cleanup
bonny Dec 11, 2023
64144e3
Log query: Add arguments users, user, loggers, messages, and more, to…
bonny Dec 12, 2023
e19d390
Log query: Limit fix for occassions
bonny Dec 13, 2023
85ae317
Log query: use %d for integers
bonny Dec 13, 2023
cfb166b
Log query: Specify columns to get when adding context instead of gett…
bonny Dec 13, 2023
539a074
Log query: Misc cleanup
bonny Dec 13, 2023
df451b6
Log query: Add query_occasions(), query_overview(), add_contexts_to_l…
bonny Dec 15, 2023
be45ae1
IP addresses are now shown on occasions
bonny Dec 15, 2023
0100bf5
Remove unused "log_rows_raw" key from occassions api response
bonny Dec 16, 2023
7baef32
Log query: Remove $_GET from cache key
bonny Dec 16, 2023
6bc6c35
Log query: Occassions cleanup
bonny Dec 16, 2023
5cc562e
Log query: Add functions get_inner_where() and get_outer_where()
bonny Dec 16, 2023
0224b52
Remove "rep" column from Log_Query response.
bonny Dec 16, 2023
8f4a4cb
Remove column "occasionsIDType" from Log Query
bonny Dec 16, 2023
43db5dc
Log query: Misc cleanup
bonny Dec 16, 2023
cb9f85c
Log query: New full group by query seems to work (lot's of debug!)
bonny Dec 18, 2023
596eefb
Remove debug
bonny Dec 18, 2023
c281c51
Log query fixes
bonny Dec 18, 2023
188afa0
Log query: Count using count(*)
bonny Dec 19, 2023
4636a23
Log query test: Disable mark as incomplete
bonny Dec 19, 2023
739eab0
Log query: Move default args to prepare_args function
bonny Dec 19, 2023
276a973
Log query: Add cache and limit to full group function, and rename to …
bonny Dec 19, 2023
b31fdd4
Log query: Remove "repeated" column
bonny Dec 19, 2023
6a233b9
Log Query: 👋 Remove old query that did not use full group by
bonny Dec 19, 2023
da360f8
Remove debug code
bonny Dec 19, 2023
06b7923
Log Query: SQLite almost working 😍
bonny Dec 22, 2023
8f4bd78
Log query: SQLite support for maxId and minId
bonny Dec 23, 2023
ba06515
Add helper functions `get_cache_group()`, `clear_cache()`
bonny Dec 23, 2023
ed57b44
Use Helpers::get_cache_group()
bonny Dec 27, 2023
809b05b
Use Helpers::clear_cache();
bonny Dec 27, 2023
13a2bd0
Use constant() to fix warning in editor
bonny Dec 27, 2023
bee4f3a
Add support for sqlite in get_num_events_per_day_last_n_days()x
bonny Dec 27, 2023
88e91a9
Log query: Use different queries for lastdays depending on db engine
bonny Dec 29, 2023
4ab73b4
Typos
bonny Dec 31, 2023
670b1ea
Add dropin that adds information about add-ons in the sidebar
bonny Dec 31, 2023
674b99d
Typos
bonny Dec 31, 2023
4109fd0
Add text to "purged_events" message with information that the number …
bonny Jan 1, 2024
01d1ddc
Add text to the clear log settings text with info how to modify numbe…
bonny Jan 1, 2024
e2e5186
Check date filtering works
bonny Jan 5, 2024
5461eeb
Readme: Add note about add-ons to sponsor headline
bonny Jan 7, 2024
8c42b61
Log query: Add caching to query_overview_sqlite()
bonny Jan 7, 2024
924c6bd
Tests: Add tests for single event occasions
bonny Jan 7, 2024
951c904
Update composer packages
bonny Jan 7, 2024
b00d045
Composer: Upgrade wp-cli
bonny Jan 7, 2024
d38c23a
Fix phpstan
bonny Jan 7, 2024
d6ce006
Log query: Fail silently
bonny Jan 7, 2024
f403dd8
Correct add-on link in sidebar
bonny Jan 7, 2024
61711aa
Load occasions using more specific link target
bonny Jan 7, 2024
ce413c6
Update readme
bonny Jan 7, 2024
4a4ae86
Prepare for add-ons info in occasions text
bonny Jan 7, 2024
005b20e
Update readme
bonny Jan 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"codeception/module-cli": "^1.0",
"codeception/util-universalframework": "^1.0",
"rector/rector": "^0.17.0",
"wp-cli/wp-cli-bundle": "2.8.1",
"wp-cli/wp-cli-bundle": "^2.9",
"phpstan/phpstan": "^1.10.0",
"szepeviktor/phpstan-wordpress": "dev-master",
"phpstan/extension-installer": "^1.3.1",
Expand Down
719 changes: 422 additions & 297 deletions composer.lock

Large diffs are not rendered by default.

7 changes: 6 additions & 1 deletion css/styles.css
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,12 @@ Style different log levels.
color: rgb(137, 143, 156);
margin-top: 0.4em;
line-height: 1;
max-height: 1em;
/* max-height: 1em; */
}

.SimpleHistoryLogitem__occasionsAddOnsText {
margin: 0.4em 0 0 0;
line-height: 1;
}

.SimpleHistoryLogitem__details {
Expand Down
4 changes: 0 additions & 4 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,6 @@ services:

db:
image: "${DB_IMAGE:-mariadb:10.5}"
#image: "${DB_IMAGE:-biarms/mysql:5.5}"
#image: "${DB_IMAGE:-biarms/mysql:5.7}"
container_name: simple-history-database
restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"
environment:
Expand All @@ -46,8 +44,6 @@ services:
- "${DB_EXPOSE_PORT:-127.0.0.1:}:3306"
volumes:
- "${DB_DATA_DIR:-./data/mysql}:/var/lib/mysql"
#- "${DB_DATA_DIR:-./data/mysql-5.5}:/var/lib/mysql"
#- "${DB_DATA_DIR:-./data/mysql-5.7}:/var/lib/mysql"

chrome:
image: seleniarm/standalone-chromium:117.0
Expand Down
1 change: 0 additions & 1 deletion dropins/class-new-rows-notifier-dropin.php
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,6 @@ public function ajax() {
);
}

// $since_id = isset( $_GET["since_id"] ) ? absint($_GET["since_id"]) : null;
$logQueryArgs = $apiArgs;

$logQuery = new Log_Query();
Expand Down
2 changes: 1 addition & 1 deletion dropins/class-quick-stats.php
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ public function output_quick_stats() {
);

$cache_key = 'quick_stats_users_today_' . md5( serialize( $sql_loggers_in ) );
$cache_group = 'simple-history-' . Helpers::get_cache_incrementor();
$cache_group = Helpers::get_cache_group();
$results_users_today = wp_cache_get( $cache_key, $cache_group );

if ( false === $results_users_today ) {
Expand Down
65 changes: 65 additions & 0 deletions dropins/class-sidebar-add-ons-dropin.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
<?php

namespace Simple_History\Dropins;

use Simple_History\Simple_History;

/**
* Dropin that displays information about add-ons in the sidebar.
*/
class Sidebar_Add_Ons_Dropin extends Dropin {
/**
* Add actions when dropin is loaded.
*/
public function loaded() {
add_action( 'simple_history/dropin/sidebar/sidebar_html', [ $this, 'on_sidebar_html' ], 5 );
}

/**
* Output HTML.
*/
public function on_sidebar_html() {
?>
<div class="postbox">

<h3 class="hndle">
<?php esc_html_e( 'Add-ons', 'simple-history' ); ?>
<em class="sh-PageHeader-settingsLinkIcon-new">New</em>
</h3>

<div class="inside">
<p>
<?php esc_html_e( 'Now you can enhance and extend Simple History with add-ons.', 'simple-history' ); ?>
</p>

<?php
/*
<ul>
<li>
<strong><?php esc_html_e( 'Extended Settings:', 'simple-history' ); ?></strong>
<?php esc_html_e( 'Control number of days to keep the log, limit number of login attempts logged, store full IP-addresses, and a bonus JSON feed!', 'simple-history' ); ?>
</li>

<li>
<strong><?php esc_html_e( 'WooCommerce Logger:', 'simple-history' ); ?></strong>
<?php esc_html_e( 'Detailed WooCommerce activity logs. (Coming soon.)', 'simple-history' ); ?>
</li>

<li>
<strong><?php esc_html_e( 'Developer Tools:', 'simple-history' ); ?></strong>
<?php esc_html_e( 'Email, HTTP request, wp-cron job monitoring. (Coming soon.)', 'simple-history' ); ?>
</li>
</ul>
*/
?>

<p>
<a href="https://simple-history.com/add-ons/?utm_source=wpadmin" class="sh-ExternalLink" target="_blank">
<?php esc_html_e( 'View add-ons.', 'simple-history' ); ?>
</a>
</p>
</div>
</div>
<?php
}
}
2 changes: 1 addition & 1 deletion dropins/class-sidebar-settings-dropin.php
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ class Sidebar_Settings_Dropin extends Dropin {
* Init
*/
public function loaded() {
add_action( 'simple_history/dropin/sidebar/sidebar_html', array( $this, 'on_sidebar_html' ), 5 );
add_action( 'simple_history/dropin/sidebar/sidebar_html', array( $this, 'on_sidebar_html' ), 7 );
}

/**
Expand Down
81 changes: 62 additions & 19 deletions inc/class-helpers.php
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,25 @@ public static function get_cache_incrementor( $refresh = false ) {
return $incrementor_value;
}

/**
* Get the cache group for the cache.
* Used by all function that use cache, so they use the
* same cache group, meaning if we invalidate the cache group
* all caches will be cleared/flushed.
*
* @return string
*/
public static function get_cache_group() {
return 'simple-history-' . self::get_cache_incrementor();
}

/**
* Clears the cache.
*/
public static function clear_cache() {
self::get_cache_incrementor( true );
}

/**
* Return a name for a callable.
*
Expand Down Expand Up @@ -881,7 +900,7 @@ public static function clear_log() {
// phpcs:ignore WordPress.DB.PreparedSQL.NotPrepared
$wpdb->query( $sql );

self::get_cache_incrementor( true );
self::clear_cache();

return $num_rows;
}
Expand Down Expand Up @@ -1127,31 +1146,55 @@ public static function get_num_events_last_n_days( $period_days = 28 ) {
*/
public static function get_num_events_per_day_last_n_days( $period_days = 28 ) {
$simple_history = Simple_History::get_instance();
$transient_key = 'sh_' . md5( __METHOD__ . $period_days . '_2' );
$transient_key = 'sh_' . md5( __METHOD__ . $period_days . '_3' );
$dates = get_transient( $transient_key );

if ( false === $dates ) {
/** @var \wpdb $wpdb */
global $wpdb;

$sqlStringLoggersUserCanRead = $simple_history->get_loggers_that_user_can_read( null, 'sql' );

$sql = sprintf(
'
SELECT
date_format(date, "%%Y-%%m-%%d") AS yearDate,
count(date) AS count
FROM
%1$s
WHERE
UNIX_TIMESTAMP(date) >= %2$d
AND logger IN (%3$d)
GROUP BY yearDate
ORDER BY yearDate ASC
',
$simple_history->get_events_table_name(),
strtotime( "-$period_days days" ),
$sqlStringLoggersUserCanRead
);
$db_engine = Log_Query::get_db_engine();

if ( $db_engine === 'mysql' ) {
$sql = sprintf(
'
SELECT
date_format(date, "%%Y-%%m-%%d") AS yearDate,
count(date) AS count
FROM
%1$s
WHERE
UNIX_TIMESTAMP(date) >= %2$d
AND logger IN %3$s
GROUP BY yearDate
ORDER BY yearDate ASC
',
$simple_history->get_events_table_name(),
strtotime( "-$period_days days" ),
$sqlStringLoggersUserCanRead
);
} elseif ( $db_engine === 'sqlite' ) {
// SQLite does not support date_format() or UNIX_TIMESTAMP so we need to use strftime().
$sql = sprintf(
'
SELECT
strftime("%%Y-%%m-%%d", date) AS yearDate,
count(date) AS count
FROM
%1$s
WHERE
unixepoch(date) >= %2$d
AND logger IN %3$s
GROUP BY yearDate
ORDER BY yearDate ASC
',
$simple_history->get_events_table_name(),
strtotime( "-$period_days days" ),
$sqlStringLoggersUserCanRead
);
}

$dates = $wpdb->get_results( $sql ); // phpcs:ignore WordPress.DB.PreparedSQL.NotPrepared

Expand Down
Loading