When a remote poller fails, the recovery process may also fail

[bmfmancini]

when a remote poller fails the recovery fails with the below errors occour and the poller does not recover

2022-12-18 08:33:05 - CMDPHP PHP ERROR Backtrace:  (CactiShutdownHandler())
2022-12-18 08:33:05 - SYSTEM MAINT STATS: Time:0.02
2022-12-18 08:34:04 - SYSTEM STATS: Time:2.2168 Method:cmd.php Processes:1 Threads:1 Hosts:10 HostsPerProcess:10 DataSources:8 RRDsProcessed:0
2022-12-18 08:34:04 - POLLER: Poller[2] PID[3100] NOTE: Remote Data Collection to force processing of boost records.
2022-12-18 08:34:05 - POLLER: Poller[2] PID[3159] RECOVERY: No pid exists, starting recovery process (PID=3159)!
2022-12-18 08:34:05 - CMDPHP ERROR: A DB Row Failed!, Error: PDOStatement::execute(): SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens
2022-12-18 08:34:05 - CMDPHP SQL Backtrace:  (/poller_recovery.php[196]:poller_push_reindex_data_to_poller(), /lib/poller.php[1872]:db_fetch_assoc_prepared(), /lib/database.php[701]:db_execute_prepared())
2022-12-18 08:34:05 - ERROR PHP ERROR: Uncaught TypeError: array_rekey(): Argument #1 ($array) must be of type array, bool given, called in /var/www/html/cacti/lib/poller.php on line 1873 and defined in /var/www/html/cacti/lib/functions.php:1247 Stack trace: #0 /var/www/html/cacti/lib/poller.php(1873): array_rekey() #1 /var/www/html/cacti/poller_recovery.php(196): poller_push_reindex_data_to_poller() #2 {main} thrown in file: /var/www/html/cacti/lib/functions.php on line: 1247
2022-12-18 08:34:05 - CMDPHP PHP ERROR Backtrace:  (CactiShutdownHandler())
2022-12-18 08:34:05 - SYSTEM MAINT STATS: Time:0.02



2022-12-18 08:36:05 - CMDPHP PHP ERROR Backtrace:  (CactiShutdownHandler())
--
2022-12-18 08:36:05 - ERROR PHP ERROR: Uncaught TypeError:  array_rekey(): Argument #1 ($array) must be of type array, bool given,  called in /var/www/html/cacti/lib/poller.php on line 1873 and defined in  /var/www/html/cacti/lib/functions.php:1247 Stack trace: #0  /var/www/html/cacti/lib/poller.php(1873): array_rekey() #1  /var/www/html/cacti/poller_recovery.php(196):  poller_push_reindex_data_to_poller() #2 {main} thrown in file:  /var/www/html/cacti/lib/functions.php on line: 1247
2022-12-18 08:36:05 - CMDPHP SQL Backtrace:   (/poller_recovery.php[196]:poller_push_reindex_data_to_poller(),  /lib/poller.php[1872]:db_fetch_assoc_prepared(),  /lib/database.php[701]:db_execute_prepared())
2022-12-18 08:36:05 - CMDPHP ERROR: A DB Row Failed!, Error:  PDOStatement::execute(): SQLSTATE[HY093]: Invalid parameter number:  number of bound variables does not match number of tokens
2022-12-18 08:36:05 - POLLER: Poller[remote] PID[3417] RECOVERY: No pid exists, starting recovery process (PID=3417)!
2022-12-18 08:36:05 - SYSTEM MAINT STATS: Time:0.02
2022-12-18 08:36:05 - POLLER: Poller[remote] PID[3361] NOTE: Remote Data Collection to force processing of boost records.
2022-12-18 08:36:05 - SYSTEM STATS: Time:2.2043 Method:cmd.php Processes:1 Thre

2022-12-18 08:36:05 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler())
2022-12-18 08:36:05 - ERROR PHP ERROR: Uncaught TypeError: array_rekey(): Argument #1 ($array) must be of type array, bool given, called in /var/www/html/cacti/lib/poller.php on line 1873 and defined in /var/www/html/cacti/lib/functions.php:1247 Stack trace: #0 /var/www/html/cacti/lib/poller.php(1873): array_rekey() #1 /var/www/html/cacti/poller_recovery.php(196): poller_push_reindex_data_to_poller() #2 {main} thrown in file: /var/www/html/cacti/lib/functions.php on line: 1247
2022-12-18 08:36:05 - CMDPHP SQL Backtrace: (/poller_recovery.php[196]:poller_push_reindex_data_to_poller(), /lib/poller.php[1872]:db_fetch_assoc_prepared(), /lib/database.php[701]:db_execute_prepared())
2022-12-18 08:36:05 - CMDPHP ERROR: A DB Row Failed!, Error: PDOStatement::execute(): SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens
2022-12-18 08:36:05 - POLLER: Poller[[remote](http://192.168.1.202/cacti/pollers.php?action=edit&id=2)] PID[3417] RECOVERY: No pid exists, starting recovery process (PID=3417)!
2022-12-18 08:36:05 - SYSTEM MAINT STATS: Time:0.02
2022-12-18 08:36:05 - POLLER: Poller[[remote](http://192.168.1.202/cacti/pollers.php?action=edit&id=2)] PID[3361] NOTE: Remote Data Collection to force processing of boost records.
2022-12-18 08:36:05 - SYSTEM STATS: Time:2.2043 Method:cmd.php Processes:1 Thre

[bmfmancini]

if you truncate poller_output_boost the poller recovers so its isolated to the recovery process

[bmfmancini]

[root@localhost cacti]# php poller_recovery.php --verbose
PHP Fatal error:  Uncaught TypeError: array_rekey(): Argument #1 ($array) must be of type array, bool given, called in /var/www/html/cacti/lib/poller.php on line 1876 and defined in /var/www/html/cacti/lib/functions.php:1247
Stack trace:
#0 /var/www/html/cacti/lib/poller.php(1876): array_rekey()
#1 /var/www/html/cacti/poller_recovery.php(196): poller_push_reindex_data_to_poller()
#2 {main}
  thrown in /var/www/html/cacti/lib/functions.php on line 1247
[root@localhost cacti]# ^C

[TheWitness]

Pretty sure this affects 1.2.23 as well. Publishing a patch there first, then 1.3.0.

[TheWitness]

I've published an update. Please redo the test as you did before.

[TheWitness]

I've now tested this and corrected a minor issue. Please continue testing from your point of view. Thanks!

[TheWitness]

@bmfmancini, keeping this open till you get a chance to retest. I've confirmed on my side that it recovers just fine now.

[bmfmancini]

confirmed all good