Skip to content
This repository has been archived by the owner on Dec 13, 2022. It is now read-only.

Disk full after Upgrade to 20.04 #8797

Open
MikeW83 opened this issue Jun 18, 2020 · 14 comments
Open

Disk full after Upgrade to 20.04 #8797

MikeW83 opened this issue Jun 18, 2020 · 14 comments

Comments

@MikeW83
Copy link

MikeW83 commented Jun 18, 2020

Hello guys,

I have seen today that our Centreon is not connecting to the database. The I realized that there is no space left on the disk.

# df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 1.9G     0  1.9G   0% /dev
tmpfs                    1.9G     0  1.9G   0% /dev/shm
tmpfs                    1.9G  8.8M  1.9G   1% /run
tmpfs                    1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/mapper/centos-root   37G   37G   20K 100% /
/dev/sda1               1014M  186M  829M  19% /boot
tmpfs                    379M     0  379M   0% /run/user/0

I found out that there are so many files in /var/lib/mysql/centreon_storage/ => More than 1250 files. And the biggest files aredata_bin#P#* from every day since running Centreon.

# systemctl status mariadb
● mariadb.service - MariaDB 10.3.22 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           └─centreon.conf, limits.conf, migrated-from-my.cnf-settings.conf
   Active: active (running) since Thu 2020-06-18 08:29:42 CEST; 9min ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 1631 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
  Process: 975 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
  Process: 968 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
 Main PID: 1088 (mysqld)
   Status: "Taking your SQL requests now..."
   CGroup: /system.slice/mariadb.service
           └─1088 /usr/sbin/mysqld

Jun 18 08:36:51 srv-mp-mon-01.mp.local mysqld[1088]: 2020-06-18  8:36:51 793 [ERROR] mysqld: Disk full (/tmp/#sql_440_0.MAI); waiting for someone to free some space... (errno: 28 "No sp... on device")
Jun 18 08:37:06 srv-mp-mon-01.mp.local mysqld[1088]: 2020-06-18  8:37:06 821 [ERROR] mysqld: Disk full (/tmp/#sql_440_0.MAI); waiting for someone to free some space... (errno: 28 "No sp... on device")
Jun 18 08:37:09 srv-mp-mon-01.mp.local mysqld[1088]: 2020-06-18  8:37:09 831 [ERROR] mysqld: Disk full (/tmp/#sql_440_0.MAI); waiting for someone to free some space... (errno: 28 "No sp... on device")
Jun 18 08:37:21 srv-mp-mon-01.mp.local mysqld[1088]: 2020-06-18  8:37:21 851 [ERROR] mysqld: Disk full (/tmp/#sql_440_0.MAI); waiting for someone to free some space... (errno: 28 "No sp... on device")
Jun 18 08:37:36 srv-mp-mon-01.mp.local mysqld[1088]: 2020-06-18  8:37:36 877 [ERROR] mysqld: Disk full (/tmp/#sql_440_0.MAI); waiting for someone to free some space... (errno: 28 "No sp... on device")
Jun 18 08:37:51 srv-mp-mon-01.mp.local mysqld[1088]: 2020-06-18  8:37:51 903 [ERROR] mysqld: Disk full (/tmp/#sql_440_0.MAI); waiting for someone to free some space... (errno: 28 "No sp... on device")
Jun 18 08:38:06 srv-mp-mon-01.mp.local mysqld[1088]: 2020-06-18  8:38:06 931 [ERROR] mysqld: Disk full (/tmp/#sql_440_0.MAI); waiting for someone to free some space... (errno: 28 "No sp... on device")
Jun 18 08:38:21 srv-mp-mon-01.mp.local mysqld[1088]: 2020-06-18  8:38:21 957 [ERROR] mysqld: Disk full (/tmp/#sql_440_0.MAI); waiting for someone to free some space... (errno: 28 "No sp... on device")
Jun 18 08:38:36 srv-mp-mon-01.mp.local mysqld[1088]: 2020-06-18  8:38:36 983 [ERROR] mysqld: Disk full (/tmp/#sql_440_0.MAI); waiting for someone to free some space... (errno: 28 "No sp... on device")
Jun 18 08:38:51 srv-mp-mon-01.mp.local mysqld[1088]: 2020-06-18  8:38:51 1009 [ERROR] mysqld: Disk full (/tmp/#sql_440_0.MAI); waiting for someone to free some space... (errno: 28 "No s... on device")
Hint: Some lines were ellipsized, use -l to show in full.

Now MariaDB isn't starting anymore and I need a solution.

Please help.

Thank you and best regards, Michael

@MikeW83 MikeW83 added the status/new New tickets label Jun 18, 2020
@MikeW83
Copy link
Author

MikeW83 commented Jun 19, 2020

This is how it looks like in the directory for the database files which are cosuming all the space.

# ls
acknowledgements.frm      data_bin#P#p20200102.ibd      issues_issues_parents.frm         log_archive_host#P#p20200106.ibd     log_archive_service#P#p20190719.ibd  log_archive_service#P#p20200214.ibd
acknowledgements.ibd      data_bin#P#p20200103.ibd      issues_issues_parents.ibd         log_archive_host#P#p20200107.ibd     log_archive_service#P#p20190720.ibd  log_archive_service#P#p20200215.ibd
centreon_acl.frm          data_bin#P#p20200104.ibd      log_action.frm                    log_archive_host#P#p20200108.ibd     log_archive_service#P#p20190721.ibd  log_archive_service#P#p20200216.ibd
centreon_acl.ibd          data_bin#P#p20200105.ibd      log_action.ibd                    log_archive_host#P#p20200109.ibd     log_archive_service#P#p20190722.ibd  log_archive_service#P#p20200217.ibd
comments.frm              data_bin#P#p20200106.ibd      log_action_modification.frm       log_archive_host#P#p20200110.ibd     log_archive_service#P#p20190723.ibd  log_archive_service#P#p20200218.ibd
comments.ibd              data_bin#P#p20200107.ibd      log_action_modification.ibd       log_archive_host#P#p20200111.ibd     log_archive_service#P#p20190724.ibd  log_archive_service#P#p20200219.ibd
config.frm                data_bin#P#p20200108.ibd      log_archive_host.frm              log_archive_host#P#p20200112.ibd     log_archive_service#P#p20190725.ibd  log_archive_service#P#p20200220.ibd
config.ibd                data_bin#P#p20200109.ibd      log_archive_host.par              log_archive_host#P#p20200113.ibd     log_archive_service#P#p20190726.ibd  log_archive_service#P#p20200221.ibd
customvariables.frm       data_bin#P#p20200110.ibd      log_archive_host#P#p20190618.ibd  log_archive_host#P#p20200114.ibd     log_archive_service#P#p20190727.ibd  log_archive_service#P#p20200222.ibd
customvariables.ibd       data_bin#P#p20200111.ibd      log_archive_host#P#p20190619.ibd  log_archive_host#P#p20200115.ibd     log_archive_service#P#p20190728.ibd  log_archive_service#P#p20200223.ibd
data_bin.frm              data_bin#P#p20200112.ibd      log_archive_host#P#p20190620.ibd  log_archive_host#P#p20200116.ibd     log_archive_service#P#p20190729.ibd  log_archive_service#P#p20200224.ibd
data_bin.par              data_bin#P#p20200113.ibd      log_archive_host#P#p20190621.ibd  log_archive_host#P#p20200117.ibd     log_archive_service#P#p20190730.ibd  log_archive_service#P#p20200225.ibd
data_bin#P#p20190618.ibd  data_bin#P#p20200114.ibd      log_archive_host#P#p20190622.ibd  log_archive_host#P#p20200118.ibd     log_archive_service#P#p20190731.ibd  log_archive_service#P#p20200226.ibd
data_bin#P#p20190619.ibd  data_bin#P#p20200115.ibd      log_archive_host#P#p20190623.ibd  log_archive_host#P#p20200119.ibd     log_archive_service#P#p20190801.ibd  log_archive_service#P#p20200227.ibd
data_bin#P#p20190620.ibd  data_bin#P#p20200116.ibd      log_archive_host#P#p20190624.ibd  log_archive_host#P#p20200120.ibd     log_archive_service#P#p20190802.ibd  log_archive_service#P#p20200228.ibd
data_bin#P#p20190621.ibd  data_bin#P#p20200117.ibd      log_archive_host#P#p20190625.ibd  log_archive_host#P#p20200121.ibd     log_archive_service#P#p20190803.ibd  log_archive_service#P#p20200229.ibd
data_bin#P#p20190622.ibd  data_bin#P#p20200118.ibd      log_archive_host#P#p20190626.ibd  log_archive_host#P#p20200122.ibd     log_archive_service#P#p20190804.ibd  log_archive_service#P#p20200301.ibd
data_bin#P#p20190623.ibd  data_bin#P#p20200119.ibd      log_archive_host#P#p20190627.ibd  log_archive_host#P#p20200123.ibd     log_archive_service#P#p20190805.ibd  log_archive_service#P#p20200302.ibd
data_bin#P#p20190624.ibd  data_bin#P#p20200120.ibd      log_archive_host#P#p20190628.ibd  log_archive_host#P#p20200124.ibd     log_archive_service#P#p20190806.ibd  log_archive_service#P#p20200303.ibd
data_bin#P#p20190625.ibd  data_bin#P#p20200121.ibd      log_archive_host#P#p20190629.ibd  log_archive_host#P#p20200125.ibd     log_archive_service#P#p20190807.ibd  log_archive_service#P#p20200304.ibd
data_bin#P#p20190626.ibd  data_bin#P#p20200122.ibd      log_archive_host#P#p20190630.ibd  log_archive_host#P#p20200126.ibd     log_archive_service#P#p20190808.ibd  log_archive_service#P#p20200305.ibd
data_bin#P#p20190627.ibd  data_bin#P#p20200123.ibd      log_archive_host#P#p20190701.ibd  log_archive_host#P#p20200127.ibd     log_archive_service#P#p20190809.ibd  log_archive_service#P#p20200306.ibd
data_bin#P#p20190628.ibd  data_bin#P#p20200124.ibd      log_archive_host#P#p20190702.ibd  log_archive_host#P#p20200128.ibd     log_archive_service#P#p20190810.ibd  log_archive_service#P#p20200307.ibd
data_bin#P#p20190629.ibd  data_bin#P#p20200125.ibd      log_archive_host#P#p20190703.ibd  log_archive_host#P#p20200129.ibd     log_archive_service#P#p20190811.ibd  log_archive_service#P#p20200308.ibd
data_bin#P#p20190630.ibd  data_bin#P#p20200126.ibd      log_archive_host#P#p20190704.ibd  log_archive_host#P#p20200130.ibd     log_archive_service#P#p20190812.ibd  log_archive_service#P#p20200309.ibd
data_bin#P#p20190701.ibd  data_bin#P#p20200127.ibd      log_archive_host#P#p20190705.ibd  log_archive_host#P#p20200131.ibd     log_archive_service#P#p20190813.ibd  log_archive_service#P#p20200310.ibd
data_bin#P#p20190702.ibd  data_bin#P#p20200128.ibd      log_archive_host#P#p20190706.ibd  log_archive_host#P#p20200201.ibd     log_archive_service#P#p20190814.ibd  log_archive_service#P#p20200311.ibd
data_bin#P#p20190703.ibd  data_bin#P#p20200129.ibd      log_archive_host#P#p20190707.ibd  log_archive_host#P#p20200202.ibd     log_archive_service#P#p20190815.ibd  log_archive_service#P#p20200312.ibd
data_bin#P#p20190704.ibd  data_bin#P#p20200130.ibd      log_archive_host#P#p20190708.ibd  log_archive_host#P#p20200203.ibd     log_archive_service#P#p20190816.ibd  log_archive_service#P#p20200313.ibd
data_bin#P#p20190705.ibd  data_bin#P#p20200131.ibd      log_archive_host#P#p20190709.ibd  log_archive_host#P#p20200204.ibd     log_archive_service#P#p20190817.ibd  log_archive_service#P#p20200314.ibd
data_bin#P#p20190706.ibd  data_bin#P#p20200201.ibd      log_archive_host#P#p20190710.ibd  log_archive_host#P#p20200205.ibd     log_archive_service#P#p20190818.ibd  log_archive_service#P#p20200315.ibd
data_bin#P#p20190707.ibd  data_bin#P#p20200202.ibd      log_archive_host#P#p20190711.ibd  log_archive_host#P#p20200206.ibd     log_archive_service#P#p20190819.ibd  log_archive_service#P#p20200316.ibd
data_bin#P#p20190708.ibd  data_bin#P#p20200203.ibd      log_archive_host#P#p20190712.ibd  log_archive_host#P#p20200207.ibd     log_archive_service#P#p20190820.ibd  log_archive_service#P#p20200317.ibd
data_bin#P#p20190709.ibd  data_bin#P#p20200204.ibd      log_archive_host#P#p20190713.ibd  log_archive_host#P#p20200208.ibd     log_archive_service#P#p20190821.ibd  log_archive_service#P#p20200318.ibd
data_bin#P#p20190710.ibd  data_bin#P#p20200205.ibd      log_archive_host#P#p20190714.ibd  log_archive_host#P#p20200209.ibd     log_archive_service#P#p20190822.ibd  log_archive_service#P#p20200319.ibd
data_bin#P#p20190711.ibd  data_bin#P#p20200206.ibd      log_archive_host#P#p20190715.ibd  log_archive_host#P#p20200210.ibd     log_archive_service#P#p20190823.ibd  log_archive_service#P#p20200320.ibd
data_bin#P#p20190712.ibd  data_bin#P#p20200207.ibd      log_archive_host#P#p20190716.ibd  log_archive_host#P#p20200211.ibd     log_archive_service#P#p20190824.ibd  log_archive_service#P#p20200321.ibd
data_bin#P#p20190713.ibd  data_bin#P#p20200208.ibd      log_archive_host#P#p20190717.ibd  log_archive_host#P#p20200212.ibd     log_archive_service#P#p20190825.ibd  log_archive_service#P#p20200322.ibd
data_bin#P#p20190714.ibd  data_bin#P#p20200209.ibd      log_archive_host#P#p20190718.ibd  log_archive_host#P#p20200213.ibd     log_archive_service#P#p20190826.ibd  log_archive_service#P#p20200323.ibd
data_bin#P#p20190715.ibd  data_bin#P#p20200210.ibd      log_archive_host#P#p20190719.ibd  log_archive_host#P#p20200214.ibd     log_archive_service#P#p20190827.ibd  log_archive_service#P#p20200324.ibd
data_bin#P#p20190716.ibd  data_bin#P#p20200211.ibd      log_archive_host#P#p20190720.ibd  log_archive_host#P#p20200215.ibd     log_archive_service#P#p20190828.ibd  log_archive_service#P#p20200325.ibd
data_bin#P#p20190717.ibd  data_bin#P#p20200212.ibd      log_archive_host#P#p20190721.ibd  log_archive_host#P#p20200216.ibd     log_archive_service#P#p20190829.ibd  log_archive_service#P#p20200326.ibd
data_bin#P#p20190718.ibd  data_bin#P#p20200213.ibd      log_archive_host#P#p20190722.ibd  log_archive_host#P#p20200217.ibd     log_archive_service#P#p20190830.ibd  log_archive_service#P#p20200327.ibd
data_bin#P#p20190719.ibd  data_bin#P#p20200214.ibd      log_archive_host#P#p20190723.ibd  log_archive_host#P#p20200218.ibd     log_archive_service#P#p20190831.ibd  log_archive_service#P#p20200328.ibd
data_bin#P#p20190720.ibd  data_bin#P#p20200215.ibd      log_archive_host#P#p20190724.ibd  log_archive_host#P#p20200219.ibd     log_archive_service#P#p20190901.ibd  log_archive_service#P#p20200329.ibd
data_bin#P#p20190721.ibd  data_bin#P#p20200216.ibd      log_archive_host#P#p20190725.ibd  log_archive_host#P#p20200220.ibd     log_archive_service#P#p20190902.ibd  log_archive_service#P#p20200330.ibd
data_bin#P#p20190722.ibd  data_bin#P#p20200217.ibd      log_archive_host#P#p20190726.ibd  log_archive_host#P#p20200221.ibd     log_archive_service#P#p20190903.ibd  log_archive_service#P#p20200331.ibd
data_bin#P#p20190723.ibd  data_bin#P#p20200218.ibd      log_archive_host#P#p20190727.ibd  log_archive_host#P#p20200222.ibd     log_archive_service#P#p20190904.ibd  log_archive_service#P#p20200401.ibd
data_bin#P#p20190724.ibd  data_bin#P#p20200219.ibd      log_archive_host#P#p20190728.ibd  log_archive_host#P#p20200223.ibd     log_archive_service#P#p20190905.ibd  log_archive_service#P#p20200402.ibd
data_bin#P#p20190725.ibd  data_bin#P#p20200220.ibd      log_archive_host#P#p20190729.ibd  log_archive_host#P#p20200224.ibd     log_archive_service#P#p20190906.ibd  log_archive_service#P#p20200403.ibd
data_bin#P#p20190726.ibd  data_bin#P#p20200221.ibd      log_archive_host#P#p20190730.ibd  log_archive_host#P#p20200225.ibd     log_archive_service#P#p20190907.ibd  log_archive_service#P#p20200404.ibd
data_bin#P#p20190727.ibd  data_bin#P#p20200222.ibd      log_archive_host#P#p20190731.ibd  log_archive_host#P#p20200226.ibd     log_archive_service#P#p20190908.ibd  log_archive_service#P#p20200405.ibd
data_bin#P#p20190728.ibd  data_bin#P#p20200223.ibd      log_archive_host#P#p20190801.ibd  log_archive_host#P#p20200227.ibd     log_archive_service#P#p20190909.ibd  log_archive_service#P#p20200406.ibd
data_bin#P#p20190729.ibd  data_bin#P#p20200224.ibd      log_archive_host#P#p20190802.ibd  log_archive_host#P#p20200228.ibd     log_archive_service#P#p20190910.ibd  log_archive_service#P#p20200407.ibd
data_bin#P#p20190730.ibd  data_bin#P#p20200225.ibd      log_archive_host#P#p20190803.ibd  log_archive_host#P#p20200229.ibd     log_archive_service#P#p20190911.ibd  log_archive_service#P#p20200408.ibd
data_bin#P#p20190731.ibd  data_bin#P#p20200226.ibd      log_archive_host#P#p20190804.ibd  log_archive_host#P#p20200301.ibd     log_archive_service#P#p20190912.ibd  log_archive_service#P#p20200409.ibd

@MikeW83
Copy link
Author

MikeW83 commented Jun 24, 2020

I have extended the disk in my VMware environment and after few days same problem. Full again.

@MikeW83
Copy link
Author

MikeW83 commented Jun 26, 2020

Is there any idea or solution? I can't expand the volume every 2-3 days. That's not possible and not usable.

@Sims24
Copy link

Sims24 commented Jun 26, 2020

hi @MikeW83

This is your database partition.

Can you share a screenshot of Administration > Parameters > [Perfdata]Options menu please ?

Also, provide the content of /var/log/centreon/centreon-partitioning.log

Thanks

@Sims24 Sims24 added kind/question and removed status/new New tickets labels Jun 26, 2020
@MikeW83
Copy link
Author

MikeW83 commented Jun 29, 2020

Sorry, but I can't log in anymore to my system.

/var/log/centreon/centreon-partitioning.log

[Sun, 14 Jun 20 04:00:01 +0200] PARTITIONING STARTED
[Sun, 14 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200624 - Range: 1592949600
[Sun, 14 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.logs : 20200624 - Range: 1592949600
[Sun, 14 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200624 - Range: 1592949600
[Sun, 14 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200624 - Range: 1592949600
[Sun, 14 Jun 20 04:00:01 +0200] PARTITIONING COMPLETED
[Mon, 15 Jun 20 04:00:01 +0200] PARTITIONING STARTED
[Mon, 15 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200625 - Range: 1593036000
[Mon, 15 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.logs : 20200625 - Range: 1593036000
[Mon, 15 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200625 - Range: 1593036000
[Mon, 15 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200625 - Range: 1593036000
[Mon, 15 Jun 20 04:00:01 +0200] PARTITIONING COMPLETED
[Tue, 16 Jun 20 04:00:01 +0200] PARTITIONING STARTED
[Tue, 16 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200626 - Range: 1593122400
[Tue, 16 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.logs : 20200626 - Range: 1593122400
[Tue, 16 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200626 - Range: 1593122400
[Tue, 16 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200626 - Range: 1593122400
[Tue, 16 Jun 20 04:00:02 +0200] PARTITIONING COMPLETED
[Wed, 17 Jun 20 04:00:01 +0200] PARTITIONING STARTED
[Wed, 17 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200627 - Range: 1593208800
[Wed, 17 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.logs : 20200627 - Range: 1593208800
[Wed, 17 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200627 - Range: 1593208800
[Wed, 17 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200627 - Range: 1593208800
[Wed, 17 Jun 20 04:00:01 +0200] PARTITIONING COMPLETED
[Thu, 18 Jun 20 04:00:02 +0200] PARTITIONING STARTED
PHP Warning: file_put_contents(): Only 0 of 220 bytes written, possibly out of free disk space in /usr/share/centreon/www/class/centreonLog.class.php on line 191
[Thu, 18 Jun 20 04:00:02 +0200] Error: cannot get table centreon_storage.data_bin last partition range
[Fri, 19 Jun 20 04:00:02 +0200] PARTITIONING STARTED
PHP Fatal error: Uncaught Exception: SQLSTATE[HY000] [2002] No such file or directory in /usr/share/centreon/www/class/centreonDB.class.php:164
Stack trace:
#0 /usr/share/centreon/cron/centreon-partitioning.php(47): CentreonDB->__construct('centreon')
#1 {main}
thrown in /usr/share/centreon/www/class/centreonDB.class.php on line 164
[Sat, 20 Jun 20 04:00:01 +0200] PARTITIONING STARTED
PHP Warning: file_put_contents(): Only 0 of 220 bytes written, possibly out of free disk space in /usr/share/centreon/www/class/centreonLog.class.php on line 191
[Sat, 20 Jun 20 04:00:01 +0200] Error: cannot get table centreon_storage.data_bin last partition range
[Sun, 21 Jun 20 04:00:01 +0200] PARTITIONING STARTED
PHP Fatal error: Uncaught Exception: SQLSTATE[HY000] [2002] Connection refused in /usr/share/centreon/www/class/centreonDB.class.php:164
Stack trace:
#0 /usr/share/centreon/cron/centreon-partitioning.php(47): CentreonDB->__construct('centreon')
#1 {main}
thrown in /usr/share/centreon/www/class/centreonDB.class.php on line 164
[Mon, 22 Jun 20 04:00:01 +0200] PARTITIONING STARTED
PHP Fatal error: Uncaught Exception: SQLSTATE[HY000] [2002] Connection refused in /usr/

/var/log/centreon/centreon-partitioning.log-20200614

[Sun, 07 Jun 20 04:00:01 +0200] PARTITIONING STARTED
[Sun, 07 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200617 - Range: 1592344800
[Sun, 07 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.logs : 20200617 - Range: 1592344800
[Sun, 07 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200617 - Range: 1592344800
[Sun, 07 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200617 - Range: 1592344800
[Sun, 07 Jun 20 04:00:02 +0200] PARTITIONING COMPLETED
[Mon, 08 Jun 20 04:00:02 +0200] PARTITIONING STARTED
[Mon, 08 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200618 - Range: 1592431200
[Mon, 08 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.logs : 20200618 - Range: 1592431200
[Mon, 08 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200618 - Range: 1592431200
[Mon, 08 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200618 - Range: 1592431200
[Mon, 08 Jun 20 04:00:02 +0200] PARTITIONING COMPLETED
[Tue, 09 Jun 20 04:00:01 +0200] PARTITIONING STARTED
[Tue, 09 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200619 - Range: 1592517600
[Tue, 09 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.logs : 20200619 - Range: 1592517600
[Tue, 09 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200619 - Range: 1592517600
[Tue, 09 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200619 - Range: 1592517600
[Tue, 09 Jun 20 04:00:02 +0200] PARTITIONING COMPLETED
[Wed, 10 Jun 20 04:00:01 +0200] PARTITIONING STARTED
[Wed, 10 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200620 - Range: 1592604000
[Wed, 10 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.logs : 20200620 - Range: 1592604000
[Wed, 10 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200620 - Range: 1592604000
[Wed, 10 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200620 - Range: 1592604000
[Wed, 10 Jun 20 04:00:02 +0200] PARTITIONING COMPLETED
[Thu, 11 Jun 20 04:00:02 +0200] PARTITIONING STARTED
[Thu, 11 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200621 - Range: 1592690400
[Thu, 11 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.logs : 20200621 - Range: 1592690400
[Thu, 11 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200621 - Range: 1592690400
[Thu, 11 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200621 - Range: 1592690400
[Thu, 11 Jun 20 04:00:02 +0200] PARTITIONING COMPLETED
[Fri, 12 Jun 20 04:00:01 +0200] PARTITIONING STARTED
[Fri, 12 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200622 - Range: 1592776800
[Fri, 12 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.logs : 20200622 - Range: 1592776800
[Fri, 12 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200622 - Range: 1592776800
[Fri, 12 Jun 20 04:00:01 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200622 - Range: 1592776800
[Fri, 12 Jun 20 04:00:02 +0200] PARTITIONING COMPLETED
[Sat, 13 Jun 20 04:00:02 +0200] PARTITIONING STARTED
[Sat, 13 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.data_bin : 20200623 - Range: 1592863200
[Sat, 13 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.logs : 20200623 - Range: 1592863200
[Sat, 13 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_host : 20200623 - Range: 1592863200
[Sat, 13 Jun 20 04:00:02 +0200][updateParts] Create new part for table centreon_storage.log_archive_service : 20200623 - Range: 1592863200
[Sat, 13 Jun 20 04:00:02 +0200] PARTITIONING COMPLETED

The second log is on a file which has not been archived.

Best regards, Michael

@MikeW83
Copy link
Author

MikeW83 commented Jul 2, 2020

I have now extended again (+20GiB) to give you the information of the parameters:

image

I have never changed any settings.
I hope that 20GiB are not filled over the weekend. Actually we don't have any monitoring.

@MikeW83
Copy link
Author

MikeW83 commented Jul 3, 2020

The Disk is full again. About 1 GiB within 25 Minutes has been generated.

@lpinsivy
Copy link
Contributor

Hi @MikeW83 did you check the documentation about disk size prerequisites ?

Do you have enough space for all partitions?

@MikeW83
Copy link
Author

MikeW83 commented Jul 14, 2020

Hi @MikeW83 did you check the documentation about disk size prerequisites ?

Do you have enough space for all partitions?

It has been installed with the ISO file. so i only have 2 logical volumes. And the root volume is always running out of space. I had to shut down the server and I'm thinking about to switch to another solution. Because in my other request I described that I can't import the configuration into a fresh installation. It's generating 1GiB within 25 minutes.... that's not normal. Before the update there was no problem with disk space.

@MikeW83
Copy link
Author

MikeW83 commented Jul 15, 2020

We had to recover the whole machine from a backup before the update. Now the machine is up & running and no extreme data growing is found. And everything is now oin Version 19.10.14 after doing a proper upgrade of all components with "yum upgrade".

@vins-ks
Copy link

vins-ks commented Dec 7, 2020

Hi all,
I am in version 20.04, with 350 hosts, my 160gb disk is saturated and is getting bigger and bigger every day. did you manage to find the issue?
Thanks for your help.

Br,

@alanzkorner
Copy link

alanzkorner commented May 14, 2021

Hello,

Is there a fix for this issue. I upgraded to 20.04.12 and the issue is present . I have reduced the Retention duration for partitioning in options to 300 days, did not help, will reducing this help ? if so will I loose old stats ?

Alan

@t3chbg
Copy link

t3chbg commented Nov 15, 2021

Hi,

We have experiencing the same problem after an upgrade from 20.04 to 21.04 - MYSQL partition has filled up and centreon broker also filled up very quickly after restarting cbd. Unable to get into access centreon web interface.
image

@watchix
Copy link

watchix commented Jan 12, 2022

Hi,

We are having the same problem ... since upgrade 19.10 to 21.04
Initial install from ISO.
Upgraded last week...
image

image

Regards,

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

8 participants