Minimising disk writes

Minimising Disk writes for SD installations

Background

One problem with running MisterHouse on an SD card is that you can only write data to an SD card some finite number of times before parts of the SD card deteriorate and become unwritable. Older or cheaper cards may only be writable 10,000 times in each memory cell, but 100,000 writes seems to be average. In newer cards, the controller in the card itself should spread writes out evenly across the card and detect when cells are going bad, remapping to a newer cell, usually without corrupting data.

The big problem is that writing even one byte to a cell that's already written requires erasing a whole block of cells and then writing the whole block with the changed byte. Block size depends on the card, but you can see your block size with
cat /sys/block/mmcblk0/device/preferred_erase_size In my 16 gig card the block size is 4mb, which means the whole card has only 4000 blocks and if each write causes an erase, that's 400 million writes before the card goes bad. If there is one write per second, the card will fail in 12.68 years.

Realistically, the OS or the card should cache writes and group them together, reducing the frequency of writes. MisterHouse itself will tend to generate some writes about once per minute as it saves objects, and other running processes will also generate some writes as they perform logging, so the actual number of writes should be far less than once per second. Still, since I don't know if I'm using a card that supports 100,000 writes per cell or only 10,000, I've disabled as many writes as possible to the SD card. Here's what I did after installing MisterHouse to /home/pi/MisterHouse:

General recommendations

• Don't buy cheap SD cards, stick to branded items such as Sandisk.
• The larger the disk the more it can spread writes out use 32G or larger.
• Have a comprehensive backup and recovery strategy. Consider having a cloned SD card for a quick recovery.

Tips to reduce MisterHouse disk writes

You can skip this section if you want. I'm guessing a good SD card will last decades under normal MisterHouse usage, but also remember that older SD cards were designed for things like cameras that don't write very frequently. Whether you reduce MisterHouse writes or not, you should at least back up your MisterHouse configuration periodically to avoid losing your custom scripts and setup.

• Disable MisterHouse logging:

  • Edit /home/pi/MisterHouse/misterhouse/bin/mh using a command-line editor like nano or vi, or the GUI editor called Leafpad you can find by clicking the lower left corner of the desktop and looking in the Accessories category.
  • Search for sub print_log
  • Below sub print_log, find a line that reads if ($logfile) {
  • Add return; right above the if ($logfile) { line.

• Disable MisterHouse web server logging:

  • Edit /home/pi/MisterHouse/local/mh.private.ini
  • Add this to the bottom of the file:no_log = http_local

• Disable object logs By default (since Version 5) MisterHouse logs all changes to object_logs, this can be disabled by adding object_logger_enable = 0 to your mh.ini

• Save objects less frequently than once a minute:

  • Edit /home/pi/MisterHouse/misterhouse/bin/mh
  • Find this line: &object_states_save if $New_Minute and !($Time_Start_time or $Time_Stop_time); # Skip if in accelerated mode
  • Change $New_Minute to $New_Day for daily save, or &new_minute(5) for every 5 minutes. This prevents writing MisterHouse data to your SD card every minute. This may cause problems if MisterHouse crashes but otherwise I don't think it's necessary to write the on/off state of all your devices unless you have some special devices that require extra care. In an Insteon environment, the startup scan should get device states when MisterHouse starts anyway, so with an Insteon setup or any setup where devices can be queried for state I'm not sure if saving state is at all necessary (I could be wrong, but I haven't seen problems).

• Disable nightly update of MisterHouse docs:

  • Visit the MisterHouse web page at http://<IP of Pi>:8080
  • Click MrHouse Home
  • Click Setup MrHouse on left.
  • Click Edit Triggers
  • In the trigger with Name "update the documentation", change Type to "Disabled".