Exception in docker.py when containers are exiting

[graemej]

I think there's a race in collecting cgroup stats when a container exits, at which point the memory.stat files will disappear (afiak). Currently the logs identify this a Docker 0.9 / 0.10 issue which I don't think is the case any longer as we're on the Docker 1.0 release:

Client version: 1.0.0
Client API version: 1.12
Go version (client): go1.2.1
Git commit (client): 63fe64c
Server version: 1.0.0
Server API version: 1.12
Go version (server): go1.2.1
Git commit (server): 63fe64c

The exception winds up looking like this. Could probably do a simple check on the container to see if it is still running before throwing. This will interrupt the current check cycle, maybe catching and going on with the next container would make sense.

IOError: Can't open /sys/fs/cgroup/memory/docker/02b07c9acbd8d936fefe7fc54d1f92982588eccdfed3acab56baaa49de1cf7d5/memory.stat. If you are using Docker 0.9 or 0.10, it is a known bug in Do
cker fixed in Docker 0.11
2014-06-17 14:40:43 UTC | INFO | dd.collector | checks.collector(collector.py:347) | Finished run #60. Collection time: 4.13s. Emit time: 0.03s
2014-06-17 14:42:20 UTC | ERROR | dd.collector | checks.docker(__init__.py:515) | Check 'docker' instance #0 failed
Traceback (most recent call last):
  File "/usr/share/datadog/agent/checks/__init__.py", line 506, in run
    self.check(copy.deepcopy(instance))
  File "/usr/share/datadog/agent/checks.d/docker.py", line 185, in check
    stats = self._parse_cgroup_file(stat_file)
  File "/usr/share/datadog/agent/checks.d/docker.py", line 287, in _parse_cgroup_file
    raise IOError("Can't open %s. If you are using Docker 0.9 or 0.10, it is a known bug in Docker fixed in Docker 0.11" % file_)
IOError: Can't open /sys/fs/cgroup/memory/docker/969fc2e6ced1697aa2cd139be09ecc6c359d76edc20f8010b56f546de03c48ce/memory.stat. If you are using Docker 0.9 or 0.10, it is a known bug in Do
cker fixed in Docker 0.11