Testing Basics

It's always good to know the basics of how components works and to be able to ensure it's actually working.

Smoke Test

Here's a very simple test to help you understand how and if the services can work together.

Start Application - start up and configure a host using either vagrant or an AWS CloudFormation stack.

Open Kibana - open Kibana (localhost:4567 via vagrant, or the Web Access output of the stack). The Kibana interface should load, but you'll see an error along the lines of: Oops! IndexMissingException[[logstash-*] missing]. This is because there's no log data at all for it to find. Let's get some log messages in there...

Send Some Logs - SSH into the host, then the easiest way to start sending logs is to use one or all files from /var/log/*:

cd /app/app/ && rake logstash:file_to_redis[other,/var/log/syslog]

A couple notes about that:

• The command will import all the log data and continue to poll the file(s) for new messages (press Ctrl-C to stop).
• The other just means we're not using a particular log format, so don't expect fancy message parsing.
• By using logstash:file_to_redis you're also able to test that redis/logstash/elasticsearch are communicating together.

Check Kibana - now there are hopefully log messages showing up in Kibana. Explore it a bit to be sure:

1. Click the 1h time range to update the time frame. Make sure some of the /var/log/* messages are showing up. Note: at first they'll all appear at the same time since it's not parsing the date from individual messages
2. Use the Query Search box to filter messages. Usually searching for info, warn, or sudo will come up with something.
3. Have a look at the table and click on one of the messages to expand it.
4. Make sure the expanded message lists out the fields. Try clicking on the magnifying glass of the @timestamp field to see other messages logged at the same exact time.

Integration Tests

For our various log format types, we've got some tests setup to ensure we're parsing fields properly and to ensure the fields are searchable.

1. Start Application - start up and configure a host using either vagrant or an AWS CloudFormation stack.
2. Login to Environment - login to the environment and cd /app/app/
3. Run Tests
   • You can run all tests with the rake task logstash:test:all_types, or
   • You can run individual tests with logstash:test:type:{log-format-name} (e.g. logstash:test:type:iis_default)