verify_both_logs_and_results.md

How to verify both logs and results

Contents

• The Problem
  • Sample Logs
• Incorrect Method
• Solution #1) Combine Result with Log File
• Solution #2) Two seperate approved files in one test
  • The Approved Log file
  • The Approved Result file
• Solution #3) Split into Two Tests
  • 1. Test Only the Logs
  • 2. Test Only the Results

The Problem

We have a function named load_person().
We want to verify the logs from this call as well as the person object that is returned.

Sample Logs

root INFO
  connecting to the database
root INFO
  querying a table
root INFO
  closing the database

^{snippet source | anchor}

Sample Results

{
    "first_name":"Britney",
    "last_name": "Spears",
    "profession":"Singer"    
}

^{snippet source | anchor}

Incorrect Method

By default, ApprovalTests allows only one verification per test method.
The following will not yield the expected outcome:

def test_load_person_with_logs():
    with verify_logging():
        verify(load_person())

Here are a couple of solutions (details below):

1. Combine Result with Log File
2. Two seperate approved files in one test
3. Split into Two Tests

Solution #1) Combine Result with Log File

With this approach, we leverage the logging mechanism to also verify the output.

def test_load_person_logs_and_results():
    with verify_logging():
        logging.info(f"result = {load_person()}")

^{snippet source | anchor}

This will create a single approved file, combining both the logs and the result:

test_load_person_logs_and_results.approved.txt

root INFO
  connecting to the database
root INFO
  querying a table
root INFO
  closing the database
root INFO
  result = {
    "first_name":"Britney",
    "last_name": "Spears",
    "profession":"Singer"    
}

^{snippet source | anchor}

Solution #2) Two seperate approved files in one test

This method introduces the .logging extension to generate two distinct .approved files.

def test_load_person_logs_and_results_separately():
    with verify_logging(options=NamerFactory.with_parameters("logging")):
        verify(load_person())

^{snippet source | anchor}

This will create the following files:

The Approved Log file

test_load_person_logs_and_results_separately.logging.approved.txt

root INFO
  connecting to the database
root INFO
  querying a table
root INFO
  closing the database

^{snippet source | anchor}

and

The Approved Result file

test_load_person_logs_and_results_separately.approved.txt

{
    "first_name":"Britney",
    "last_name": "Spears",
    "profession":"Singer"    
}

^{snippet source | anchor}

Solution #3) Split into Two Tests

This option divides the verification into two distinct tests: one for logs and another for results.

1. Test Only the Logs

def test_load_person_logs():
    with verify_logging():
        load_person()

^{snippet source | anchor}

This will create the following file:
test_load_person_logs.approved.txt

root INFO
  connecting to the database
root INFO
  querying a table
root INFO
  closing the database

^{snippet source | anchor}

2. Test Only the Results

def test_load_person_results():
    verify(load_person())

^{snippet source | anchor}

This will create the following file:
test_load_person_results.approved.txt

{
    "first_name":"Britney",
    "last_name": "Spears",
    "profession":"Singer"    
}

^{snippet source | anchor}