TestJsonAttribute

Especially when the developer needs to create a large number of test cases, the organization/management of these test cases can be daunting. And while it is possible to hard-code test input parameters and expected values, this practice often leads to substantial lines of code in unit/integration tests.

To keep unit/integration tests as streamlined as possible, the developer should externalize the test input parameters and expected values. One option is to place test parameters and expected results in individual or structured files in the test project. Such an approach provides a significant improvement in test case organization; however, creating and editing the files can be a daunting task, especially if multiple test cases are included in a single file.

Another option is to put test parameters and expected results in a database in a format that is easily queried. This is the approach taken by the current library. The EDennis.NetCoreTestingUtilities library only addresses retrieval of test records during unit/integration testing. For creation of test records, one additional library is needed:

• EDennis.MigrationsExtensions -- which creates a TestJson table and stored procedures for inserting and viewing test records

TestJsonAttribute Class

The TestJsonAttribute class provides developers with the ability to obtain test input parameters and expected results from a database. The class works with XUnit and SQL Server.

The TestJsonAttribute class and its supporting classes do several things behind the scenes, including:

• Retrieving and caching one or more set of test records from a TestJson table
• Building a parameterized query from the TestJsonAttribute parameters supplied to a test method
• Retrieving all records (input parameters and expected result) for a particular test case
• Supplying typed input parameters and expected results as structured data back to test methods
• Deserializing data from test records into typed objects, arrays, or primitive values
• Throwing informative exceptions when one or more test records are missing from the database

Constructor

The TestJsonAttribute class extends Xunit's DataAttribute class, which provides developers with a way of supplying parameterized test cases to test methods. The constructor for the TestJsonAttribute class takes a large number of parameters:

Constructor Parameters

Name	Description
string databaseName	The name of the database (often the same as the application's database)
string projectName	The name of the project under test
string className	The name of the class under test
string methodName	The name of the method under test
string testScenario	A top-level category or significant variation of the test
string testCase	A label for the repetition of a test with different input parameters
string serverName	the name of the database server (default: (LocalDb)\MSSQLLocalDb)
string testJsonSchema	the name of test schema (default: _)
string testJsonTable	the name of test table (default: TestJson)

Extending TestJsonAttribute To Reduce Constructor Arity

TestJsonAttribute requires a large number of constructor arguments, most of which will have the same value for test methods in the same test class. For this reason, it is often helpful to create one or more subclasses of TestJsonAttribute where most of the constructor arguments are hard-coded. In essence, this is partial application applied to a constructor.

As a strategy, the developer could embed within a test class an internal subclass of TestJsonAttribute where all but the methodName, testScenario, and testCase arguments are hard-coded.

Example

public class SomeTestClass {
    //...
    
    //note: this class can be called anything
    internal class TestJson_ : TestJsonAttribute {
        public TestJson_(string methodName, string testScenario, string testCase) 
            : base("ColorDb", "EDennis.Samples.ColorDb", "ColorRepo", 
                  methodName, testScenario, testCase, \* defaults for other params *\) {
        }
    }
   //... test methods to follow
}

Xunit Test Method Setup

Each Xunit test method will have a '[Theory]attribute and one instance of[TestJson_]` (or whatever subclass has extended TestJsonAttribute) for every test scenario/test case combination.

[TestJson_] Arguments

The arguments to [TestJson_] (the subclassed [TestJson]) will be

(a) the name of the method under test (b) the name of the test scenario (which could be an empty string, if not needed) (c) the name of the test case (which could be arbitrary, such as "A" or "B," or meaningful, such as the value of a primary or business key).

Test Method Parameters

Test method parameters for a method using [TestJson] or [TestJson_] are always the following:

Name	Description
string t	A label for the test case (auto-generated). t is used to make Test Explorer nodes more compact
JsonTestCase jsonTestCase	An object holding an array input parameters and expected results

Developers familiar with XUnit [InlineData] may question the discrepancy between the [TestJson_] arguments and the test method parameters. (With [InlineData] there is a correspondence between attribute arguments and method parameters). In reality, the discrepancy actually makes things easier for the developer. The JsonTestCase class has a GetObject method that makes it trivial to retrieve the expected result and all of the input parameters and cast these values to the appropriate types.

Example Test Method

[Theory]
[TestJson_("MethodA", "TestScenarioA", "TestCaseA")]
[TestJson_("MethodA", "TestScenarioA", "TestCaseB")]
[TestJson_("MethodA", "TestScenarioB", "TestCaseA")]
[TestJson_("MethodA", "TestScenarioB", "TestCaseB")]
public void MethodA(string t, JsonTestCase jsonTestCase) {
    Output.WriteLine($"Test case: {t}"); //write test case label to output

    var input = jsonTestCase.GetObject<int>("Input"); //retrieve an input parameter
    var expected = jsonTestCase.GetObject<List<Person>>("Expected"); //retrieve expected result
    var actual = Repo.GetById(input); //using EDennis.AspNetCore.Base

    Assert.True(actual.IsEqualOrWrite(expected, PROPS_TO_IGNORE, Output)); //using EDennis.NetCoreTestingUtilities
}

Generating JSON for Expected Results and Inputs

The success of the TestJson approach relies upon the creation of JSON objects or arrays as expected results and sometimes as test inputs (e.g., when testing creation or updating of records). There are several ways to create these JSON objects or arrays. For example, one could ...

• Hand-code the JSON. For simple JSON structures, this could be relatively easy to do; however, for more complicated objects or arrays, this approach could quickly become cumbersome.
• Save the actual results of a test as JSON for use as the expected results. This approach is very easy to do, but it does introduce risk that the expected results are not accurate.
• Generate JSON from test data using a tool. For example, if the developer has a set of base test data stored in a SQL Server database, the developer could use the built-in FOR JSON clause to generate JSON.
  • EDennis.MigrationsExtensions provides some stored procedures that make inserting JSON data (and even primitive values) into a TestJson table easier.
  • There are many examples of this FOR-JSON approach in the sample projects in EDennis.AspNetCore.Base.
  • NOTE: Both in this solution and in EDennis.AspNetCore.Base, there is an included PowerShell script (ExecuteTestSql.ps1) that allows developers to execute all .sql files in all test projects in a solution. This script is especially helpful when the test cases are stored in LocalDb and a developer other than the author of the .sql files wishes to run unit/integration tests that use test data from the database.

Example .sql File for Generating TestJson Test Records

-- from EDennis.Samples.Hr.InternalApi.Tests in EDennis.AspNetCore.Base solution
use hr;

declare @FirstName varchar(30) = 'Curly'

declare @Id int
select @Id = max(id) + 1 from Employee

declare @Input varchar(max) = 
( 
	select @FirstName FirstName
	for json path, include_null_values, without_array_wrapper
);

begin transaction
insert into Employee(FirstName)
	select @FirstName;

declare @Expected varchar(max) = 
(
	select * from Employee
	where Id = @Id
	for json path, include_null_values, without_array_wrapper
);

rollback transaction
exec _.ResetIdentities

exec _.SaveTestJson 'EDennis.Samples.Hr.InternalApi1', 'EmployeeRepo', 'Create','CreateAndGet',@FirstName,'Id', @Id
exec _.SaveTestJson 'EDennis.Samples.Hr.InternalApi1', 'EmployeeRepo', 'Create','CreateAndGet',@FirstName,'Input', @Input
exec _.SaveTestJson 'EDennis.Samples.Hr.InternalApi1', 'EmployeeRepo', 'Create','CreateAndGet',@FirstName,'Expected', @Expected

exec  _.GetTestJson 'EDennis.Samples.Hr.InternalApi1', 'EmployeeRepo', 'Create','CreateAndGet',@FirstName