Skip to content

Latest commit

 

History

History
349 lines (259 loc) · 15.2 KB

README.md

File metadata and controls

349 lines (259 loc) · 15.2 KB
Raw

6502 Unit Test executor

example workflow Codacy Badge Awesome

6502 unit test

This tool enables unit testing for MOS Technology 6502 assembly programs on a cross-platform basis.

Table of Contents

Basic Design

Execute Tests on Built-In Emulator

Write Tests with JSON Schema

  • The tool provides JSON Schema document that simplifies test creation.

Evaluate Tests like Modern Frameworks

Supported Testing Features

Testable Program

  • Test targets
    • JSR procedure
    • NMI procedure
    • IRQ procedure
    • Address range
  • Type of interrupts that can occur during testing
    • NMI
    • IRQ
  • Mockable call instructions that set the results in registers and memory
    • JSR
    • JMP

Conditions

  • Setting up device conditions before testing
    Register Memory Stack
    Set value ✔️ ✔️ ✔️
    Mock read value history ✔️
  • Evaluating device conditions after testing
    Register Memory Stack
    Check value ✔️ ✔️ ✔️
    Check read/write count ✔️ ✔️
    Check write value history ✔️
  • Evaluating processor information
    • Cycle count

Assertions

  • Assertion operators that determine how tests are evaluated
    • "eq" (Equal to)
    • "ne" (Not equal to)
    • "gt" (Greater than)
    • "ge" (Greater than or equal to)
    • "lt" (Less than)
    • "le" (Less than or equal to)
    • "anyOf" (Any of)
      Representing logical OR operator
    • Composite operators
      Same as logical AND operator. e.g. "gt"+"lt" for range selection
  • Error handling
    • Write access to readonly memory
      Able to detect write operations to readonly memory.
    • Read access from uninitialized memory
      Able to detect read operations from memory that has never had a write operation.
    • Access to undefined memory
      Able to detect memory operations using addresses outside the segment defined at build time.
    • Access to unauthorized memory
      Able to detect access outside the memory for which authorized area has been defined.
    • Illegal instruction
      Able to detect illegal instruction code execution.

Coverage

  • Testable coverage

Test Classification

  • Aggregating tests for the entire project
  • Common configuration settings for the entire project
  • Grouping of individual tests

Additional Useful Features

  • When a test terminates with an error, its callback trace can be output.
  • Test evaluations can be executed at any memory address.
  • Test ROM images are detected from debug information file.

Getting Started

Prerequisites

This tool is intended for the projects based on [CC65](https://cc65.github.io/).

The easiest way to install CC65 on [Ubuntu](https://ubuntu.com/) linux is by running:

# sudo apt-get install cc65

Verified CA65 version:

# ca65 --version
ca65 V2.18 - Ubuntu 2.19-1

Install

Since this tool repository includes submodules, you must specify the --recurse-submodules option when cloning:

# git clone --recurse-submodules https://github.com/AsaiYusuke/6502_test_executor.git
# cd 6502_test_executor
# make
# ./6502_tester --help

Running Examples

Unit Testing

The simple example project includes many test cases that demonstrate the features:

# cd example
# make
mkdir -p build
ca65 --cpu 6502 --target nes --debug-info -o build/example.o src/example.asm
mkdir -p dist
ld65  -o dist/example.nes --dbgfile dist/example.dbg --config cfg/nes.cfg --obj build/example.o
rm coverage/lcov.info
../6502_tester --debug=dist/example.dbg --coverage=coverage/lcov.info --segment="CODE" --quiet-summary --quiet-ok -t test/ok/customize.configurations.test.json
../6502_tester --debug=dist/example.dbg --coverage=coverage/lcov.info --segment="CODE" --quiet-summary --quiet-ok -t test/ok/definitions.test.json
../6502_tester --debug=dist/example.dbg --coverage=coverage/lcov.info --segment="CODE" --quiet-summary --quiet-ok -t test/ok/error.timeout.test.json
:
All tests passed.

Coverage File

When the project is built, the coverage file is saved in the example/coverage/lcov.info in the case of the example project.

The coverage file can be used to integrate with tools like Coveralls GitHub Action, and more.

The results of the example project can be seen on Coveralls.
Coverage Status

Usage

flowchart LR;
  A(Assembly program);
  C[[CA65]];
  B(Binary);
  D(Debug information);
  V[[Visual Studio Code]];
  S(Test scenario);
  U[[6502 Unit Test executor]];
  R[(Test result)];
  L[(Test coverage)];

  subgraph Build CA65 project
    A --> C --> B & D;
  end

  subgraph Create unit test
    V -- Json schema --> S;
  end

  subgraph Run test
    B & D & S --> U --> R & L;
  end
Loading

Build CA65 Project with Debug Option

Build your 6502 project using the CA65 assembler and LD65 linker with debug information generation enabled.

Create Unit Test

Create test scenario files containing three key items in JSON format:

  • Test target
    The starting address of the test procedure
  • Setup condition
    The settings of the registers and memory before the test
  • Expected condition
    The expected responses of the registers and memory after the test

JSON Schema File

The tool also provides a JSON Schema document that simplifies creating test scenario files.

If you use Visual Studio Code, it will provide formatting error notifications and auto-completion based on JSON Schema.

Run Test

Run the tool with the prepared debug information file and test scenario file:

6502_tester -d <debug information> -t <test scenario>

Test coverage can also be measured. Both the coverage file and the segment names used in the debug information file must be specified to enable coverage reporting.

6502_tester -d <debug information> -t <test scenario> -c <coverage> -s <segment names>

You can find all command-line arguments in the help:

# ./6502_tester --help
  ./6502_tester {OPTIONS}

    6502 test executor

  OPTIONS:

      -h, --help                        Show this help menu.
      -d[DEBUG], --debug=[DEBUG]        Specify the path of the debug
                                        information file used for testing.
      -t[TEST], --test=[TEST]           (REQUIRED)
                                        Specify the path of the test scenario
                                        file.
      -c[COVERAGE],
      --coverage=[COVERAGE]             Specify the path of the coverage file.
      -s[SEGMENT], --segment=[SEGMENT]  Specify the segment names for coverage.
      -i[ID], --id=[ID]                 Specify the test ID.
      --timeout=[TIMEOUT]               Specify the timeout period before the
                                        test becomes an error.
      -q, --quiet                       Do not show any output.
      --quiet-ok                        Do not show the successful output.
      --quiet-fail                      Do not show the failed output.
      --quiet-summary                   Do not show the test summary output.

Note:

Some options can be specified either as command-line arguments or test scenario file (See example).
If both are specified, the values in the test scenario file take precedence.

Test Scenario Examples

Register Conditions

Memory Conditions

Stack Conditions

Processor Conditions

Testable Procedures

Error Handling

Test Classification

Misc

Dependencies

This project uses the following libraries:

License

This project is available under the MIT license. See the LICENSE file for more information.