Skip to content

Test Naming Conventions

Gavin Fielder edited this page Jun 25, 2020 · 18 revisions

These are the naming conventions currently used in the included unit_tests.c. You can use this as a reference in knowing what queries to run by name or wildcard search in order to select specific types of tests to run, enable, or disable as needed.

Mandatory Tests (enabled by default)

  • d, i, u, x, X, c, s, p tests start with d_, i_, u_, etc.
  • %% tests start with pct_
  • Precision tests (.) have 'prec' in the name
  • Field width tests usually have 'width' or just 'w' in the name
  • * Tests have star in the name.
  • Mandatory Format flags
    • 0 (zero padding) tests usually have 'zp' in the name
    • - (left justify) tests usually have 'lj' in the name
    • Combined flags will sometimes concatenate the above, e.g. _zplj_. There is no defined order.
  • Tests that start with mix_ are tests that combine multiple specifiers at random. This block is good for testing whether you handle variadic arguments correctly.

Disabled by default

  • nocrash tests start with nocrash_. These tests are specially handled and will always pass provided ft_printf does not crash while executing the test. They are disabled by default, except for tests that start with nocrash_mandatory.

Bonus Tests (disabled by default)

  • All tests for bonuses that are mentioned in the subject start with bonus_
  • All tests for bonuses that are mentioned in the subject, but do not test functionality that is tested by moulinette to validate those bonuses, start with bonus_notrequired_
  • Tests to validate %f that are deemed mandatory for the bonus start with bonus_f.
  • Tests to validate %g that are deemed mandatory for the bonus start with bonus_g.
  • Tests to validate %e that are deemed mandatory for the bonus start with bonus_e.
  • hh h l ll tests start with bonus_length
  • Bonus Format flags
    • # tests start with bonus_af. Other bonus tests that include # have af in the name.
    • + tests start with bonus_as. Other bonus tests that include + have as in the name.
    • ' ' (space padding) tests start with bonus_sp. Other bonus tests that include have sp in the name.
    • Combined flags will sometimes concatenate the above, e.g. _afsp_. There is no defined order.
  • %n tests will probably start with bonus_n_, but these tests, and the capability of running these tests, do not exist yet.

Other Non-Mandatory Tests (Disabled by default)

  • All tests for bonuses that are mentioned in the subject, but do not test functionality that is tested by moulinette to validate those bonuses, start with bonus_notrequired_
  • Tests for the arbitrary argument selector $ (not in the subject) start with argnum_
  • Tests for %o start with notinsubject_o and notinsubject_moul_o
  • Other tests that test functionality that is not in the subject start with notinsubject_. Less commonly, tests may have the _notinsubject_ tag within them.
  • The tests starting with bonus_mix is the mix block from the version on the old curriculum. Unless you validate all of its bonuses, it's probably no longer useful.

%f %g %e Special Tests

  • Tests for %f, %g, or %e which test functionality that is not tested by moulinette, start with bonus_notrequired_f, etc. All %f, %g, or %e tests including these can be selected with the wildcard search "bonus*_f_", etc.
  • Tests for rounding have rnd in the name.
  • Tests for %Lf start with notinsubject_f_L
  • Tests for the special values of %f, %g, or %e (such as infinity, NaN, and negative zero) will have _reserved_values_ in the name.
  • %f, %g, or %e stress tests will have _stress_ in the name.

Other test naming flags

  • The notrequired_ and notmandatory_ tags are otherwise also used either at the start or within the name and are context-dependent with the expectation of being mostly self-explanatory.
  • Tests that throw a '...flag is ignored when...' warning when you use them with printf will have a _ignoreflag in the name. Since this is not undefined behavior per se, they are not necessarily disabled by default.
  • Tests that throw some other format warning when you use them with printf will have throwswarning. They are not necessarily disabled by default.
  • Tests that may have no defined behavior as per printf(3) will have undefbehav in the name. They are explicitly disabled by default.

Notes about the Naming Conventions

These naming conventions are usually followed. There's a lot of tests, so making it more consistent will always be an uphill battle. The first rule is the one most users will care about most, and it is consistent.

This is not necessarily a complete list. Test names aim to be descriptive, so that it is clear what the test is testing, as well as organized so that they are selectable. Looking at unit_tests.c, and the show-disabled-tests script are both tools that can help.

See also Enabling and Disabling Tests and querying by Wildcard Search

Disclaimer about the moul_ tests

The moul_ tests were taken from a version of moulinette that was two versions of the subject ago. Do not make the mistake of thinking the moul_ block is the only mandatory one.