Generated constraint causes infinite recursion

[derrell]

Using asn1c from github, master as of 11 Nov 2020, hash 00fa516

To replicate:

In an otherwise empty directory, create test.asn:

TestModule DEFINITIONS ::= BEGIN

Test ::= SEQUENCE {
--    foo INTEGER (-2..MAX),
--    bar INTEGER (-1..MAX),
--    biz INTEGER (1..MAX),
--    buzz INTEGER (2..MAX),
    baz INTEGER (0..MAX)
}

END

Create main program test.c

#include "Test.h"

int
validate_constraints(
    asn_TYPE_descriptor_t *type_descriptor,
    const void *struct_ptr)
{
    char            error_buffer[128];
    size_t          error_length = sizeof(error_buffer);
    const int       return_value =
        asn_check_constraints(type_descriptor,
                              struct_ptr,
                              error_buffer,
                              &error_length);
    
    if (return_value) {
        perror("asn_check_constraints() failed");
    }
    return (return_value == 0);
}

int
main(int argc, char *argv[])
{
    int             ret;
    Test_t *        test = (Test_t *) calloc(1, sizeof(Test_t));
    
    if (test == NULL)
    {
        perror("calloc() failed");
        exit(EXIT_FAILURE);
    }
    
    ret = validate_constraints(&asn_DEF_Test, test);
    printf("validate_constraints returned %d\n", ret);
}

Compile the ASN.1 file:

asn1c -no-gen-example -gen-PER test.asn

Copy the missing files:

cp /usr/local/share/asn1c/{OCTET,BIT}_STRING_oer.c .

Compile all of the C files:

gcc -g -I. -I/usr/local/share/asn1c -o foo *.c

Run it. It will segfault due to stack exhaustion as a result of infinite recursion:

./foo

Why? In the generated code, the constraint check appears to be improperly generated when the minimum value is 0 or -1 and the maximum value is MAX. Specifically, compiling the above test.asn generates this constraint checking code:

static int
baz_2_constraint(const asn_TYPE_descriptor_t *td, const void *sptr,
			asn_app_constraint_failed_f *ctfailcb, void *app_key) {
	unsigned long value;
	
	if(!sptr) {
		ASN__CTFAIL(app_key, td, sptr,
			"%s: value not given (%s:%d)",
			td->name, __FILE__, __LINE__);
		return -1;
	}
	
	value = *(const unsigned long *)sptr;
	
	if(1 /* No applicable constraints whatsoever */) {
		(void)value; /* Unused variable */
		/* Nothing is here. See below */
	}
	
	return td->encoding_constraints.general_constraints(td, sptr, ctfailcb, app_key);
}

/*
 * This type is implemented using NativeInteger,
 * so here we adjust the DEF accordingly.
 */
static int
memb_baz_constraint_1(const asn_TYPE_descriptor_t *td, const void *sptr,
			asn_app_constraint_failed_f *ctfailcb, void *app_key) {
	unsigned long value;
	
	if(!sptr) {
		ASN__CTFAIL(app_key, td, sptr,
			"%s: value not given (%s:%d)",
			td->name, __FILE__, __LINE__);
		return -1;
	}
	
	value = *(const unsigned long *)sptr;
	
	if(1 /* No applicable constraints whatsoever */) {
		(void)value; /* Unused variable */
		/* Nothing is here. See below */
	}
	
	return td->encoding_constraints.general_constraints(td, sptr, ctfailcb, app_key);
}

In memb_baz_constraint_1, decides there's nothing to do, and its last line calls td->encoding_constraints.general_constraints which points to memb_baz_constraint_2. All good so far. In memb_baz_constraint_2, it also decides there's nothing to do (see note below), and calls td->encoding_constraints.general_constraints which also points to memb_baz_constraint_2, i.e., a recursive call to itself.

Note that one of these two functions is incorrectly deciding that there's nothing to do. Although there's nothing to do for MAX, there should be something to do for 0.

I tracked this down part way. In asn1c_constraint.c, function emit_range_comparison_code is called from line 193, where value_unassigned is true, so the third argument is 0, which corresponds to the 128-bit argument natural_start. In the early code, deciding the value of ignore_left, natural_start != -1 is true, and range->left.value is in fact equal to natural_start (0), so ignore_left is true. This seems incorrect. It seems that value passed in, 0, for natural_start, is incorrect but I may be off base there.