Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse tuple types and expressions #93

Merged
merged 1 commit into from
Jul 12, 2023
Merged

Parse tuple types and expressions #93

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 16 additions & 7 deletions specs/src/lang/language_primitives.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Identifiers have the following syntax:
<ident> ::= _?[A-Za-z][A-Za-z0-9]* % excluding keywords
```

A number of keywords are reserved and cannot be used as identifiers. The keywords are: `bool`, `constraint`, `else`, `false`, `real`, `fn`, `if`, `int`, `let`, `maximize`, `minimize`, `satisfy`, `solve`, `true`, `type`.
A number of keywords are reserved and cannot be used as identifiers. The keywords are: `bool`, `constraint`, `else`, `false`, `real`, `fn`, `if`, `int`, `let`, `maximize`, `minimize`, `satisfy`, `solve`, `true`.

## High-level Intent Structure

Expand Down Expand Up @@ -138,7 +138,8 @@ Expressions represent values and have the following syntax:
| <int-literal>
| <real-literal>
| <string-literal>
| <tuple-literal>
| <tuple-expr>
| <tuple-index-expr>
| <if-expr>
| <call-expr>
```
Expand Down Expand Up @@ -249,15 +250,23 @@ let string = "first line\
third line";
```

#### Tuple Literals
#### Tuple Expressions and Tuple Indexing Expressions

Tuple literals are written as:
Tuple Expressions are written as:

```ebnf
<tuple-literal> ::= "(" <expr> "," [ <expr> "," ... ] ")"
<tuple-expr> ::= "(" <expr> "," [ <expr> "," ... ] ")"
```

For example: `let t = (5, 3, "foo")`;
For example: `let t = (5, 3, "foo");`.

Tuple indexing expressions are written as:

```ebnf
<tuple-index-expr> ::= <expr-atom> "." [0-9]+
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's an expr-atom? It can be any expression which has a tuple type, which might be a fn call, identifier or other tuple index. So it should just be expr?

Copy link
Contributor Author

@mohammadfawaz mohammadfawaz Jul 12, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I will restructure this. See #99

```

For example: `let second = t.1;` which extracts the second element of tuple `t` and stores it into `second`.

#### "If" Expressions

Expand All @@ -279,7 +288,7 @@ Call expressions are used to call functions and have the following syntax:
<call-expr> ::= <ident> "(" ( <expr> "," ... ) ")"
```

For example, `x = foo(5, 2);`
For example: `x = foo(5, 2);`.

The type of the expressions passed as arguments must match the argument types of the called function. The return type of the function must also be appropriate for the calling context.

Expand Down
6 changes: 6 additions & 0 deletions yurtc/src/ast.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ pub(super) enum Type {
Int,
Bool,
String,
Tuple(Vec<Type>),
}

#[derive(Clone, Debug, PartialEq)]
Expand All @@ -62,6 +63,11 @@ pub(super) enum Expr {
},
Block(Block),
If(IfExpr),
Tuple(Vec<Expr>),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is a tuple a kind of immediate or literal?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could have something more complex such as:

let tuple = (
    { ... }, // code block
    if cond { .. } else { .. }, // if expressions
    foo(), // function call,
    etc.
);

I suppose adding tests for these would be quite useful.

TupleIndex {
tuple: Box<Expr>,
index: usize,
},
}

#[derive(Clone, Debug, PartialEq)]
Expand Down
6 changes: 6 additions & 0 deletions yurtc/src/error.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ pub(super) enum ParseError<'a> {
"type annotation or initializer needed for decision variable \"{}\"", name.0
)]
UntypedDecisionVar { span: Span, name: ast::Ident },
#[error("Invalid integer value \"{}\" for tuple index", index)]
InvalidIntegerForTupleIndex { span: Span, index: Token<'a> },
#[error("Invalid value \"{}\" for tuple index", index)]
InvalidTupleIndex { span: Span, index: Token<'a> },
}

fn format_expected_found_error<'a>(
Expand Down Expand Up @@ -120,6 +124,8 @@ impl<'a> CompileError<'a> {
ParseError::ExpectedFound { span, .. } => span.clone(),
ParseError::KeywordAsIdent { span, .. } => span.clone(),
ParseError::UntypedDecisionVar { span, .. } => span.clone(),
ParseError::InvalidIntegerForTupleIndex { span, .. } => span.clone(),
ParseError::InvalidTupleIndex { span, .. } => span.clone(),
},
}
}
Expand Down
28 changes: 18 additions & 10 deletions yurtc/src/lexer.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ pub(super) enum Token<'sc> {
ParenClose,
#[token("->")]
Arrow,
#[token(".")]
Dot,

#[token("real")]
Real,
Expand Down Expand Up @@ -150,6 +152,7 @@ impl<'sc> fmt::Display for Token<'sc> {
Token::ParenOpen => write!(f, "("),
Token::ParenClose => write!(f, ")"),
Token::Arrow => write!(f, "->"),
Token::Dot => write!(f, "."),
Token::Real => write!(f, "real"),
Token::Int => write!(f, "int"),
Token::Bool => write!(f, "bool"),
Expand Down Expand Up @@ -258,12 +261,17 @@ fn lex_one_success(src: &str) -> Token<'_> {
toks[0].0.clone()
}

#[cfg(test)]
fn lex_one_error(src: &str) -> CompileError {
// Tokenise src, assume a single error.
let (_, errs) = lex(src);
assert_eq!(errs.len(), 1, "Testing for single error only.");
errs[0].clone()
#[test]
fn control_tokens() {
assert_eq!(lex_one_success(":"), Token::Colon);
assert_eq!(lex_one_success(";"), Token::Semi);
assert_eq!(lex_one_success(","), Token::Comma);
assert_eq!(lex_one_success("{"), Token::BraceOpen);
assert_eq!(lex_one_success("}"), Token::BraceClose);
assert_eq!(lex_one_success("("), Token::ParenOpen);
assert_eq!(lex_one_success(")"), Token::ParenClose);
assert_eq!(lex_one_success("->"), Token::Arrow);
assert_eq!(lex_one_success("."), Token::Dot);
}

#[test]
Expand All @@ -275,12 +283,12 @@ fn reals() {
assert_eq!(lex_one_success("0.34"), Token::RealLiteral("0.34"));
assert_eq!(lex_one_success("-0.34"), Token::RealLiteral("-0.34"));
check(
&format!("{:?}", lex_one_error(".34")),
expect_test::expect![[r#"Lex { span: 0..1, error: InvalidToken }"#]],
&format!("{:?}", lex(".34")),
expect_test::expect![[r#"([(Dot, 0..1), (IntLiteral("34"), 1..3)], [])"#]],
);
check(
&format!("{:?}", lex_one_error("12.")),
expect_test::expect!["Lex { span: 2..3, error: InvalidToken }"],
&format!("{:?}", lex("12.")),
expect_test::expect![[r#"([(IntLiteral("12"), 0..2), (Dot, 2..3)], [])"#]],
);
}

Expand Down
162 changes: 151 additions & 11 deletions yurtc/src/parser.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,22 +213,48 @@ fn expr<'sc>() -> impl Parser<Token<'sc>, ast::Expr, Error = ParseError<'sc>> +
.delimited_by(just(Token::ParenOpen), just(Token::ParenClose));

let call = ident()
.then(args)
.then(args.clone())
.map(|(name, args)| ast::Expr::Call { name, args });

let tuple = args.map(ast::Expr::Tuple);

let atom = choice((
immediate().map(ast::Expr::Immediate),
unary_op(expr.clone()),
code_block_expr(expr.clone()).map(ast::Expr::Block),
if_expr(expr.clone()),
call,
tuple,
ident().map(ast::Expr::Ident),
));

comparison_op(additive_op(multiplicative_op(atom)))
comparison_op(additive_op(multiplicative_op(tuple_index(atom))))
})
}

fn tuple_index<'sc, P>(
parser: P,
) -> impl Parser<Token<'sc>, ast::Expr, Error = ParseError<'sc>> + Clone
where
P: Parser<Token<'sc>, ast::Expr, Error = ParseError<'sc>> + Clone,
{
// This extracts a `usize` index. Fails for everything else (therefore, `t.0.0` is not
// supported - but `t.0 .0` is fine).
mohammadfawaz marked this conversation as resolved.
Show resolved Hide resolved
let index = filter_map(|span, token| match token {
Token::IntLiteral(num_str) => num_str
.parse::<usize>()
.map_err(|_| ParseError::InvalidIntegerForTupleIndex { span, index: token }),
_ => Err(ParseError::InvalidTupleIndex { span, index: token }),
});

parser
.then(just(Token::Dot).ignore_then(index).repeated())
.foldl(|expr, index| ast::Expr::TupleIndex {
tuple: Box::new(expr),
index,
})
}

fn multiplicative_op<'sc, P>(
parser: P,
) -> impl Parser<Token<'sc>, ast::Expr, Error = ParseError<'sc>> + Clone
Expand Down Expand Up @@ -321,12 +347,20 @@ fn ident<'sc>() -> impl Parser<Token<'sc>, ast::Ident, Error = ParseError<'sc>>
}

fn type_<'sc>() -> impl Parser<Token<'sc>, ast::Type, Error = ParseError<'sc>> + Clone {
choice((
just(Token::Real).to(ast::Type::Real),
just(Token::Int).to(ast::Type::Int),
just(Token::Bool).to(ast::Type::Bool),
just(Token::String).to(ast::Type::String),
))
recursive(|type_| {
let tuple = type_
.separated_by(just(Token::Comma))
.allow_trailing()
.delimited_by(just(Token::ParenOpen), just(Token::ParenClose));

choice((
just(Token::Real).to(ast::Type::Real),
just(Token::Int).to(ast::Type::Int),
just(Token::Bool).to(ast::Type::Bool),
just(Token::String).to(ast::Type::String),
tuple.map(ast::Type::Tuple),
))
})
}

fn immediate<'sc>() -> impl Parser<Token<'sc>, ast::Immediate, Error = ParseError<'sc>> + Clone {
Expand Down Expand Up @@ -373,6 +407,25 @@ fn check(actual: &str, expect: expect_test::Expect) {
expect.assert_eq(actual);
}

#[test]
fn types() {
check(&run_parser!(type_(), "int"), expect_test::expect!["Int"]);
check(&run_parser!(type_(), "real"), expect_test::expect!["Real"]);
check(&run_parser!(type_(), "bool"), expect_test::expect!["Bool"]);
check(
&run_parser!(type_(), "string"),
expect_test::expect!["String"],
);
check(
&run_parser!(type_(), "(int, real, string)"),
expect_test::expect!["Tuple([Int, Real, String])"],
);
check(
&run_parser!(type_(), "(int, (real, int), string)"),
expect_test::expect!["Tuple([Int, Tuple([Real, Int]), String])"],
);
}

#[test]
fn let_decls() {
check(
Expand Down Expand Up @@ -839,7 +892,7 @@ fn code_blocks() {
check(
&format!("{:?}", run_parser!(let_decl(expr()), "let x = {};")),
expect_test::expect![[
r#""@9..10: found \"}\" but expected \"!\", \"+\", \"-\", \"{\", \"if\", \"var\", \"let\", or \"constraint\"\n""#
r#""@9..10: found \"}\" but expected \"!\", \"+\", \"-\", \"{\", \"(\", \"if\", \"var\", \"let\", or \"constraint\"\n""#
]],
);
}
Expand Down Expand Up @@ -881,6 +934,93 @@ fn if_exprs() {
);
}

#[test]
fn tuple_expressions() {
check(
&run_parser!(expr(), r#"(0,)"#),
expect_test::expect!["Tuple([Immediate(Int(0))])"],
);

check(
&run_parser!(expr(), r#"(0, 1.0, "foo")"#),
expect_test::expect![[
r#"Tuple([Immediate(Int(0)), Immediate(Real(1.0)), Immediate(String("foo"))])"#
]],
);

check(
&run_parser!(expr(), r#"(0, (1.0, "bar"), "foo")"#),
expect_test::expect![[
r#"Tuple([Immediate(Int(0)), Tuple([Immediate(Real(1.0)), Immediate(String("bar"))]), Immediate(String("foo"))])"#
]],
);

check(
&run_parser!(expr(), r#"( { 42 }, if cond { 2 } else { 3 }, foo() )"#),
expect_test::expect![[
r#"Tuple([Block(Block { statements: [], final_expr: Immediate(Int(42)) }), If(IfExpr { condition: Ident(Ident("cond")), then_block: Block { statements: [], final_expr: Immediate(Int(2)) }, else_block: Block { statements: [], final_expr: Immediate(Int(3)) } }), Call { name: Ident("foo"), args: [] }])"#
]],
);

check(
&run_parser!(expr(), r#"t.0 + t.9999999"#),
expect_test::expect![[
r#"BinaryOp { op: Add, lhs: TupleIndex { tuple: Ident(Ident("t")), index: 0 }, rhs: TupleIndex { tuple: Ident(Ident("t")), index: 9999999 } }"#
]],
);

check(
&run_parser!(expr(), r#"(0, 1).0"#),
expect_test::expect![
"TupleIndex { tuple: Tuple([Immediate(Int(0)), Immediate(Int(1))]), index: 0 }"
],
);

check(
&run_parser!(expr(), r#"t.0 .0"#),
expect_test::expect![[
r#"TupleIndex { tuple: TupleIndex { tuple: Ident(Ident("t")), index: 0 }, index: 0 }"#
]],
);

check(
&run_parser!(expr(), r#"foo().0"#),
expect_test::expect![[
r#"TupleIndex { tuple: Call { name: Ident("foo"), args: [] }, index: 0 }"#
]],
);

check(
&run_parser!(expr(), r#"{ (0, 0) }.0"#),
expect_test::expect!["TupleIndex { tuple: Block(Block { statements: [], final_expr: Tuple([Immediate(Int(0)), Immediate(Int(0))]) }), index: 0 }"],
);

check(
&run_parser!(expr(), r#"if true { (0, 0) } else { (0, 0) }.0"#),
expect_test::expect!["TupleIndex { tuple: If(IfExpr { condition: Immediate(Bool(true)), then_block: Block { statements: [], final_expr: Tuple([Immediate(Int(0)), Immediate(Int(0))]) }, else_block: Block { statements: [], final_expr: Tuple([Immediate(Int(0)), Immediate(Int(0))]) } }), index: 0 }"],
);

// This parses because `1 + 2` is an expression, but it should fail in semantic analysis.
check(
&run_parser!(expr(), "1 + 2 .3"),
expect_test::expect!["BinaryOp { op: Add, lhs: Immediate(Int(1)), rhs: TupleIndex { tuple: Immediate(Int(2)), index: 3 } }"],
);

check(
&run_parser!(let_decl(expr()), "let x = t.0xa;"),
expect_test::expect![[r#"
@10..13: Invalid integer value "0xa" for tuple index
"#]],
);

check(
&run_parser!(let_decl(expr()), "let x = t.xx;"),
expect_test::expect![[r#"
@10..12: Invalid value "xx" for tuple index
"#]],
);
}

#[test]
fn basic_program() {
let src = r#"
Expand All @@ -907,7 +1047,7 @@ fn with_errors() {
check(
&run_parser!(yurt_program(), "let low_val: bad = 1.23"),
expect_test::expect![[r#"
@13..16: found "bad" but expected "real", "int", "bool", or "string"
@13..16: found "bad" but expected "(", "real", "int", "bool", or "string"
"#]],
);
}
Expand All @@ -924,7 +1064,7 @@ fn fn_errors() {
check(
&run_parser!(yurt_program(), "fn foo() -> real {}"),
expect_test::expect![[r#"
@18..19: found "}" but expected "!", "+", "-", "{", "if", "var", "let", or "constraint"
@18..19: found "}" but expected "!", "+", "-", "{", "(", "if", "var", "let", or "constraint"
"#]],
);
}
Expand Down