Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move annotate(struct1, struct2) to IR #3201

Merged
merged 3 commits into from
Mar 22, 2018

Conversation

catoverdrive
Copy link
Contributor

@catoverdrive catoverdrive commented Mar 21, 2018

this moves the struct-level annotate that's in the AST (which calls tstruct1.annotate(tstruct2)) This will allow the python struct-level annotate operation to pass through the IR when able.

@@ -662,8 +662,16 @@ case class Apply(posn: Position, fn: String, args: Array[AST]) extends AST(posn,

def toIR(agg: Option[String] = None): Option[IR] = {
fn match {
case "merge" | "select" | "drop" | "annotate" | "index" =>
case "merge" | "select" | "drop" | "index" =>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cseed why are these specially called out? They should always fail to be found in the function registry nor are they primitive operations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this has to do with incomplete typechecking of the arguments of these methods.

None
case "annotate" =>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this special? is there something wrong with doing primo conversion on it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The AST for Apply("annotate", Seq(structA,structB)) doesn't place limitations on what structB is, since it just uses structA.typ.annotate(structB.typ). I don't want to handle the general case, so I'm only converting in the case that the annotate node looks like Apply("annotate", Seq(structA, StructConstructor(...))), which is I believe enough to handle all of our current uses in the Python.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will something like this work?

def maybeFindIRFunction(fn: String, args: Seq[AST]): Option[IR] = for {
  irArgs <- anyFailAllFail(args.map(_.toIR(agg)))
  ir <- tryPrimOpConversion(args.map(_.`type`).zip(irArgs)).orElse(
    IRFunctionRegistry.lookupFunction(fn, args.map(_.`type`))
      .map { irf => irf(irArgs) })
} yield ir

and

fn match {
  // ...
  case "annotate" =>
    if (!args(1).isInstanceOf[StructConstructor])
      None
    else
      maybeFindIRFunction(fn, args)
  case _ =>
    maybeFindIRFunction(fn, args)

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. I'll change.

None
case "annotate" =>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will something like this work?

def maybeFindIRFunction(fn: String, args: Seq[AST]): Option[IR] = for {
  irArgs <- anyFailAllFail(args.map(_.toIR(agg)))
  ir <- tryPrimOpConversion(args.map(_.`type`).zip(irArgs)).orElse(
    IRFunctionRegistry.lookupFunction(fn, args.map(_.`type`))
      .map { irf => irf(irArgs) })
} yield ir

and

fn match {
  // ...
  case "annotate" =>
    if (!args(1).isInstanceOf[StructConstructor])
      None
    else
      maybeFindIRFunction(fn, args)
  case _ =>
    maybeFindIRFunction(fn, args)

?

@@ -660,17 +660,24 @@ case class Apply(posn: Position, fn: String, args: Array[AST]) extends AST(posn,
} yield ir.ApplyBinaryPrimOp(op, x, y, t)
}

def tryIRConversion(agg: Option[String]): Option[IR] =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be private[this] since it's not intended to be used by other classes.

@jbloom22 jbloom22 merged commit 2154185 into hail-is:master Mar 22, 2018
jbloom22 pushed a commit to jbloom22/hail that referenced this pull request Mar 22, 2018
* move annotate(struct1, struct2) to IR

* address comments

* adrress comment, fix rebase error
konradjk pushed a commit to konradjk/hail that referenced this pull request Jun 12, 2018
* move annotate(struct1, struct2) to IR

* address comments

* adrress comment, fix rebase error
jackgoldsmith4 pushed a commit to jackgoldsmith4/hail that referenced this pull request Jun 25, 2018
* move annotate(struct1, struct2) to IR

* address comments

* adrress comment, fix rebase error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants