Merge upstream changes (#92)

* [batch] Worker cleanup (hail-is#10155)

* [batch] Worker cleanup

* more changes

* wip

* delint

* additions?

* fix

* [query] Add `source_file_field` to `import_table` (hail-is#10164)

* [query] Add `source_file_field` to `import_table`

CHANGELOG: Add `source_file_field` parameter to `hl.import_table` to allow lines to be associated with their original source file.

* ugh

* [ci] add authorize sha and action items table to user page (hail-is#10142)

* [ci] add authorize sha and action items table to user page

* [ci] track review requested in addition to assigned for PR reviews

* [ci] add CI dropdown with link to user page (hail-is#10163)

* [batch] add more logs and do not wait for asyncgens (hail-is#10136)

* [batch] add more logs and do not wait for asyncgens

I think there is some unresolved issue with asyncgen shutdown that is keeping
workers alive. This is not an issue in worker because worker calls sys.exit
which forcibly stops execution. cc: @daniel-goldstein @jigold.

* fix lint

* [query-service] maybe fix event loop not initialized (hail-is#10153)

* [query-service] maybe fix event loop not initialized

The event loop is supposed to be initialized in the main thread. Sometimes
our tests get placed in the non-main thread (always a thread named Dummy-1).
Hopefully the session-scoped fixture is run in the main thread.

* fix

* [prometheus] add prometheus to track SLIs (hail-is#10165)

* [prometheus] add prometheus to track SLIs

* add wraps

* [query] apply nest-asyncio as early as possible (hail-is#10158)

* [query] apply nest-asyncio as early as possible

* fix

* [grafana] set pod fsGroup to grafana user (hail-is#10162)

* fix linting errors (hail-is#10171)

* [query] Remove verbose print (hail-is#10167)

Looks like this got added in some dndarray work

* [ci] update assignees and reviewers on PR github update (hail-is#10168)

* [query-service] fix receive logic (hail-is#10159)

* [query-service] fix receive logic

Only one coro waits on receive now. We still error if a message is sent before
we make our first response.

* fix

* fix

* CHANGELOG: Fixed incorrect error message when incorrect type specified with hl.loop (hail-is#10174)

* [linting] add curlylint check for any service that renders jinja2 (hail-is#10172)

* [linting] add curlylint check for any service that renders jinja2 templates

* [linting] spaces not tabs

* [website] fix website (hail-is#10173)

* [website] fix website

I build old versions of the docs and use them in new websites. This does not
work for versions of the docs before I introduced the new system. In particular
versions 0.2.63 and before generate old-style docs.

* tutorials are templated

* [ci] change mention for deploy failure (hail-is#10178)

* [gateway] move ukbb routing into gateway (hail-is#10179)

* [query] Fix filter intervals (keep=False) memory leak (hail-is#10182)

* [query-service] remove service backend tests (hail-is#10180)

They are too flaky currently due to the version issue.

* [website] pass response body as kwarg (hail-is#10176)

* Release 0.2.64 (hail-is#10183)

* Bump version number

* Updated changelog

* [nginx] ensure nginx configs dont overwrite each other in build.yaml (hail-is#10181)

* [query-service] teach query service to read MTs and Ts created by Spark (hail-is#10184)

* [query-service] teach query service to read MTs and Ts created by Spark

Hail-on-Spark uses HadoopFS which emulates directories by creating size-zero files with
the name `gs://bucket/dirname/`. Note: the object name literally ends in a slash. Such files
should not be included in `listStatus` (they should always be empty anyway). Unfortunately,
my fix in hail-is#9914 was wrong because `GoogleStorageFileStatus` removes
the trailing slash. This prevented the path from matching `path`, which always ends in a `/`.

* fix

* [website] dont jinja render any of the batch docs (hail-is#10190)

* [googlestoragefs] ignore the directory check entirely (hail-is#10185)

* [googlestoragefs] ignore the directory check entirely

If a file exists with the *same name as the directory we are listing*,
then it must be a directory marker. It does not matter if that file is
a directory or not.

* Update GoogleStorageFS.scala

* [ci] fix focus on slash and search job page for PRs (hail-is#10194)

* [query] Improve file compatibility error (hail-is#10191)

* Call init_service from init based on HAIL_QUERY_BACKEND value. (hail-is#10189)

* [query] NDArray Sum (hail-is#10187)

* Attempt implementing the sum rule in Emit

* Connected the python code, but not working yet

* NDArrayExpression.sum is working now

* Add default arg when no axis is provided

* More comprehensive test

* Unused imports

* Use sum appropriately in linear_regression_rows_nd

* Deleted extra blank line

* Don't use typeToTypeInfo, make NumericPrimitives the source of these decisions

* Better assertions, with tests

* Got the summation index correct

* Add documentation

* [website] fix resource path for non-html files in the docs (hail-is#10196)

* [query] Remove tcode from primitive orderings (hail-is#10193)

Co-authored-by: jigold <jigold@users.noreply.github.com>
Co-authored-by: Tim Poterba <tpoterba@broadinstitute.org>
Co-authored-by: Daniel Goldstein <danielgold95@gmail.com>
Co-authored-by: Dan King <daniel.zidan.king@gmail.com>
Co-authored-by: John Compitello <johnc@broadinstitute.org>

ci/ci/ci.py

@@ -83,11 +83,6 @@ async def get_job(request, userdata):
     return await render_template('ci', request, userdata, 'job.html', page_context)
 
 
-@routes.get('/healthcheck')
-async def healthcheck(request):  # pylint: disable=unused-argument
-    return web.Response(status=200)
-
-
 @routes.post('/api/v1alpha/dev_deploy_branch')
 @monitor_endpoint
 @rest_authenticated_developers_only
@@ -216,6 +211,7 @@ def run():
 
     setup_common_static_routes(routes)
     app.add_routes(routes)
+    app.router.add_get("/metrics", server_stats)
 
     web.run_app(
         deploy_config.prefix_application(app, 'ci'),

ci/ci/templates/pr.html

@@ -1,6 +1,9 @@
 {% from "job-table.html" import job_table with context %}
 {% extends "layout.html" %}
 {% block title %}PR {{ number }}{% endblock %}
+{% block head %}
+    <script src="{{ base_path }}/common_static/focus_on_keyup.js"></script>
+{% endblock %}
 {% block content %}
     <h1>{{ pr.title }} <a href="https://github.com/{{ repo }}/pull/{{ pr.number }}"><span class="gh-number">#{{ pr.number }}</span></a></h1>
 
@@ -20,7 +23,7 @@ <h1>{{ pr.title }} <a href="https://github.com/{{ repo }}/pull/{{ pr.number }}">
       <button type="submit">Retry</button>
     </form>
     <h2>Jobs</h2>
-    {{ job_table(jobs) }}
+    {{ job_table(jobs, "jobs", "jobsSearchBar") }}
     {% elif exception is defined %}
     <p>Build error:</p>
     <pre>
@@ -57,4 +60,7 @@ <h2>Build History</h2>
     {% else %}
     No builds.
     {% endif %}
+    <script type="text/javascript">
+      focusOnSlash("jobsSearchBar");
+    </script>
 {% endblock %}

hail/python/hail/backend/service_backend.py

@@ -87,6 +87,8 @@ def __init__(self, billing_project: str = None, bucket: str = None, *, deploy_co
 
         if bucket is None:
             bucket = get_user_config().get('batch', 'bucket', fallback=None)
+        if bucket is None:
+            bucket = os.environ.get('HAIL_BUCKET')
         if bucket is None:
             raise ValueError(
                 'the bucket parameter of ServiceBackend must be set '

hail/python/hail/context.py

@@ -160,6 +160,10 @@ def init(sc=None, app_name='Hail', master=None, local='local[*]',
     To initialize Hail explicitly with non-default arguments, be sure to do so
     directly after importing the module, as in the above example.
 
+    To facilitate the migration from Spark to the ServiceBackend, this method
+    calls init_service when the environment variable HAIL_QUERY_BACKEND is set
+    to "service".
+
     Note
     ----
     If a :class:`pyspark.SparkContext` is already running, then Hail must be
@@ -214,15 +218,26 @@ def init(sc=None, app_name='Hail', master=None, local='local[*]',
         Local temporary directory.  Used on driver and executor nodes.
         Must use the file scheme.  Defaults to TMPDIR, or /tmp.
     """
-    from hail.backend.spark_backend import SparkBackend
-
     if Env._hc:
         if idempotent:
             return
         else:
             warning('Hail has already been initialized. If this call was intended to change configuration,'
                     ' close the session with hl.stop() first.')
 
+    if os.environ.get('HAIL_QUERY_BACKEND') == 'service':
+        return init_service(
+            log=log,
+            quiet=quiet,
+            append=append,
+            tmpdir=tmp_dir,
+            local_tmpdir=local_tmpdir,
+            default_reference=default_reference,
+            global_seed=global_seed,
+            skip_logging_configuration=skip_logging_configuration)
+
+    from hail.backend.spark_backend import SparkBackend
+
     log = _get_log(log)
     tmpdir = _get_tmpdir(tmp_dir)
     local_tmpdir = _get_local_tmpdir(local_tmpdir)

hail/python/hail/expr/expressions/typed_expressions.py

@@ -17,7 +17,7 @@
     identity, nullable, tupleof, sliceof, dictof
 from hail.utils.java import Env, warning
 from hail.utils.linkedlist import LinkedList
-from hail.utils.misc import wrap_to_list, get_nice_field_error, get_nice_attr_error
+from hail.utils.misc import wrap_to_list, wrap_to_tuple, get_nice_field_error, get_nice_attr_error
 
 import numpy as np
 
@@ -4099,6 +4099,36 @@ def __matmul__(self, other):
 
         return res if result_ndim > 0 else res[()]
 
+    @typecheck_method(axis=nullable(oneof(int, tupleof(int))))
+    def sum(self, axis=None):
+        """Sum out one or more axes of an ndarray.
+
+        Parameters
+        ----------
+        axis : :class:`int` :class:`tuple`
+            The axis or axes to sum out.
+
+        Returns
+        -------
+        :class:`.NDArrayNumericExpression`
+        """
+        if axis is None:
+            axis = tuple(range(self.ndim))
+
+        axis = wrap_to_tuple(axis)
+        res_ir = ir.NDArrayAgg(self._ir, axis)
+
+        axes_set = set(axis)
+        if len(axes_set) < len(axis):
+            raise ValueError("duplicate value in 'axis'")
+        for element in axes_set:
+            if element < 0 or element >= self.ndim:
+                raise ValueError(f"axis {element} is out of bounds for ndarray of dimension {self.ndim}")
+
+        num_axes_deleted = len(axes_set)
+
+        return construct_expr(res_ir, tndarray(self._type.element_type, self.ndim - num_axes_deleted), self._indices, self._aggregations)
+
 
 scalars = {tbool: BooleanExpression,
            tint32: Int32Expression,

hail/python/hail/methods/statgen.py

@@ -434,7 +434,7 @@ def select_array_indices(hl_array, indices):
         return indices.map(lambda i: hl_array[i])
 
     def dot_rows_with_themselves(matrix):
-        return (matrix * matrix) @ hl.nd.ones(matrix.shape[1])
+        return (matrix * matrix).sum(1)
 
     def no_missing(hail_array):
         return hail_array.all(lambda element: hl.is_defined(element))
@@ -491,7 +491,7 @@ def process_block(block):
         def process_y_group(idx):
             X = hl.nd.array(block[entries_field_name].map(lambda row: mean_impute(select_array_indices(row, ht.kept_samples[idx])))).T
             n = ht.ns[idx]
-            sum_x = (X.T @ hl.nd.ones((n,)))
+            sum_x = X.sum(0)
             Qtx = ht.__cov_Qts[idx] @ X
             ytx = ht.__y_nds[idx].T @ X
             xyp = ytx - (ht.__Qtys[idx].T @ Qtx)

hail/python/test/hail/expr/test_ndarrays.py

@@ -479,16 +479,25 @@ def test_ndarray_save():
             assert np.array_equal(expected, actual)
 
 
-@skip_unless_spark_backend()
-@run_with_cxx_compile()
 def test_ndarray_sum():
     np_m = np.array([[1, 2], [3, 4]])
     m = hl.nd.array(np_m)
 
-    assert_all_eval_to(
+    assert_ndarrays_eq(
         (m.sum(axis=0), np_m.sum(axis=0)),
         (m.sum(axis=1), np_m.sum(axis=1)),
-        (m.sum(), np_m.sum()))
+        (m.sum(), np_m.sum()),
+        (m.sum(tuple([])), np_m.sum(tuple([]))),
+        (m.sum((0, 1)), np_m.sum((0, 1)))
+    )
+
+    with pytest.raises(ValueError) as exc:
+        m.sum(3)
+    assert "out of bounds for ndarray of dimension 2" in str(exc.value)
+
+    with pytest.raises(ValueError) as exc:
+        m.sum((1, 1))
+    assert "duplicate" in str(exc.value)
 
 
 def test_ndarray_transpose():

hail/src/main/scala/is/hail/expr/ir/AbstractMatrixTableSpec.scala

@@ -43,7 +43,10 @@ object RelationalSpec {
     }
 
     if (!FileFormat.version.supports(fileVersion))
-      fatal(s"incompatible file format when reading: $path\n  supported version: ${ FileFormat.version }, found $fileVersion")
+      fatal(s"incompatible file format when reading: $path\n  supported version: ${ FileFormat.version }, found $fileVersion" +
+        s"\n  The cause of this error is usually an attempt to use an older version of Hail to read files " +
+        s"generated by a newer version. This is not supported (Hail native files are back-compatible, but not forward-compatible)." +
+        s"\n  To read this file, use a newer version of Hail.")
     jv
   }
 

hail/src/main/scala/is/hail/expr/ir/Emit.scala

@@ -1678,6 +1678,7 @@ class Emit[C](
       case x: NDArrayConcat => emitDeforestedNDArrayI(x)
       case x: NDArraySlice => emitDeforestedNDArrayI(x)
       case x: NDArrayFilter => emitDeforestedNDArrayI(x)
+      case x: NDArrayAgg => emitDeforestedNDArrayI(x)
       case x@RunAgg(body, result, states) =>
         val newContainer = AggContainer.fromBuilder(cb, states.toArray, "run_agg")
         emitVoid(body, container = Some(newContainer))
@@ -2799,6 +2800,33 @@ class Emit[C](
               }
             })
           }
+        case NDArrayAgg(child, axesToSumOut) =>
+          deforest(child).map(cb) { childEmitter =>
+            val childDims = child.typ.asInstanceOf[TNDArray].nDims
+            val axesToKeep = (0 until childDims).filter(axis => !axesToSumOut.contains(axis))
+            val newOutputShape = axesToKeep.map(idx => childEmitter.outputShape(idx))
+            val newOutputShapeComplement = axesToSumOut.map(idx => childEmitter.outputShape(idx))
+
+            new NDArrayEmitter(newOutputShape) {
+              override def outputElement(cb: EmitCodeBuilder, idxVars: IndexedSeq[Value[Long]]): PCode = {
+                val numericElementType = coerce[PNumeric](child.pType.asInstanceOf[PNDArray].elementType)
+                val runningSum = NumericPrimitives.newLocal(cb, "ndarray_agg_running_sum", numericElementType.virtualType)
+                cb.assign(runningSum, numericElementType.zero)
+
+                SNDArray.forEachIndex(cb, newOutputShapeComplement, "NDArrayAgg_Sum_loop"){ case (cb, coordsBeingSummedOut) =>
+                  // Build the new list we need to pass down into child
+                  val idxVarsIt = idxVars.toIterator
+                  val summedOutIt = coordsBeingSummedOut.toIterator
+                  val fullIndicesForChild = (0 until childDims).map(idx =>
+                    if (axesToSumOut.contains(idx)) summedOutIt.next() else idxVarsIt.next()
+                  )
+                  cb.assign(runningSum, numericElementType.add(runningSum, childEmitter.outputElement(cb, fullIndicesForChild).code))
+                }
+
+                PCode.apply(numericElementType, runningSum)
+              }
+            }
+          }
         case _ =>
           val ndI = emit(x)
           ndI.map(cb){ ndPCode =>

hail/src/main/scala/is/hail/expr/ir/NumericPrimitives.scala

@@ -0,0 +1,17 @@
+package is.hail.expr.ir
+
+import is.hail.asm4s.Settable
+import is.hail.types.virtual.{Type, TInt32, TInt64, TFloat32, TFloat64}
+
+object NumericPrimitives {
+
+  def newLocal(cb: EmitCodeBuilder, name: String, typ: Type): Settable[Any] = {
+    coerce[Any](typ match {
+      case TInt32 => cb.newLocal[Int](name)
+      case TInt64 => cb.newLocal[Long](name)
+      case TFloat32 => cb.newLocal[Float](name)
+      case TFloat64 => cb.newLocal[Double](name)
+    })
+  }
+
+}

hail/src/main/scala/is/hail/expr/ir/orderings/PrimitiveOrdering.scala

@@ -13,17 +13,17 @@ object Int32Ordering {
       val type2: SInt32 = t2
 
       def _compareNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Int] =
-        Code.invokeStatic2[java.lang.Integer, Int, Int, Int]("compare", x.tcode[Int], y.tcode[Int])
+        Code.invokeStatic2[java.lang.Integer, Int, Int, Int]("compare", x.asInt.intCode(cb), y.asInt.intCode(cb))
 
-      def _ltNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Int] < y.tcode[Int]
+      def _ltNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asInt.intCode(cb) < y.asInt.intCode(cb)
 
-      def _lteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Int] <= y.tcode[Int]
+      def _lteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asInt.intCode(cb) <= y.asInt.intCode(cb)
 
-      def _gtNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Int] > y.tcode[Int]
+      def _gtNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asInt.intCode(cb) > y.asInt.intCode(cb)
 
-      def _gteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Int] >= y.tcode[Int]
+      def _gteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asInt.intCode(cb) >= y.asInt.intCode(cb)
 
-      def _equivNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Int].ceq(y.tcode[Int])
+      def _equivNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asInt.intCode(cb).ceq(y.asInt.intCode(cb))
     }
   }
 }
@@ -37,17 +37,17 @@ object Int64Ordering {
       val type2: SInt64 = t2
 
       def _compareNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Int] =
-        Code.invokeStatic2[java.lang.Long, Long, Long, Int]("compare", x.tcode[Long], y.tcode[Long])
+        Code.invokeStatic2[java.lang.Long, Long, Long, Int]("compare", x.asLong.longCode(cb), y.asLong.longCode(cb))
 
-      def _ltNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Long] < y.tcode[Long]
+      def _ltNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asLong.longCode(cb) < y.asLong.longCode(cb)
 
-      def _lteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Long] <= y.tcode[Long]
+      def _lteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asLong.longCode(cb) <= y.asLong.longCode(cb)
 
-      def _gtNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Long] > y.tcode[Long]
+      def _gtNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asLong.longCode(cb) > y.asLong.longCode(cb)
 
-      def _gteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Long] >= y.tcode[Long]
+      def _gteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asLong.longCode(cb) >= y.asLong.longCode(cb)
 
-      def _equivNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Long].ceq(y.tcode[Long])
+      def _equivNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asLong.longCode(cb).ceq(y.asLong.longCode(cb))
     }
   }
 }
@@ -60,17 +60,17 @@ object Float32Ordering {
       val type2: SFloat32 = t2
 
       def _compareNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Int] =
-        Code.invokeStatic2[java.lang.Float, Float, Float, Int]("compare", x.tcode[Float], y.tcode[Float])
+        Code.invokeStatic2[java.lang.Float, Float, Float, Int]("compare", x.asFloat.floatCode(cb), y.asFloat.floatCode(cb))
 
-      def _ltNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Float] < y.tcode[Float]
+      def _ltNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asFloat.floatCode(cb) < y.asFloat.floatCode(cb)
 
-      def _lteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Float] <= y.tcode[Float]
+      def _lteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asFloat.floatCode(cb) <= y.asFloat.floatCode(cb)
 
-      def _gtNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Float] > y.tcode[Float]
+      def _gtNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asFloat.floatCode(cb) > y.asFloat.floatCode(cb)
 
-      def _gteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Float] >= y.tcode[Float]
+      def _gteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asFloat.floatCode(cb) >= y.asFloat.floatCode(cb)
 
-      def _equivNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Float].ceq(y.tcode[Float])
+      def _equivNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asFloat.floatCode(cb).ceq(y.asFloat.floatCode(cb))
     }
   }
 }
@@ -83,17 +83,17 @@ object Float64Ordering {
       val type2: SFloat64 = t2
 
       def _compareNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Int] =
-        Code.invokeStatic2[java.lang.Double, Double, Double, Int]("compare", x.tcode[Double], y.tcode[Double])
+        Code.invokeStatic2[java.lang.Double, Double, Double, Int]("compare", x.asDouble.doubleCode(cb), y.asDouble.doubleCode(cb))
 
-      def _ltNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Double] < y.tcode[Double]
+      def _ltNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asDouble.doubleCode(cb) < y.asDouble.doubleCode(cb)
 
-      def _lteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Double] <= y.tcode[Double]
+      def _lteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asDouble.doubleCode(cb) <= y.asDouble.doubleCode(cb)
 
-      def _gtNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Double] > y.tcode[Double]
+      def _gtNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asDouble.doubleCode(cb) > y.asDouble.doubleCode(cb)
 
-      def _gteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Double] >= y.tcode[Double]
+      def _gteqNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asDouble.doubleCode(cb) >= y.asDouble.doubleCode(cb)
 
-      def _equivNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.tcode[Double].ceq(y.tcode[Double])
+      def _equivNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Boolean] = x.asDouble.doubleCode(cb).ceq(y.asDouble.doubleCode(cb))
     }
   }
 }
@@ -106,7 +106,7 @@ object BooleanOrdering {
       val type2: SBoolean = t2
 
       def _compareNonnull(cb: EmitCodeBuilder, x: PCode, y: PCode): Code[Int] =
-        Code.invokeStatic2[java.lang.Boolean, Boolean, Boolean, Int]("compare", x.tcode[Boolean], y.tcode[Boolean])
+        Code.invokeStatic2[java.lang.Boolean, Boolean, Boolean, Int]("compare", x.asBoolean.boolCode(cb), y.asBoolean.boolCode(cb))
     }
   }
 }

libhail/src/hail/query/backend/compile.hpp

@@ -47,7 +47,7 @@ class CompileFunction {
   /* Indexed by parameter index, the entry is the index of the first
      `llvm_function` parameter. */
   std::vector<size_t> param_llvm_start;
-  
+
   llvm::Function *llvm_function;
   // FIXME rename llvm_builder
   llvm::IRBuilder<> llvm_ir_builder;
@@ -56,7 +56,7 @@ class CompileFunction {
 
   // FIXME move to SType
   const SType *get_default_stype(const Type *t);
-  
+
   llvm::Type *get_llvm_type(PrimitiveType pt) const;
   llvm::AllocaInst *make_entry_alloca(llvm::Type *llvm_type);