Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates for compatibility with recent Hy changes #9

Merged
merged 10 commits into from
Aug 31, 2018
Merged

Updates for compatibility with recent Hy changes #9

merged 10 commits into from
Aug 31, 2018

Conversation

brandonwillard
Copy link
Owner

This PR renames/replaces #8..

…hanges.

XXX: There appear to be breaking changes in how reader/tags are used in this
library and parsed/lexed by Hy.  Essentially, tags can't take empty arguments:
e.g.  `(blah #s)` fails (apparently because `)` is taken as an argument to
`#s`), but `(blah #s None)` doesn't.
Regrettably, all iterators returned by `rest` are explicitly evaluated as lists.
This was done to work around Hy issues involving `islice` in macros, and it needs
to re-assessed.

Also, now that there are no built-in cons cells, all the logic that used them
needs to be reworked.  This commit only makes minor changes that allow the code
to compile.
@brandonwillard brandonwillard changed the title Recent hy updates Updates for compatibility with recent Hy changes Jun 28, 2018
@algernon algernon self-assigned this Jul 4, 2018
Unification was failing in cases where `cdr` was expected to return an element
instead of a collection with a single element.  Specifically, `(cdr (cons 1 2))`
should return `2` instead of `(2)`, while `(cdr (, 1 2))` should return `(2)`.

Extending Python's `list` (`UserList`, really) seemed like an easy way to
re-introduce `cons` and allow one to treat `cons` pairs as generic Python lists.
As well, it wasn't clear how one could easily distinguish `cons` from other
Python collections otherwise, which is used to produce the aforementioned `cdr`
behavior.
@brandonwillard
Copy link
Owner Author

brandonwillard commented Jul 9, 2018

The current issue(s) can be demonstrated by the following:

(fresh [x] 
  ( x 2)
  (project [x] 
    ( q (type x))))

Attempting to evaluate this (whether in a run or not) yields the error message HyMacroExpansionError: bind targets must be symbols.

Expanded, the above is (after removing "_;" prefixes from some symbol names)

(do
  (setv let|1319 {}
        (get let|1319 "x") (LVar (gensym 'x)))
  (all ( (get let|1319 "x") 2)
       (fn [s|1320]
         (do
           (require [hy.contrib.walk [let :as let|1321]])
           (let|1321 [__x (get let|1319 "x")
                      (get let|1319 "x") (reify __x s|1320)]
                     ((all ( q (type (get let|1319 "x"))))
                      s|1320))))))

where the relevant line is

           (let|1321 [__x (get let|1319 "x")
                      (get let|1319 "x") (reify __x s|1320)]

Apparently let does not evaluate the variables to be bound when they're unevaluated expressions (and I don't see why it would), so we might need to explicitly evaluate those parts of the bindings lists ourselves. It would be more convenient to have something like Emacs' generalized variables and cl-letf.

@brandonwillard
Copy link
Owner Author

brandonwillard commented Jul 24, 2018

I think the root problem has to do with macro expansion and the use of require within macros.

Consider the following:

(require [hy.contrib.walk [let]])

(defmacro fresh-good [a b]
  `(do (let [~a 1] ~b)))

(defmacro/g! fresh-bad [a b]
  `(do
     (require [hy.contrib.walk [let :as ~g!let]])
     (~g!let [~a 1] ~b)))

(defmacro project-good [y]
  `(do (let [~y 2] ~y)))

(defmacro/g! project-bad [y]
  `(do
     (require [hy.contrib.walk [let :as ~g!let]])
     (~g!let [~y 2] ~y)))

The functions labeled -bad are an extremely distilled version of the problem above.

Evaluating them analogous to how we would above, we can confirm that the output also corresponds:

;; Reference case
=> (fresh-good a (project-good a))
_hyx_XsemicolonXletXvertical_lineX1665 = {}
_hyx_XsemicolonXletXvertical_lineX1665['a'] = 1
_hyx_XsemicolonXletXvertical_lineX1666 = {}
_hyx_XsemicolonXletXvertical_lineX1666['a'] = 2
_hyx_XsemicolonXletXvertical_lineX1666['a']

2

;; Direct correspondence case
=> (fresh-bad a (project-bad a))
File "<input>", unknown location
HyMacroExpansionError: bind targets must be symbols

;; Test case 1
=> (fresh-bad a (project-good a))
_hyx_XsemicolonXletXvertical_lineX1762 = {}
_hyx_XsemicolonXletXvertical_lineX1762['a'] = 1
_hyx_XsemicolonXletXvertical_lineX1763 = {}
_hyx_XsemicolonXletXvertical_lineX1763['a'] = 2
_hyx_XsemicolonXletXvertical_lineX1763['a']

2

;; Test case 2
=> (fresh-good a (project-bad a))
File "<input>", unknown location
HyMacroExpansionError: bind targets must be symbols

These examples hint toward the inner require (i.e. -bad) macro being the cause of the problem.

Since let expands (using macroexpand-all) the macros of its body argument and replaces its bindings keys — which are symbols — with get expressions to the let-bindings env/dictionary it generates, the expression (fresh-good a (project-bad a)) performs the naive replacement in the unexpanded inner gensymed let form, which then leads to a malformed let. The replacement is only possible because the inner let hasn't been expanded by the outer let's call to macroexpand-all, as this would've otherwise replaced all occurrences of HySymbol("a"). The only time the inner let is expand is during evaluation, but then it's already a malformed let without the possibility of our intervention.

Besides reverting the use of require within the macros, I can't think of a particularly good fix right now. I've tried forcing expansion of the requireed let in project-bad, but that seems to necessitate supplementary eval logic elsewhere. Functionality like cl-letf would handle this particular case, but perhaps not much else. It might also be possible to extend the considerations of hy.contrib.walk.macroexpand-all, so that it expands aliased gensymed symbols corresponding to macro requires.
@gilch, I've probably missed something simpler (e.g. regarding how I'm using let, require, etc.), but, as of now, this is where I'm at with the situation.

@gilch
Copy link

gilch commented Jul 25, 2018

From your description, I think the problem is that macroexpand-all is still not meeting the spec I described in hylang/hy#1420.

It probably needs to handle require and eval-and-compile/eval-when-compile specially.

@brandonwillard
Copy link
Owner Author

@gilch, just put in a PR for this functionality: hylang/hy#1664

@brandonwillard
Copy link
Owner Author

FYI: A lot of the remaining test failures are related to cons semantics.

The tests were changed from implicit cons expressions like `[1 2 z]` to
`(cons 1 2 z)`, which is now equivalent to Scheme's `'(1 2 . z)` under
our new `cons`.  This also makes the tests read almost exactly like their
sources in The Reasoned Schemer.
@brandonwillard
Copy link
Owner Author

All right, all the tests should pass now, but I had to enforce a stricter use of cons.
I'm not sure if that clashes with the desired semantics and compatibility with Hy/Python, but it's definitely not hard to use and it lines up well with all the miniKanren work in Scheme.

@brandonwillard
Copy link
Owner Author

Updated the bench tests.

The remaining Travis failures are for versions of Python no longer supported by Hy; @algernon, should we remove those versions and add Pythons 3.5, 3.6, 3.7?

@algernon
Copy link
Contributor

Thank you for the incredible amount of work you put in this! <3

@algernon algernon merged commit a3e86d9 into brandonwillard:master Aug 31, 2018
@brandonwillard
Copy link
Owner Author

@algernon, anytime!

I'll take a crack at updating hydiomatic next, then the basics of some symbolic math and proof assistant/type theory capabilities via miniKanren (in Hy, using this project, of course). Ultimately, I plan to work Theano (and maybe SymPy) into the mix.

If you're interested, I can provide more details (Gitter, IRC?); regardless, would gladly have your input on these follow-up projects.

@algernon
Copy link
Contributor

algernon commented Sep 1, 2018

I lurk on IRC (in #hy too) with the same username, feel free to poke me there. I'm more than happy to chat about related projects!

The only problem is, I'm not very active around Hy nowadays. I eventually merge PRs but that's about it. Just the other day, I was pondering about transferring monaxhyd and adderall to someone else (or even 'hylang/`)...

@brandonwillard
Copy link
Owner Author

@algernon, I can help with managerial things, if you'd like.

Personally, I think miniKanren-like concepts and DSLs are — and will be — sorely needed in certain areas of work, and a project like this (through Hy) is a much better approach than shoe-horning miniKanren and/or Python. That said, I'm convinced this project is well worth maintaining.

Also, I had wondered why hydiomatic was under hylang/ and this wasn't.

@brandonwillard
Copy link
Owner Author

What's the deal with that Travis failure? That pypy3.5 archive download URL seems valid. Just an AWS hiccup?

@brandonwillard brandonwillard deleted the recent-hy-updates branch September 3, 2018 00:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants