Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

documentation: build-system phases + build-time tests #2780

Merged

Conversation

alalazo
Copy link
Member

@alalazo alalazo commented Jan 9, 2017

This PR adds the information needed to grasp how build-system base classes work, and how to code functions that implement build-time checks activated by the --run-tests option of spack install. It also improves the formatting of in-code documentation for build_system classes.

Modifications

@alalazo alalazo added documentation Improvements or additions to documentation WIP labels Jan 9, 2017
@alalazo alalazo force-pushed the documentation/build_systems_and_tests branch from 9196ca4 to 471754b Compare January 9, 2017 15:12
@alalazo
Copy link
Member Author

alalazo commented Jan 9, 2017

@adamjstewart @davydden @hartzell Not ready for a review yet, but early comments are always welcome

@adamjstewart
Copy link
Member

Might as well include documentation on PythonPackage. #2778 should be finished fairly soon. Some of the changes to autotools.py will conflict with #2742, but we can work those out once it is merged.

@alalazo
Copy link
Member Author

alalazo commented Jan 9, 2017

@adamjstewart I tried to describe the general infrastructure for a base class tied to a build-system. The idea is that build-system specific overrides should be documented in code within each module and a line should be added in the table present in the packaging guide. This should make documenting further extensions more scalable. What do you think?

@adamjstewart
Copy link
Member

I think this sounds like a good plan. I agree with documenting things in the API docs and linking to that documentation in the packaging guide. A single example of a complicated package with multiple overrides would probably suffice to demonstrate what we mean.

@alalazo alalazo force-pushed the documentation/build_systems_and_tests branch from 471754b to 27f45d9 Compare January 9, 2017 15:45
@alalazo
Copy link
Member Author

alalazo commented Jan 9, 2017

@adamjstewart Do you know any way to speed-up building docs while writing them? make in the doc folder is annoyingly slow...

@adamjstewart
Copy link
Member

Hmm, I'm not sure. In the Makefile, we allow it to run in parallel with the JOBS variable. In run-doc-tests, we actually force it to run in serial. I'm not sure why.

+------------------------------------+----------------------------------+
| :py:class:`.RPackage` | Specialized class for |
| | :py:class:`.R` extensions |
+------------------------------------+----------------------------------+
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs should be consistent for R (and Python) too... that the package superclass depends on the BUILD SYSTEM being used, not the language of the package. For example, a Python extension installed with CMake (they do exist) would extends('python'), but it would also subclass from CMakePackage.

Similarly, some upstream packages come with a "configure" script that is not a real Autotools configure script. Although this duck has the same name, it is not really a duck: it can't be regenerated with autoreconf, its command line options may be different from those of a standard Autotools-built configure, etc. Therefore, such packages need to subclass Packagd.

PythonPackage should be added to this list.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@citibeth This is consistent with the current state of things. I'll add what is in #2778 and #2742 as soon as it will be merged into develop. I was also hoping to get by for now saying that spack create does the right thing most of the times... 😄

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's now a note after the table stressing what you say above.

* fetches an archive for the correct version of the software
* expands the archive
* sets the current working directory to the root directory of the expanded archive

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Start bullet points with capital letters...

.. note::
Each specific build-system has a list of attributes that can be overridden to
fine-tune the installation of a package without overriding an entire phase. To
have more information on them the place to go is the API docs at the end of this manual.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should include a link here to build system-specific sections of the manual: CMakePackage, PythonPackage, etc. (I see that such sections are below in this PR, in the API docs).

I think we also need at least 1-2 examples that "put it all together". We should see cases where configure_args() or cmake_args() is used, and where entire phases are overridden. That would go in a tutorial section, this looks more like a reference manual section.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to see if it is possible to reference the table at the beginning of the subsection, which has the correct links. I'll leave the tutorial part for another PR.

(``self.spec``), and ``prefix`` is actually an attribute of ``spec``
(``spec.prefix``).
The arguments ``spec`` and ``prefix`` are passed only for convenience, as they always
correspond to ``self.spec`` and ``self.spec.prefix`` respectively.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe there is inconsistency on which methods get spec and prefix, and which do not. For example, I believe that configure_args() does not? This should be pointed out. Or even better... we agree to get rid of these convenience args across the board. I think they are of diminishing convenience, since we write fewer install() methods.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disclaimer: I fully agree with

Or even better... we agree to get rid of these convenience args across the board. I think they are of diminishing convenience, since we write fewer install() methods.

and endorse the proposal of @tgamblin of using funcargs in the long term to provide certain arguments where needed. This will get you both convenience and clean code.

That said the rule is that phases have the same signature as install:

def install(self, spec, prefix):
    pass

for backward compatibility, while other methods don't.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I think all methods should provide self, spec, and prefix. They aren't always needed, but for the case of configure_args, they usually are. The first line of pretty much every configure_args in Spack right now looks like spec = self.spec.

I am also fine with @tgamblin's proposal which would allow all of the following to work:

def install(self):
def install(self, spec):
def install(self, spec, prefix):

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am also fine with @tgamblin's proposal which would allow all of the following to work:

I'd rather not go that route. It is f***ing magic, for marginal gains. I think we should decide on function signatures and use them. I would rather we choose the version of signatures I don't like, than go with more magic.

Copy link
Member

@tgamblin tgamblin Jan 15, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@citibeth dependency injection is common practice in a lot of frameworks (Spring, pytest, others). It takes a little getting used to, but it's not "magic".

IMHO it makes a lot of sense for hook APIs like the methods in packages. Currently the packager has to know a method signature, and the signature is kind of arbitrary (convenience args? no convenience args?). With injection, there's no signature, and the packager can ask for the things they want to be passed in.

I think it's actually more natural to use these argument semantics for a hook API -- control is already inverted for the packager, in the sense that the method author controls the behavior of the framework. It makes sense for the method author to also decide what parameters are needed (pull semantics) instead of the framework pushing them in.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alalazo: before I get carried away with dependency injection, though, it might be useful to look at how this would affect the spec attributes API. Those methods have some mandated parameters/structure, so they're a little different.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After thinking about it, I still disagree. See @adamjstewart comment below: "Is there any difference between prefix, self.prefix and self.spec.prefix?" The fact remains that these arguments we're proposing to dependency-inject are ALREADY being passed to the methods, under self. Dependency injection adds no functional gains. For methods that want to use (say) the argument spec, there is already a way to make that clear, using plain vanilla programming:

def mymethod(self):
    spec = self.spec
    ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but for commonly used arguments like spec, it would be nice to have easy access to them. This is the reason that the old Package install method gave you access to self, spec, and prefix. It's the reason that url_for_version gives you self and version, even though version probably equals self.version. If Spack were a Python module for Python developers, I would tend to agree with you. But my hope is that people who are unfamiliar with Python and OO-programming can easily modify or create packages.

At the very least, I think we can agree that we should at least be consistent. Either only pass self to every method, or pass self, spec, and prefix. I vote for the latter obviously. With dependency injection, everyone can get what they want, so it's a good compromise.

Copy link
Member

@davydden davydden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM apart from a few minor suggestions

| | specialized for any build system |
+------------------------------------+----------------------------------+
| :py:class:`.MakefilePackage` | Specialized class for packages |
| | that are built invoking an |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an->a

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to let you know where you got me into. Apparently you are right and I am 'old-fashioned' to say the least. Went with plural in the end 😄

Description:
GNU M4 is an implementation of the traditional Unix macro processor.

Typically, phases have default implementations that fit most of the common cases:
Copy link
Member

@davydden davydden Jan 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

double space in have default.

Defining an installation procedure means overriding a set of methods or attributes
that will be called at some point during the installation of the package.
What determines the actual set of entities that are available for overriding
is the package base class, which is usually specialized for a given build system.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consistently use either build-system or build system

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"build system" is better.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tgamblin Do you want it quoted everywhere or just here ?

Copy link
Member

@tgamblin tgamblin Jan 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh gosh. No quotes! I just meant to use a space. No dash unless you have to (e.g. if it's used as an adjective).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahaha ok! As I already had a version with spaces when you commented I thought the "stress" "was" "on" "quotes" 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's hard to stress a space. build" "system? Bleh.

Sometimes packages finish to build "correctly" and issues with their run-time
behavior are discovered only at a later stage, maybe after a full software stack
relying on them has already been built. To avoid situation of that kind it's possible
to code build-time tests that will be executed only if the option ``--run-tests``
Copy link
Member

@davydden davydden Jan 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code-> run? (somebody who contribute a package usually does not code tests in C++ or alike)

to code build-time tests that will be executed only if the option ``--run-tests``
of ``spack install`` has been activated.

The proper way to write these tests is relying on two decorators that come with
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should probably mention somewhere here the default implementation which will try to do make test when --run-tests is used.

"""This phase cannot be defaulted for obvious reasons..."""
"""Edits the Makefile before calling make. This phase cannot
be defaulted for obvious reasons.
"""
tty.msg('Using default implementation: skipping edit phase.')
Copy link
Member

@davydden davydden Jan 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OT: shouldn't you throw an error here if the parent's class implementation (this one) gets called?

@davydden
Copy link
Member

p.s. thanks for working on this!

@tgamblin
Copy link
Member

@alalazo

Do you know any way to speed-up building docs while writing them? make in the doc folder is annoyingly slow...

If you figure this out let me know. I tried to make it parallel locally, but that doesn't help much either. I think sphinx is just unnecessarily slow.

Copy link
Member

@tgamblin tgamblin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside from a few minor change requests, this LGTM. See the comments below for specifics.


The last element of a package is its ``install()`` method. This is
---------------------------------------
Implementing the installation procedure
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this more clear than "the install() method"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea here is that we don't have a single install method anymore. I came up with installation procedure, but a better name for this would be really welcome.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see -- I'm still thinking on it. "Build pipeline" is the best I can think of so far. Let's just leave it as-is. I agree it's not just install() anymore.

Implementing the installation procedure
---------------------------------------

The last element of a package is its **installation procedure**. This is
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this section, though probably the most important one, has gotten buried in the packaging guide. I wonder if there's a way to bring it closer to the top and make it more obvious. Or maybe the packaging guide needs to be a top-level section...

Defining an installation procedure means overriding a set of methods or attributes
that will be called at some point during the installation of the package.
What determines the actual set of entities that are available for overriding
is the package base class, which is usually specialized for a given build system.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"build system" is better.

subclass from :py:class:`.CMakePackage`.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mechanics of an installation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we call this "installation pipeline" or something more evocative of what's going on?

(``self.spec``), and ``prefix`` is actually an attribute of ``spec``
(``spec.prefix``).
The arguments ``spec`` and ``prefix`` are passed only for convenience, as they always
correspond to ``self.spec`` and ``self.spec.prefix`` respectively.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alalazo: before I get carried away with dependency injection, though, it might be useful to look at how this would affect the spec attributes API. Those methods have some mandated parameters/structure, so they're a little different.


^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
``sanity_check_is_file`` and ``sanity_check_is_dir``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Not all builds are like this. Many builds of scientific
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since there is a section break, can you replace "this" with something more explicit?

@@ -2695,6 +2781,43 @@ the build will fail and the install prefix will be removed. If they
succeed, Spack considers the build successful and keeps the prefix in
place.

^^^^^^^^^^^^^^^^
Build-time tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section LGTM in that it documents what is currently there, but I want to rework how tests are done. I've been working with @BarrySmith on this. We're working on fleshing out an API for doing short (smoke) tests and long tests on the build and install directories. Some packages only support testing in the build directory, and others can test on an install directory, and I'd like to make it so that you can run a command like spack short-test <spec> or spack long-test <spec> after installation and have it work either way. I'd also like to make that easy for the user...

So can we put a note on this section that the test API is expected to change rather drastically in future versions?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tgamblin I'll put a note, but I think that the interface and API for build-time tests should not change drastically. Right now:

  • a lot of things come for free
  • running code after a phase requires just to use a decorator

@adamjstewart
Copy link
Member

@alalazo Sorry about the conflicts from #2742! Let me know if you need any help resolving conflicts. Hopefully they aren't too bad.

@alalazo
Copy link
Member Author

alalazo commented Jan 16, 2017

@adamjstewart No worries. VCS conflicts are part of a programmer's life 😄

@tgamblin tgamblin mentioned this pull request Jan 17, 2017
@alalazo
Copy link
Member Author

alalazo commented Jan 17, 2017

@tgamblin @adamjstewart I won't change this, but pycharm is yelling at me every time I open a doc file because:

This file is indented with 3 spaces instead of 4

@alalazo alalazo force-pushed the documentation/build_systems_and_tests branch 2 times, most recently from 22eb6a9 to 4454844 Compare January 17, 2017 12:43
@adamjstewart
Copy link
Member

Haha, I actually changed my editor to use 3 space indentation for rst because that seemed like the standard. We could probably change it to 4 in another PR if that's what people want to use.

@alalazo
Copy link
Member Author

alalazo commented Jan 18, 2017

This is ready for a second review.

Copy link
Member

@citibeth citibeth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These docs are looking good!

:linenos:
Defining an installation procedure means overriding a set of methods or attributes
that will be called at some point during the installation of the package.
What determines the actual set of entities that are available for overriding
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to: "The package base class, usually specialized for a given build system, determines the actual set of entities available for overriding."

+------------------------------------+----------------------------------+
| :py:class:`.CMakePackage` | Specialized class for packages |
| | that are built using CMake |
+------------------------------------+----------------------------------+
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove "that are" in descriptions of MakefilePackage, AutotoolsPackage, CMakePackage.

rare cases where manual intervention is needed we need to stress that a
package base class depends on the *build system* being used, not the language of the package.
For example, a Python extension installed with CMake would ``extends('python')`` and
subclass from :py:class:`.CMakePackage`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This paragraph feels on the wordy side.


* Fetches an archive for the correct version of the software
* Expands the archive
* Sets the current working directory to the root directory of the expanded archive
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this list be numbered, as it as a step-1-2-3 kind of thing? Put periods at the end of each item.


Installation Phases:
autoreconf configure build install

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider cutting the rest off and replacing with '...'

return self.stage.source_path

def edit(self, spec, prefix):
"""This phase cannot be defaulted for obvious reasons..."""
"""Edits the Makefile before calling make. This phase cannot
be defaulted for obvious reasons.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove "for obvious reasons." Don't make the reader feel stupid.

tty.msg('Using default implementation: skipping edit phase.')

def build(self, spec, prefix):
"""Make the build targets"""
"""Calls make passing :py:attr:`~.MakefilePackage.build_targets`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comma after make.

with working_dir(self.build_directory()):
inspect.getmodule(self).make(*self.build_targets)

def install(self, spec, prefix):
"""Make the install targets"""
"""Calls make passing :py:attr:`~.MakefilePackage.install_targets`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comma after make

@@ -34,7 +34,7 @@ class RPackage(PackageBase):

This class provides a single phase that can be overridden:

* install
1. :py:meth:`~.RPackage.install`

It has sensible defaults and for many packages the only thing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comma after defaults

# To be used in UI queries that require to know which
# build-system class we are using
#: This attribute is used in UI queries that require to know which
#: build-system class we are using
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above.

@citibeth
Copy link
Member

citibeth commented Jan 18, 2017 via email

@tgamblin
Copy link
Member

@citibeth: I think at this point we don't need to argue the args thing, since it's not implemented, and the docs cover the current code. FWIW, though, I think we could use it to inject more than just attributes of the spec, but the use cases aren't fully fleshed out.

@alalazo: can you address @citibeth's comments above and we'll get this merged?

@alalazo alalazo force-pushed the documentation/build_systems_and_tests branch from 6896de1 to e4d9126 Compare January 20, 2017 19:04
@alalazo alalazo closed this Jan 22, 2017
@alalazo alalazo reopened this Jan 22, 2017
@alalazo
Copy link
Member Author

alalazo commented Jan 22, 2017

@tgamblin ping

@tgamblin tgamblin merged commit a8e1d78 into spack:develop Jan 23, 2017
@alalazo alalazo deleted the documentation/build_systems_and_tests branch January 23, 2017 22:19
diaena pushed a commit to diaena/spack that referenced this pull request May 26, 2017
* documentation: reworked packaging guide to add build-system phases

* documentation: improvements to AutotoolsPackage autodocs

* build_systems: updated autodocs

* run-tests: added a few information on how to run tests fixes spack#2606 fixes#2605

* documentation: fixed items brought up by @davydden

    * typos in docs
    * consistent use of 'build system' (i.e. removed 'build-system' from docs)
    * added a note on possible default implementations for build-time tests

* documentation: fixed items brought up by @citibeth

    * added note to explain the difference between build system and language used in a package
    * capitalized bullet items
    * added link to API docs

* documentation: fixed multiple cross-references after rebase

* documentation: fixed minor issues raised by @tgamblin

* documentation: added entry in table for the `PythonPackage` class

* docs: fixed issues brought up by @citybeth in the second review
amklinv pushed a commit that referenced this pull request Jul 17, 2017
* documentation: reworked packaging guide to add build-system phases

* documentation: improvements to AutotoolsPackage autodocs

* build_systems: updated autodocs

* run-tests: added a few information on how to run tests fixes #2606 fixes#2605

* documentation: fixed items brought up by @davydden

    * typos in docs
    * consistent use of 'build system' (i.e. removed 'build-system' from docs)
    * added a note on possible default implementations for build-time tests

* documentation: fixed items brought up by @citibeth

    * added note to explain the difference between build system and language used in a package
    * capitalized bullet items
    * added link to API docs

* documentation: fixed multiple cross-references after rebase

* documentation: fixed minor issues raised by @tgamblin

* documentation: added entry in table for the `PythonPackage` class

* docs: fixed issues brought up by @citybeth in the second review
healther pushed a commit to electronicvisions/spack that referenced this pull request Jul 26, 2017
* documentation: reworked packaging guide to add build-system phases

* documentation: improvements to AutotoolsPackage autodocs

* build_systems: updated autodocs

* run-tests: added a few information on how to run tests fixes spack#2606 fixes#2605

* documentation: fixed items brought up by @davydden

    * typos in docs
    * consistent use of 'build system' (i.e. removed 'build-system' from docs)
    * added a note on possible default implementations for build-time tests

* documentation: fixed items brought up by @citibeth

    * added note to explain the difference between build system and language used in a package
    * capitalized bullet items
    * added link to API docs

* documentation: fixed multiple cross-references after rebase

* documentation: fixed minor issues raised by @tgamblin

* documentation: added entry in table for the `PythonPackage` class

* docs: fixed issues brought up by @citybeth in the second review
@tgamblin tgamblin added this to the v0.11.0 milestone Nov 12, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
5 participants