Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Roadmap #80

Merged
merged 1 commit into from
Dec 28, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 6 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,42 +20,9 @@ The option of having pre-compiled binaries is a workaround, but we don't like to
keep third-party binary artifacts in source code,
(see https://github.com/doorstop-dev/doorstop/pull/431#issuecomment-549237579).

## Goals

The first goal is to make FileCheck.py pass on Mull's current integration
tests suite which uses only a very limited subset of FileCheck's features which
are as follows:

- Commands (both substring and regex matching):
- [x] `CHECK`
- [x] `CHECK-NEXT`
- [x] `CHECK-NOT`
- [x] `CHECK-EMPTY`

- Options:
- [x] `--strict-whitespace`
- [x] `--match-full-lines`
- [x] `--check-prefix`

When this is done, it feels like a good idea to implement full FileCheck's
contract and be 100% compatible with the C++ version.

There is a file in this repository: [FileCheck.pdf](FileCheck.pdf) which has
some rough implementation coverage: green color means something is implemented,
yellow color means that the text is not relevant to implementation, no
highlighting means "still to be implemented".

## Notes on implementation

The implementation is done against the latest
[FileCheck - Flexible pattern matching file verifier](https://llvm.org/docs/CommandGuide/FileCheck.html)
document. The plan is to skip looking into the LLVM's implementation of
FileCheck, unless some very advanced implementation details are encountered.

It a nice programming exercise of implementing something from a spec but we also
want to make FileCheck.py to conform to the same contract this is why
FileCheck.py is tested using LIT and the LLVM's FileCheck:

First, the LLVM FileCheck is run against a LIT/FileCheck test suite, then
FileCheck.py is run on the same tests to make sure that it has the same behavior
as the LLVM FileCheck.
## Documentation

Documentation is hosted on Read the Docs:

[FileCheck.py documentation](https://filecheck.readthedocs.io/en/latest/index.html).

3 changes: 3 additions & 0 deletions docs/01-what-is-filecheck.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ possible to keep third-party binary artifacts in source code
(see
https://github.com/doorstop-dev/doorstop/pull/431#issuecomment-549237579).

**Note:** FileCheck.py is not intended to be a replacement for LLVM's FileCheck
in any way. See :doc:`07-roadmap`.

What's next?
------------

Expand Down
50 changes: 48 additions & 2 deletions docs/07-roadmap.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,50 @@
Roadmap
-------
=======

...
As described in :doc:`01-what-is-filecheck`, FileCheck.py is only a Python port
for LLVM's FileCheck. It is not intended to be a replacement for LLVM's
FileCheck.

FileCheck.py is being tested against Mull mutation testing system. The first
goal is to make FileCheck.py pass on Mull's current integration tests suite that
uses only a very limited subset of FileCheck's features which are as follows:

- Commands (both substring and regex matching):

- ``CHECK``
- ``CHECK-NEXT``
- ``CHECK-NOT``
- ``CHECK-EMPTY``

- Options:

- ``--strict-whitespace``
- ``--match-full-lines``
- ``--check-prefix``

When the above features are implemented and considered stable, it might be a
good idea to implement full FileCheck's contract and be 100% compatible with the
C++ version.

There is a file in FileCheck.py's repository:
`FileCheck.pdf <https://github.com/stanislaw/FileCheck.py/blob/master/FileCheck.pdf>`_
which has some rough implementation coverage: green color means something is
implemented, yellow color means that the text is not relevant to implementation,
no highlighting means "still to be implemented".

Notes on implementation
-----------------------

The implementation is done against the latest
`FileCheck - Flexible pattern matching file verifier <https://llvm.org/docs/CommandGuide/FileCheck.html>`_
document. The plan is to skip looking into the LLVM's implementation of
FileCheck, unless some very advanced implementation details are encountered.

It a nice programming exercise of implementing something from a spec but we also
want to make FileCheck.py conform to the same contract this is why FileCheck.py
is tested using LIT and LLVM's FileCheck:

- First, the LLVM FileCheck is run against a FileCheck.py test suite based on
LIT/FileCheck.
- Second, FileCheck.py is run on the same tests to make sure that it has the
same behavior as the LLVM FileCheck.