From 0e78e7833fe8f314b01d1e7bab90809740a19c5b Mon Sep 17 00:00:00 2001 From: Leo Date: Tue, 19 Sep 2023 08:47:55 -0500 Subject: [PATCH 1/7] Submission of Botan / Cryptography Community Project: Leg 1 Proposal --- proposals/052-cryptography-leg-1.md | 329 ++++++++++++++++++++++++++++ 1 file changed, 329 insertions(+) create mode 100644 proposals/052-cryptography-leg-1.md diff --git a/proposals/052-cryptography-leg-1.md b/proposals/052-cryptography-leg-1.md new file mode 100644 index 00000000..1cf194c9 --- /dev/null +++ b/proposals/052-cryptography-leg-1.md @@ -0,0 +1,329 @@ +# Cryptography Community Project: Leg 1 Proposal + +# Abstract + +This community project proposal is for full-time funding for the development of a suite of Haskell cryptography libraries and tooling suitable a wide range of uses including data integrity, privacy, security, and networking. This is necessary to relieve the burden [of correctly implementing cryptography] from Haskell developers seeking to provide privacy and security to their users. We propose the development of a community-owned set of cryptography libraries of increasing capability, starting with bindings to a compatible open-source cryptography library. This is expected to require a considerable effort over time, and if accepted will be broken up into several legs, each of which will be accompanied by its own specific proposal and set of goals. + +The first leg of this proposal is for the development of high-quality, production-ready bindings to the Botan C++ cryptography library, and from acceptance to release on Hackage is estimated to take 3 months of full-time development. We see several challenges but no significant technical risk to this proposal; known challenges involve strictness and developing additional bindings to C++; these items are of high-value and may take significant effort to resolve, but are known to be solvable. Considerable progress has already been made, and the proposed funding will help ensure its completion. + +# Background + +Cryptography is an essential part of modern computing and information security, but it requires care, nuance, and knowledge to properly implement, and is extremely easy to mess up. It covers a diverse range of functions, with individual algorithms often having their own particular quirks that must be acknowledged to properly and safely use. Failure to properly implement cryptography carries the risk of severely compromising the security of any system in which it is used. Manually implementing common cryptographic primitives such as hashes, ciphers, digital signatures, big-integer arithmetic is thus considered extremely unwise, both in terms of security and performance. + +Cryptography is even harder for functional programming languages. Properly implementing techniques such as sanitizing memory, performing operations in constant time, are already difficult enough in languages such as C, and in a lazy functional language it is made even more difficult by such things as the potential for references to be kept alive within thunks. As a result, the space of functional cryptography is relatively under-explored and under-developed, having favored other languages where such things are easier to manage. + +However, if we accept this downside as a part of a trade-off, and explore the ways that functional programming is beneficial to cryptography (monads, type systems, referential transparency), there is then also the potential for functional cryptography to be made *easier*. The functional machinery of Haskell is well-suited for adding important contextual control to cryptography, and can help prevent or eliminate many issues and errors in ways that languages such as C cannot. I believe that there is room for significant improvement, given a directed effort to develop the space. + +# Problem Statement + +Cryptography in Haskell lacks significant capability beyond basic primitives. This places a significant burden on developers to properly implement various security techniques, and exposes end users to significant risk in the event of a lapse in security. Some companies have built their own solution to this, but there is no community-driven, community-owned solution. + +Cryptography in Haskell is also fragile and outdated, having seen considerable flux / churn over the years as various libraries have been developed and then deprecated or abandoned in favor of newer ones. This has placed the long-term stability of important libraries such as `tls` and `x509` at risk of falling behind as compilers are upgraded, old standards are updated, and new standards are implemented. + +There are several reasons for this: + +1) Cryptography involves a lot of stateful low-level bit twiddling and random IO, something Haskell is not known for being good at. Classes like `Bits` and `FiniteBits` classes are critical for cryptography, but are terribly awkward to implement (like `Num`). Also, `ByteString` does not implement them for nuanced reasons. + +2) Cryptography has a high knowledge and technical skill floor. Cryptography is more than just the usual primitives of ciphers, public and private keys, hashes, and MACs, and the scope of knowledge can grow quickly as primitives are combined into more advanced algorithms. + +3) The notion that functional programming is "bad at cryptography" (ie, underperformant and has insurmountable challenges) is a self-fulfilling prophecy, and will remain so unless we invest in enough effort to disprove it. + +4) A fair lot of cryptographic projects in recent years has been specifically towards *cryptocurrencies*, rather than *cryptography in general*, and their results are at large fairly useless if they cannot be used outside of their ecosystems. + +All of these reasons have contributed to the stagnant state of Haskell cryptography, and have make it an unfavorable environment for developers of cryptosystems and distributed systems. However, none of these are actual blockers, and each of these things can be corrected with effort. There is much that can be done to improve the Haskell cryptography ecosystem, ranging from better low-level support, to higher-level abstractions: + +- Up-to-date algorithms +- Hierarchy of typeclasses +- Monadic interfaces +- Safety abstractions +- Type-level composition +- Advanced data structure (eg, merkle trees) +- Use of Generic for crypto-serialization + +As a result, there is much to be gained from investing in foundational Haskell cryptography. Haskell is well-suited to providing many aspects of information security through use of monadic interfaces and the type system, if the lower-level challenges of strictness and performant primitives can be solved. The community needs a cryptography library that is easily maintained, with minimal dependencies / baggage, and with documentation, tutorials, and unit tests to ease development. + +A first step would be developing bindings to a suitably-licensed open-source C or C++ library containing modern and up-to-date cryptography algorithms, including post-quantum cryptography schemes. + +There are [several C/C++ library candidates](https://en.wikipedia.org/wiki/Comparison_of_cryptography_libraries): + +- Botan +- libgcrypt +- Crypto++ +- libsodium / NaCl +- NSS + +Of these, Botan was selected for several reasons: + +- It has a BSD license +- It has a C FFI +- It offers a broad variety of functionality and algorithms, including post-quantum cryptography +- It has been [audited](https://botan.randombit.net/releases/audit_1.11.18.pdf) in the past +- There are dead bindings targeting Botan 2 / GHC 8 that can be examined / revived +- Significant implementation work has already been performed in order to validate the feasibility of Botan + +The others were not selected for several reasons: + +- libgcrypt is C so it requires no FFI, but it has a GNU LGPL license, and provides less functionality than Botan +- Crypto++ has a Boost license (similar to BSD), but is C++ and has no C FFI, and provides less functionality than Botan +- libsodium / NaCL does not provide the necessary functionality and there are several Haskell libsodium bindings already +- NSS has a copyleft license + +Bindings to Botan would solve the significant problem of dealing with common cryptography by providing much of the necessary 'cryptographic kitchen sink' via bindings to a suitably performant, suitably licensed, open-source library. The Haskell community would not be required to maintain the Botan cryptography library, only bindings to it, and this can be accomplished readily with some effort which has already begun. Furthermore, the Botan library supports several post-quantum cryptography algorithms, which is highly desirable. + +It is important that the Haskell community has a strong cryptography library / presence that is not bent towards a particular agenda. It must be **free as in speech (licensing)** and **free as in beer (no mining)**, in order to allow the growth of advanced cryptographic techniques the way that Haskell has furthered the growth of functional programming techniques. + +Furthermore, bindings to a dedicated cryptography library may yield a performance benefit, as one would expect a popular C++ crypto library such as Botan to be highly performant. Testing in GHCi with `:set +s` for a requested example of `bcrypt` yielded the following results: + +``` +-- Botan +ghci> bcryptGenerateIO "Fee fi fo fum!" r 16 +"$2a$16$j3Obv.HNT8zQ9OXFXEJeQeq3nT5WmUecPYDF0tc3grNsX9N0uDSCm\NUL" +(3.73 secs, 996,152 bytes) + +-- cryptonite +ghci> hashPassword 16 ("Fee fi fo fum!" :: ByteString) :: IO ByteString +"$2b$16$nmHHDe/wnfSl85dIFtqjMuAHxqAmL.JQ6knp8e6kf.X1V9/LCZxUy" +(8.84 secs, 2,138,266,840 bytes) +``` + +It was quick and dirty testing in GHCi so your mileage may vary, but the results indicate that bindings to Botan may be significantly faster and consumes less memory than `crypton/ite`, if other modules / functions are similarly performant. Further implentation, testing, and benchmarking will be necessary to confirm this. + +# Prior Art and Related Efforts + +We can group Haskell cryptography efforts into several categories, which are not necessarily distinct: + +- Libraries for specific algorithms +- Cryptographic "kitchen sinks" +- Bindings to foreign libraries +- Efforts by [Thomas DuBuisson](https://hackage.haskell.org/user/ThomasDuBuisson) +- Efforts by [Vincent Hanquez](https://hackage.haskell.org/user/VincentHanquez) + +The wiki lists [several libraries](https://wiki.haskell.org/Applications_and_libraries/Cryptography) and Hackage lists [many more](https://hackage.haskell.org/packages/#cat:Cryptography) according to the category. + +> NOTE: Cryptocurrency libraries and libraries not on Hackage are excluded from consideration. + +Specific libraries worth noting are: + +- Predecessors to `cryptonite` - all deprecated + - [cryptohash](https://hackage.haskell.org/package/cryptohash) + - [cryptocipher](https://hackage.haskell.org/package/cryptocipher) + - [crypto-pubkey](https://hackage.haskell.org/package/crypto-pubkey) + - [crypto-random](https://hackage.haskell.org/package/crypto-random) +- Cryptographic "kitchen sinks" + - [cryptonite](https://hackage.haskell.org/package/cryptonite) + - [crypton](https://hackage.haskell.org/package/cryptonite) - a fork of cryptonite +- Bindings to libsodium + - [NaCl](https://github.com/serokell/haskell-crypto) + - [saltine](https://hackage.haskell.org/package/saltine) + - [sel](https://hackage.haskell.org/package/sel-0.0.1.0/candidate) +- Bindings to botan + - [Z-Botan](https://hackage.haskell.org/package/Z-Botan) - part of Z-Haskell, GHC8 +- Honorable mentions + - [Crypto](https://hackage.haskell.org/package/Crypto) - a very old sink + - [crypto-api](https://hackage.haskell.org/package/crypto-api) - cryptographic classes + - [cryptol](https://hackage.haskell.org/package/cryptol) - a DSL for developing new cryptographic primitives + +These prior efforts can be considered to be somewhat partially successful in presenting Haskell developers with access to cryptographic primitives, but present a significant development challenge and a high refactoring cost if dependences become deprecated, which happens often. Of these libraries, a considerable amount are deprecated or have been dead for many years, with `cryptonite`, `crypton`, and `saltine` being clear front-runners in terms of current usage and transitive dependencies (and `sel` being under active development). Of these four, one is a fork of another (`crypton` and `cryptonite`), and the other two are bindings to `libsodium` which does not sufficiently capture our need for low-level cryptographic primitives in general. + +None of these sufficiently express cryptography through the type-system and with type-classes. `crypton/ite` exposes a few highly- opinionated / specific classes for hashing, block and stream ciphers, but otherwise is focused on concrete implementations and functions. Both `crypton` and `cryptonite` rely on bindings to C implementations of unknown veracity; though the code is itself not suspect, the lack of provenance makes it ill-suited for recommendation in trusted environments, and maintenance of the C implementation poses burden of high technical skill. Libsodium bindings are more trustworthy, but the limited options severely hamper interop with other systems unless they too are using libsodium, making them ill-suited for recommendation for some use cases. `Z-Botan` has been apparently dead for several years, and even if revived has a has a sizeable dependency in the form of the `Z-Haskell` ecosystem. + +Furthermore, none of these libraries contain implementations of post-quantum cryptography schemes; although existing quantum computers are of insufficient capability to break commonly used algorithms such as Ed25519, this will not remain true forever, and it would be best to have quantum-resistant implementations ready for adoption long before such attacked become practical. + +Each of these efforts has one or more of these issues that preclude them from being considered a complete success: + +- Insufficient abstraction +- Maintenance of handwritten C +- Bindings to an insufficiently general library +- Burdensome dependencies +- Dead project +- Lack of post-quantum algorithms + +However, there is also a lot to be learned from the ways these prior efforts *were* successful. This proposal will assess these existing efforts in an attempt to consolidate their learnings and functionality. + +## An example of a successful ecosystem + +The health of the Haskell server ecosystem may be used as an example of a successful community-driven niche, and we can aspire to bring that same quality and health to the Haskell cryptography ecosystem. There are many pertinent parallels in terms of managing performance, complexity, and safety that we can learn from: + +- Low-level packages like `wai` supporting multiple backends +- Backends range from simple like `scotty` to complex like `servant` +- Frontends like `blaze-html` and `lucid` +- Libraries can take advantage of type system to provide good abstractions +- Flexible enough abstractions to explore the problem space with multiple solutions +- Appropriate licensing + +`servant` (and the `wai` ecosystem in general) is a success story and an example of higher-order programming making something safer and easier, something that we can aspire to do with cryptography. + +## Technical Content + +This proposal is for the contruction of a suite of cryptography libraries, ranging from low-level bindings to specific algorithms, to higher-level abstractions and typeclasses. + +This leg of the proposal focuses on bindings to the `botan` cryptography, in the form of the following libraries: + +### botan-bindings + +The `botan-bindings` library will contain raw bindings and is an almost direct, 1-1 translation of the C API into Haskell FFI calls. As such, it exposes and operates over C FFI types, requires buffer and pointer and pointer management, and returns error codes. This library only exposes FFI calls and constants, and is suitable for building your own abstraction over Botan. + +### botan-low + +The `botan-low` library will contain low-level bindings which wrap the FFI calls into monadic IO actions. This library handles the translation between buffers and ByteStrings, and throws exceptions in the case of errors, but is otherwise a fairly faithful translation of the interface. Thus this library is suitable for use, but is not recommended for day-to-day programming due to statefulness and a lack of type safety. + +### botan + +The `botan` library will contain high-level bindings which hides the stateful IO and exposes appropriately idiomatic and pure interfaces. It will provide data types for algorithms, enumerations for constants, convenience methods for key and nonce generation, incremental processing of lazy bytestrings, and otherwise bring significant safety and ergonomics over `botan-low` and `botan-bindings`. + +## Optional / Future Technical Content + +More advanced concepts such as typeclasses may be delivered in a future leg focused on generalization, but some development occur earlier in order to provide a testable 'use case' for the stack of libraries. If the first leg is completed ahead of schedule, the remaining effort will go towards these libraries. + +### crypto-schemes + +The `crypto-schemes` library will contain a hierarchy of abstract type classes and data families for cryptography and scheme identifiers, allowing for a unified interface to multiple back-ends. It will also provide various utility functions including sized bit and byte vectors, block and padding functions. + +### crypto-schemes-botan + +The `crypto-schemes-botan` library will be an implementation of `crypto-schemes` using `botan`, implementing class instances for the specific algorithms that `botan` provides. + +### botanium + +The `botanium` library will contain user-friendly bindings to specific algorithms, designed for ease of use much like `libsodium`, but using `botan` or `crypto-schemes-botan` as a backend. + +### botanite + +The `botanite` library will contain an as-close-as-possible reimplementation of `crypton/ite` using `botan` or `crypto-schemes-botan` as a back-end, and may be kept as a separate library or merged into `crypton` at a future time. + +## Goals and Non-goals + +The following are the immediate goals of this first leg: + +- To publish the `botan`, `botan-low`, and `botan-bindings` libraries +- To build proper documentation and tutorials covering the subject material +- To establish a community project presence focused on Haskell and cryptography + +The following are overarching goals, and will not be complete by this first leg: + +- To publish the `botanium` libsodium- and `botanium` crypton- compatible libraries +- To publish the `crypto-schemes` library and `crypto-schemes-botan` backend. +- To provide a suitable alternative interface / backend (as an option) for `crypton`, `tls`, `x509` +- To consolidate Haskell cryptography into a consistent, unified, user-friendly system +- To enable the safe use and implementation of advanced cryptographic data structures +- To provide cryptographic typeclasses for safe and consistent use + +The following are not goals: + +- Finishing every goal by the first leg of the project +- Matching interfaces 1:1 with existing libraries / interfaces +- Implementing every algorithm permutation possible + +## Timeline + +This timeline will focus on the first leg of the proposal, and the immediate goal of publishing the following libraries: + +- botan-bindings +- botan-low +- botan + +The full project has a large scope, and will require significant time to complete. More advanced concepts such as typeclasses may be delivered in a future leg focused on generalizing. + +### Expected project leg duration + +We estimate that it will take 3 months for this leg, to achieve our goal of developing and publishing these libraries. This estimate is based on the significant amount of work already completed, the length of time in which it was accomplished, and the remaining amount of work. + +Several work items (strictness, memory safety, C++ shims for missing FFI functionality) are difficult to estimate, and so time has been allocated for them in the estimate, but their delivery is considered optional, and may roll over into a future leg as necessary. + +### Intermediate deliverables + +The following are intermediate deliverables intended to keep the community apprised of any progress that is made, and any challenges that arise, and to seek community feedback: + +- Frequent (2-3x weekly) activity update on devlog +- Weekly major update that includes github push +- Monthly report that summarizes what specific items have been worked on, what items will be worked on, and what challenges have arisen, if any. + +### Deliverables + +The following are the intended deliverables of this leg of the proposal: + +- `botan-bindings` +- `botan-low` +- `botan` + +Each library should be beta- or / production-ready, and submitted to hackage as a candidate or accepted package. + +Each library should have: + +- Documentation +- Tutorials and examples +- Tracked issues +- Unit tests + +A project website will also be created. + +The official repo will be owned by the Haskell Cryptography Group, and another member of the Haskell Cryptography Group will be selected to hold backup keys and permissions to the website, server, and official repositories as needed, in order to give project members access and to reduce any conflicts about ownership or maintenance. + +## Budget + +The first leg of this proposal presents a minimum budget of $7000 USD per month, for one full-time engineer, for a duration of 3 months, for a total of $21,000 USD. This budget is based on cost-of-living and industry experience, and will cover housing, food, bills, taxes, and other life necessities for one engineer, as well as any project necessities such website and server hosting. This budget is roughly equivalent to $40 / hr or $84k / yr at full-time of 40 hrs / wk. Industry rates for an engineer of the necessary skill are on average considerably higher, and so we consider this budget to be reasonable. The exact legal contract / arrangement is left to the Haskell Foundation. + +The question of continued funding will be revisited for each future leg of this proposal, wherein the budget and direction of future work will be decided again for each leg. + +## Stakeholders + +Several parties stand to gain from the implementation of this proposal. + +- Leon Dillinger (Myself) + +As the intended engineer carrying out the work, I have already invested significant effort in this project in order to establish its viability, and will benefit from this proposal in the form of being the recipient of the funding. + +- [Haskell Foundation](https://haskell.foundation/) + +As the intended source of funding, the Haskell Foundation would be invested in this proposal financially, and its effective success or failure would reflect respectively upon the Haskell Foundation. + +- [Haskell Cryptography Group ](https://github.com/haskell-cryptography) + +As the intended owner of the project, this project will be new maintenance burden on the Haskell Cryptography Group. + +- Users of existing cryptography libraries such as `crypton`, `cryptonite`, `libsodium / saltine`. + +As the intended target audience of this project, this proposal aims to directly benefit developers by providing high-quality cryptography libraries at several appropriate levels of abstraction, but will also place the burden of migration to these new libraries on developers. + +- Downstream users of Haskell software, developers with + +As the eventual user of the developed software, their safety and security will rest on the proper execution of this proposal. + +## Risks + +There are a number of risks. + +- Cryptography is hard and each algorithm may have individual nuances or uncommon requirements. +- Botan FFI is incompletely documented and has gaps (stream ciphers, X509 stores, a few broken functions) which need to be fixed with C++ shims +- Strictness, memory safety & scrubbing, C++ shims will take an unknown amount of time +- A thorough / safe implementation will require consulting the C++ source of the FFI +- Polynomial combinations of algorithms makes exhaustive testing complicated. + +These risks are amortized in that while they are tedious and potentially time-consuming, they may be considered extended goals of high value, and there is still substantial value that can be delivered without them, and the completion of risk items may continue in a future proposal leg. + +- Single point of failure + +This is a proposal for one full-time engineer; this constitutes a single point of failure. To combat this, the official github repo will be owned by the Haskell Cryptography Group, and another member of the Haskell Cryptography Group will be selected to hold backup keys and permissions to the website, server, and official repositories as needed. In the case of an emergency for which I am unreachable, this will allow for others to take over as necessary. + +## Success + +This leg of the project will be considered a success if it results in the submission of a candidate or accepted package to hackage, with production-ready bindings, documentation, tutorials, and unit tests as appropriate for the following botan-related libraries: + +- botan-bindings +- botan-low +- botan + +The following Botan library features are considered optional to the success of this leg, due to requirement of C++ / FFI shims or additional research which will take an unknown amount of work, but effort will be made to include them if possible within the timeframe. + +- Extended X509 support +- Stream ciphers +- Test vectors for algorithms, especially [CAVP / FIPS / NIST-approved](https://csrc.nist.gov/projects/cryptographic-algorithm-validation-program) + +The following libraries are considered non-essential to the success of this leg, but may be worked on as they provide a valuable use case with which to test the lower libraries against, and may be subsumed into another at any time (eg, botanite could be merged into crypton). + +- botanium +- botanite +- crypto-schemes +- crypto-schemes-botan + +The next leg of this proposal is still undetermined, but tentatively will be focused on compatibility with `crypton` through `botanite`, and on the development of abstract cryptography classes in `crypto-schemes`. From 687d7db836cddff54d9eb99918ee10d72c70a2f4 Mon Sep 17 00:00:00 2001 From: Leo Date: Mon, 9 Oct 2023 13:35:05 -0500 Subject: [PATCH 2/7] Editing in response to feedback and comments --- proposals/052-cryptography-leg-1.md | 244 ++++++++++++++++++---------- 1 file changed, 154 insertions(+), 90 deletions(-) diff --git a/proposals/052-cryptography-leg-1.md b/proposals/052-cryptography-leg-1.md index 1cf194c9..fd74f759 100644 --- a/proposals/052-cryptography-leg-1.md +++ b/proposals/052-cryptography-leg-1.md @@ -1,49 +1,67 @@ -# Cryptography Community Project: Leg 1 Proposal +# Botan Cryptography Community Project Proposal # Abstract -This community project proposal is for full-time funding for the development of a suite of Haskell cryptography libraries and tooling suitable a wide range of uses including data integrity, privacy, security, and networking. This is necessary to relieve the burden [of correctly implementing cryptography] from Haskell developers seeking to provide privacy and security to their users. We propose the development of a community-owned set of cryptography libraries of increasing capability, starting with bindings to a compatible open-source cryptography library. This is expected to require a considerable effort over time, and if accepted will be broken up into several legs, each of which will be accompanied by its own specific proposal and set of goals. +This community project proposal is for full-time funding for the development of a suite of Haskell cryptography libraries and tooling suitable a wide range of uses including data integrity, privacy, security, and networking. This is necessary to relieve the burden [of correctly implementing cryptography] from Haskell developers seeking to provide privacy and security to their users. We propose the development of a community-owned set of cryptography libraries of increasing capability, starting with bindings to a compatible open-source cryptography library. This is expected to require a considerable effort over time, and will be broken up into individual proposals, each of which will be accompanied by its own specific set of goals. -The first leg of this proposal is for the development of high-quality, production-ready bindings to the Botan C++ cryptography library, and from acceptance to release on Hackage is estimated to take 3 months of full-time development. We see several challenges but no significant technical risk to this proposal; known challenges involve strictness and developing additional bindings to C++; these items are of high-value and may take significant effort to resolve, but are known to be solvable. Considerable progress has already been made, and the proposed funding will help ensure its completion. +This first proposal is for the development of high-quality, production-ready bindings to the Botan C++ cryptography library, and from acceptance to release on Hackage is estimated to take 3 months of full-time development. We see several challenges but no significant technical risk to this proposal; known challenges involve strictness and developing additional bindings to C++; these items are of high-value and may take significant effort to resolve, but are known to be solvable. Considerable progress has already been made, and the proposed funding will help ensure its completion. # Background Cryptography is an essential part of modern computing and information security, but it requires care, nuance, and knowledge to properly implement, and is extremely easy to mess up. It covers a diverse range of functions, with individual algorithms often having their own particular quirks that must be acknowledged to properly and safely use. Failure to properly implement cryptography carries the risk of severely compromising the security of any system in which it is used. Manually implementing common cryptographic primitives such as hashes, ciphers, digital signatures, big-integer arithmetic is thus considered extremely unwise, both in terms of security and performance. -Cryptography is even harder for functional programming languages. Properly implementing techniques such as sanitizing memory, performing operations in constant time, are already difficult enough in languages such as C, and in a lazy functional language it is made even more difficult by such things as the potential for references to be kept alive within thunks. As a result, the space of functional cryptography is relatively under-explored and under-developed, having favored other languages where such things are easier to manage. +Cryptography is even more difficult for functional programming languages. Properly implementing techniques such as sanitizing memory, performing operations in constant time, and zeroing and ejecting from caches / registers, are already a challenge in a strict imperative language such as C, and in a lazy functional language it is even more difficult. Lazy evaluation presents a challenge towards managing and clearing sensitive data from memory, as care must be taken in order to ensure that it is not captured as part of a closure or thunk, and to force finalizers / cleanup appropriately or else the data may stay in memory for longer than intended. As a result, the space of functional cryptography appears to be relatively under-explored and under-developed, having favored other languages where such things are easier to manage. -However, if we accept this downside as a part of a trade-off, and explore the ways that functional programming is beneficial to cryptography (monads, type systems, referential transparency), there is then also the potential for functional cryptography to be made *easier*. The functional machinery of Haskell is well-suited for adding important contextual control to cryptography, and can help prevent or eliminate many issues and errors in ways that languages such as C cannot. I believe that there is room for significant improvement, given a directed effort to develop the space. +However, if we accept this downside as a part of a trade-off, and explore the ways that functional programming is beneficial to cryptography, there is then also the potential for functional cryptography to be made *easier*. + +The collective properties of referential transparency, type systems, and monadic interfaces provide a safer and more lawful context in which to write functions than the free-for-all that many imperative / OOP languages can offer, and so the functional machinery of Haskell is well-suited for adding important measures of safety and control to cryptography, helping to prevent or eliminate many issues and errors in ways that languages such as C cannot. I believe that there is room for significant improvement, given a directed effort to develop the space. # Problem Statement -Cryptography in Haskell lacks significant capability beyond basic primitives. This places a significant burden on developers to properly implement various security techniques, and exposes end users to significant risk in the event of a lapse in security. Some companies have built their own solution to this, but there is no community-driven, community-owned solution. +The Haskell cryptography ecosystem lacks significant capability beyond basic primitives. This places a significant burden on developers to properly implement various security techniques, and it exposes end users to significant risk in the event of a lapse in security. There are commercial solutions, but there is no community-driven, community-owned solution, and one is unlikely to be developed without deliberate community effort (see Appendix A - Commercial Incentive Misalignment). + +The Haskell cryptography ecosystem is also brittle and outdated. It is the product of considerable flux / churn over the years as various libraries have been developed and then deprecated or abandoned in favor of newer ones. The resulting libraries have highly-concrete interfaces and are strongly influenced by their implementation, both violating the abstraction barrier and preserving the design across successive libraries - as if their backend were to be changed, it would likely require changing or breaking the interface in turn. + +As a result, there are multiple cryptography libraries with similar interfaces, and it is not always clear which should be used preferentially. This promotes rot as downstream users unknownly rely on unmaintained libraries that slowly stop working. -Cryptography in Haskell is also fragile and outdated, having seen considerable flux / churn over the years as various libraries have been developed and then deprecated or abandoned in favor of newer ones. This has placed the long-term stability of important libraries such as `tls` and `x509` at risk of falling behind as compilers are upgraded, old standards are updated, and new standards are implemented. +Furthermore, a number of important libraries in the ecosystem (eg, `cryptonite`) have recently been abandoned by their author. Their backend is written in C, which presents a long-term maintenance burden with a high knowledge and skill requirement. NaCl bindings such as `saltine` are a high-quality alternative, but solve only a limited set of problems. -There are several reasons for this: +The resulting brittleness has placed the long-term stability of important libraries such as `crypton`, `tls` and `x509` at risk of falling behind as compilers are upgraded, old standards are updated, and new standards are implemented. Additionally the instability creates an unattractive environment for developers, effectively driving them away to other ecosystems that can provide the necessary stability. -1) Cryptography involves a lot of stateful low-level bit twiddling and random IO, something Haskell is not known for being good at. Classes like `Bits` and `FiniteBits` classes are critical for cryptography, but are terribly awkward to implement (like `Num`). Also, `ByteString` does not implement them for nuanced reasons. +There are several additional reasons for this lack of suitable engineers: + +1) Cryptography involves a lot of stateful low-level bitwise manipulation and non-determinism, something Haskell is not known for being good at. Classes like `Bits` and `FiniteBits` classes are critical for cryptography, but are terribly awkward to implement (like `Num`). Notably, `ByteString` does not implement them for nuanced reasons (see Appendix B - Lack of ByteString `Bits` instance). 2) Cryptography has a high knowledge and technical skill floor. Cryptography is more than just the usual primitives of ciphers, public and private keys, hashes, and MACs, and the scope of knowledge can grow quickly as primitives are combined into more advanced algorithms. -3) The notion that functional programming is "bad at cryptography" (ie, underperformant and has insurmountable challenges) is a self-fulfilling prophecy, and will remain so unless we invest in enough effort to disprove it. +3) There is a persistent notion that functional languages are too slow for cryptography, despite it being based on historical reasons that have been mostly since addressed. Well-written modern Haskell approaches C in terms of speed, and has a much higher ergonomic-use and type-safety factor, but the public image has not caught up yet, and so the ecosystem has not received the investment and attention necessary to attract the attention of skilled engineers for developing either pure implementations or foreign bindings. -4) A fair lot of cryptographic projects in recent years has been specifically towards *cryptocurrencies*, rather than *cryptography in general*, and their results are at large fairly useless if they cannot be used outside of their ecosystems. +4) A fair number of cryptographic projects in recent years has been specifically towards *cryptocurrencies*, rather than *cryptography in general*, and their results are at large fairly useless if they cannot be used outside of their ecosystems. -All of these reasons have contributed to the stagnant state of Haskell cryptography, and have make it an unfavorable environment for developers of cryptosystems and distributed systems. However, none of these are actual blockers, and each of these things can be corrected with effort. There is much that can be done to improve the Haskell cryptography ecosystem, ranging from better low-level support, to higher-level abstractions: +All of these reasons have contributed to the stagnant state of Haskell cryptography, and have make it an unfavorable environment for application developers, especially developers of cryptosystems and distributed systems. However, none of these are actual blockers, and each of these things can be corrected with effort. There is much that can be done to improve the Haskell cryptography ecosystem, ranging from better low-level support, to higher-level abstractions: - Up-to-date algorithms -- Hierarchy of typeclasses -- Monadic interfaces -- Safety abstractions -- Type-level composition -- Advanced data structure (eg, merkle trees) +- Idiomatic interfaces +- Type safety +- Memory safety abstractions +- Advanced data structures (eg, Merkle trees) - Use of Generic for crypto-serialization -As a result, there is much to be gained from investing in foundational Haskell cryptography. Haskell is well-suited to providing many aspects of information security through use of monadic interfaces and the type system, if the lower-level challenges of strictness and performant primitives can be solved. The community needs a cryptography library that is easily maintained, with minimal dependencies / baggage, and with documentation, tutorials, and unit tests to ease development. + + +As a result, there is much to be gained from investing in foundational Haskell cryptography. Haskell is well-suited to providing information security through use of type-safe interfaces, and the community needs a cryptography library that is easily maintained, with minimal dependencies / baggage, and with documentation, tutorials, and unit tests to ease development. + +There are several factors to consider: + +1) Resistance to modern (eg, side channel) attacks requires very low-level control over the implementation and computer, while Haskell is high-level and more concerned with abstraction and hiding the implementation +2) Trustworthy crypto libraries are extremely expensive to develop, but bindings are much cheaper +3) Haskell's C FFI can connect well to C implementations, and allows us to build an idiomatic interface on top +4) The cryptography community is more heavily invested into C, and so there are more mature libraries and resources in their ecosystem A first step would be developing bindings to a suitably-licensed open-source C or C++ library containing modern and up-to-date cryptography algorithms, including post-quantum cryptography schemes. +# Candidate Libraries + There are [several C/C++ library candidates](https://en.wikipedia.org/wiki/Comparison_of_cryptography_libraries): - Botan @@ -52,26 +70,42 @@ There are [several C/C++ library candidates](https://en.wikipedia.org/wiki/Compa - libsodium / NaCl - NSS +There are several factors for consideration: + +- Licensing: + - Does the library have an appropriate license, such as an MIT or BSD license? + - It must be free to use, but not so free as to impose a burden upon on its users. + - Commercial users are likely to be unwilling to depend on copyleft code, and we want to provide value to them too. +- Community: + - Does the library have an active development community? + - Does it have a large userbase? +- Bindings: + - Does the library have a Haskell FFI-capable C API? + - How difficult will it be to bind to this library? +- Auditing: + - Has the library been audited for security concerns? +- Coverage: + - What algorithms / primitives does this library implement? + Of these, Botan was selected for several reasons: - It has a BSD license -- It has a C FFI +- It has a Haskell FFI-capable C API - It offers a broad variety of functionality and algorithms, including post-quantum cryptography +- It is developed and maintained by an active community - It has been [audited](https://botan.randombit.net/releases/audit_1.11.18.pdf) in the past -- There are dead bindings targeting Botan 2 / GHC 8 that can be examined / revived +- There are unmaintained Haskell bindings targeting Botan 2 / GHC 8 that can be examined / revived - Significant implementation work has already been performed in order to validate the feasibility of Botan The others were not selected for several reasons: -- libgcrypt is C so it requires no FFI, but it has a GNU LGPL license, and provides less functionality than Botan -- Crypto++ has a Boost license (similar to BSD), but is C++ and has no C FFI, and provides less functionality than Botan -- libsodium / NaCL does not provide the necessary functionality and there are several Haskell libsodium bindings already +- libgcrypt is C, and so the Haskell FFI can bind directly to it, but it has a GNU LGPL license, and provides less functionality than Botan +- Crypto++ is C++, and has a Boost license (similar to BSD), but it does not export a C API, and provides less functionality than Botan +- libsodium / NaCL does not provide the necessary functionality, and there are several Haskell libsodium bindings already (see Appendix C - NaCl / libsodium) - NSS has a copyleft license Bindings to Botan would solve the significant problem of dealing with common cryptography by providing much of the necessary 'cryptographic kitchen sink' via bindings to a suitably performant, suitably licensed, open-source library. The Haskell community would not be required to maintain the Botan cryptography library, only bindings to it, and this can be accomplished readily with some effort which has already begun. Furthermore, the Botan library supports several post-quantum cryptography algorithms, which is highly desirable. -It is important that the Haskell community has a strong cryptography library / presence that is not bent towards a particular agenda. It must be **free as in speech (licensing)** and **free as in beer (no mining)**, in order to allow the growth of advanced cryptographic techniques the way that Haskell has furthered the growth of functional programming techniques. - Furthermore, bindings to a dedicated cryptography library may yield a performance benefit, as one would expect a popular C++ crypto library such as Botan to be highly performant. Testing in GHCi with `:set +s` for a requested example of `bcrypt` yielded the following results: ``` @@ -121,11 +155,12 @@ Specific libraries worth noting are: - Honorable mentions - [Crypto](https://hackage.haskell.org/package/Crypto) - a very old sink - [crypto-api](https://hackage.haskell.org/package/crypto-api) - cryptographic classes - - [cryptol](https://hackage.haskell.org/package/cryptol) - a DSL for developing new cryptographic primitives -These prior efforts can be considered to be somewhat partially successful in presenting Haskell developers with access to cryptographic primitives, but present a significant development challenge and a high refactoring cost if dependences become deprecated, which happens often. Of these libraries, a considerable amount are deprecated or have been dead for many years, with `cryptonite`, `crypton`, and `saltine` being clear front-runners in terms of current usage and transitive dependencies (and `sel` being under active development). Of these four, one is a fork of another (`crypton` and `cryptonite`), and the other two are bindings to `libsodium` which does not sufficiently capture our need for low-level cryptographic primitives in general. +These prior efforts can be considered to be somewhat partially successful in presenting Haskell developers with access to cryptographic primitives, but present a significant development challenge and a high refactoring cost if dependencies become deprecated, which happens often. Of these libraries, a considerable amount are deprecated or have been dead for many years, with `cryptonite`, `crypton`, and `saltine` being clear front-runners in terms of current usage and transitive dependencies (and `sel` being under active development). + +Of these four, one is a fork of another (`crypton` and `cryptonite`), and the other two are bindings to `libsodium` which does not sufficiently capture our need for low-level cryptographic primitives in general. `crypton/ite` exposes a few highly- opinionated / specific classes for hashing, block and stream ciphers, but otherwise is focused on concrete implementations and functions. Both `crypton` and `cryptonite` rely on bindings to C implementations that have not undergone the same vetting as commonly-used C libraries; though the code is itself not suspect, the lack of provenance makes it ill-suited for recommendation in trusted environments, and maintenance of the C implementation poses burden of high technical skill. `libsodium` bindings are more trustworthy, but the limited options severely hamper interop with other systems unless they too are using `libsodium`, making them ill-suited for recommendation for some use cases. -None of these sufficiently express cryptography through the type-system and with type-classes. `crypton/ite` exposes a few highly- opinionated / specific classes for hashing, block and stream ciphers, but otherwise is focused on concrete implementations and functions. Both `crypton` and `cryptonite` rely on bindings to C implementations of unknown veracity; though the code is itself not suspect, the lack of provenance makes it ill-suited for recommendation in trusted environments, and maintenance of the C implementation poses burden of high technical skill. Libsodium bindings are more trustworthy, but the limited options severely hamper interop with other systems unless they too are using libsodium, making them ill-suited for recommendation for some use cases. `Z-Botan` has been apparently dead for several years, and even if revived has a has a sizeable dependency in the form of the `Z-Haskell` ecosystem. +Following that, there is also `Z-Botan`, which has been apparently unmaintained for several years. Reviving it was considered, but would require a comprehensive and near-total rewrite in order to divest itself of `Z-Haskell` as a dependency. New bindings could instead draw significantly from `Z-Botan`, while presenting a much smaller maintenance and dependency surface. Furthermore, none of these libraries contain implementations of post-quantum cryptography schemes; although existing quantum computers are of insufficient capability to break commonly used algorithms such as Ed25519, this will not remain true forever, and it would be best to have quantum-resistant implementations ready for adoption long before such attacked become practical. @@ -135,7 +170,7 @@ Each of these efforts has one or more of these issues that preclude them from be - Maintenance of handwritten C - Bindings to an insufficiently general library - Burdensome dependencies -- Dead project +- Unmaintained project - Lack of post-quantum algorithms However, there is also a lot to be learned from the ways these prior efforts *were* successful. This proposal will assess these existing efforts in an attempt to consolidate their learnings and functionality. @@ -149,15 +184,13 @@ The health of the Haskell server ecosystem may be used as an example of a succes - Frontends like `blaze-html` and `lucid` - Libraries can take advantage of type system to provide good abstractions - Flexible enough abstractions to explore the problem space with multiple solutions -- Appropriate licensing +- Appropriate licensing that is free and unrestrictive, such as an MIT or BSD license -`servant` (and the `wai` ecosystem in general) is a success story and an example of higher-order programming making something safer and easier, something that we can aspire to do with cryptography. +`servant` (and the `wai` ecosystem in general) is a success story and an example of Haskell making something safer and easier, something that we can aspire to do with cryptography. ## Technical Content -This proposal is for the contruction of a suite of cryptography libraries, ranging from low-level bindings to specific algorithms, to higher-level abstractions and typeclasses. - -This leg of the proposal focuses on bindings to the `botan` cryptography, in the form of the following libraries: +This proposal is for the contruction of a set of bindings to the `botan` cryptography library, in the form of the following libraries: ### botan-bindings @@ -165,70 +198,51 @@ The `botan-bindings` library will contain raw bindings and is an almost direct, ### botan-low -The `botan-low` library will contain low-level bindings which wrap the FFI calls into monadic IO actions. This library handles the translation between buffers and ByteStrings, and throws exceptions in the case of errors, but is otherwise a fairly faithful translation of the interface. Thus this library is suitable for use, but is not recommended for day-to-day programming due to statefulness and a lack of type safety. +The `botan-low` library will contain low-level bindings which wrap the FFI calls into IO actions. This library will handle the translation between buffers and ByteStrings, and throw exceptions in the case of errors, but will otherwise be a fairly faithful translation of the Botan interface. This library is suitable for use, but is not recommended for day-to-day programming due to statefulness and use of strings for algorithm selection. ### botan -The `botan` library will contain high-level bindings which hides the stateful IO and exposes appropriately idiomatic and pure interfaces. It will provide data types for algorithms, enumerations for constants, convenience methods for key and nonce generation, incremental processing of lazy bytestrings, and otherwise bring significant safety and ergonomics over `botan-low` and `botan-bindings`. - -## Optional / Future Technical Content - -More advanced concepts such as typeclasses may be delivered in a future leg focused on generalization, but some development occur earlier in order to provide a testable 'use case' for the stack of libraries. If the first leg is completed ahead of schedule, the remaining effort will go towards these libraries. - -### crypto-schemes - -The `crypto-schemes` library will contain a hierarchy of abstract type classes and data families for cryptography and scheme identifiers, allowing for a unified interface to multiple back-ends. It will also provide various utility functions including sized bit and byte vectors, block and padding functions. - -### crypto-schemes-botan +The `botan` library will contain high-level bindings which hide the stateful IO and expose appropriately idiomatic or pure interfaces. It will provide data types for algorithms, enumerations for constants, convenience methods for key and nonce generation, incremental processing of lazy bytestrings, and otherwise bring significant safety and ergonomics over `botan-low` and `botan-bindings`. -The `crypto-schemes-botan` library will be an implementation of `crypto-schemes` using `botan`, implementing class instances for the specific algorithms that `botan` provides. +## Related Technical Content -### botanium - -The `botanium` library will contain user-friendly bindings to specific algorithms, designed for ease of use much like `libsodium`, but using `botan` or `crypto-schemes-botan` as a backend. - -### botanite - -The `botanite` library will contain an as-close-as-possible reimplementation of `crypton/ite` using `botan` or `crypto-schemes-botan` as a back-end, and may be kept as a separate library or merged into `crypton` at a future time. +More advanced concepts and data structures may be delivered in a future proposal focused on generalization, but some development may occur earlier in order to provide a testable downstream use case for the `botan` libraries. ## Goals and Non-goals -The following are the immediate goals of this first leg: +The following are the immediate goals of this proposal: - To publish the `botan`, `botan-low`, and `botan-bindings` libraries - To build proper documentation and tutorials covering the subject material -- To establish a community project presence focused on Haskell and cryptography +- To build unit tests and provide test vectors for provided algorithms -The following are overarching goals, and will not be complete by this first leg: +The following are goals, but may not be complete by this proposal: -- To publish the `botanium` libsodium- and `botanium` crypton- compatible libraries -- To publish the `crypto-schemes` library and `crypto-schemes-botan` backend. - To provide a suitable alternative interface / backend (as an option) for `crypton`, `tls`, `x509` - To consolidate Haskell cryptography into a consistent, unified, user-friendly system - To enable the safe use and implementation of advanced cryptographic data structures -- To provide cryptographic typeclasses for safe and consistent use -The following are not goals: +The following are not goals of this proposal: -- Finishing every goal by the first leg of the project +- Implementing a hierarchy of high-level cryptographic abstractions - Matching interfaces 1:1 with existing libraries / interfaces - Implementing every algorithm permutation possible ## Timeline -This timeline will focus on the first leg of the proposal, and the immediate goal of publishing the following libraries: +This timeline will focus on the immediate goal of publishing the following libraries: - botan-bindings - botan-low - botan -The full project has a large scope, and will require significant time to complete. More advanced concepts such as typeclasses may be delivered in a future leg focused on generalizing. +This proposal has a defined scope and set of deliverables, and will require substantial time to complete. -### Expected project leg duration +### Expected project duration -We estimate that it will take 3 months for this leg, to achieve our goal of developing and publishing these libraries. This estimate is based on the significant amount of work already completed, the length of time in which it was accomplished, and the remaining amount of work. +We estimate that it will take 3 months for this project, to achieve our goal of developing and publishing these libraries. This estimate is based on the significant amount of work already completed, the length of time in which it was accomplished, and the remaining amount of work. -Several work items (strictness, memory safety, C++ shims for missing FFI functionality) are difficult to estimate, and so time has been allocated for them in the estimate, but their delivery is considered optional, and may roll over into a future leg as necessary. +Several work items (strictness, memory safety, C++ shims for missing FFI functionality) are difficult to estimate, and so while time has been allocated for them in the estimate, their delivery is considered optional within the proposed timeline in order to ensure completion of the immediate goals. If necessary, the project may be extended at the discretion of the Haskell Foundation in order to ensure their completion. ### Intermediate deliverables @@ -240,7 +254,7 @@ The following are intermediate deliverables intended to keep the community appri ### Deliverables -The following are the intended deliverables of this leg of the proposal: +The following are the intended deliverables of this proposal: - `botan-bindings` - `botan-low` @@ -250,32 +264,32 @@ Each library should be beta- or / production-ready, and submitted to hackage as Each library should have: +- Installation instructions for the Botan library - Documentation - Tutorials and examples - Tracked issues - Unit tests +- CI tests for Linux, MacOS, and Windows -A project website will also be created. - -The official repo will be owned by the Haskell Cryptography Group, and another member of the Haskell Cryptography Group will be selected to hold backup keys and permissions to the website, server, and official repositories as needed, in order to give project members access and to reduce any conflicts about ownership or maintenance. +The official repo will be owned by the [Haskell Cryptography Group ](https://github.com/haskell-cryptography) and maintained by myself as a member of the group. Another member of the Haskell Cryptography Group, or a member of the Haskell Foundation, will be selected to hold backup keys and permissions to the website, server, and official repositories as needed, in order to give project members access and to reduce any conflicts about ownership or maintenance. ## Budget -The first leg of this proposal presents a minimum budget of $7000 USD per month, for one full-time engineer, for a duration of 3 months, for a total of $21,000 USD. This budget is based on cost-of-living and industry experience, and will cover housing, food, bills, taxes, and other life necessities for one engineer, as well as any project necessities such website and server hosting. This budget is roughly equivalent to $40 / hr or $84k / yr at full-time of 40 hrs / wk. Industry rates for an engineer of the necessary skill are on average considerably higher, and so we consider this budget to be reasonable. The exact legal contract / arrangement is left to the Haskell Foundation. - -The question of continued funding will be revisited for each future leg of this proposal, wherein the budget and direction of future work will be decided again for each leg. +This proposal presents a minimum budget of $7000 USD per month, for one full-time engineer, for a minimum duration of 3 months, for an estimated total cost of $21,000 USD. This budget is based on cost-of-living and industry experience, and will cover housing, food, bills, taxes, and other life necessities for one engineer, as well as any project necessities such website and server hosting. This budget is roughly equivalent to $40 / hr or $84k / yr at full-time of 40 hrs / wk. Industry rates for an engineer of the necessary skill are on average considerably higher, and so we consider this budget to be reasonable. The exact legal contract / arrangement is left to the Haskell Foundation, and this timeline and budget may be extended at their discretion under the same terms. ## Stakeholders +### Positive stakeholders + Several parties stand to gain from the implementation of this proposal. - Leon Dillinger (Myself) -As the intended engineer carrying out the work, I have already invested significant effort in this project in order to establish its viability, and will benefit from this proposal in the form of being the recipient of the funding. +As the intended engineer and member of the Haskell Cryptography Group carrying out the work, I have already invested significant effort in this project in order to establish its viability, and will benefit from this proposal in the form of being the recipient of the funding. - [Haskell Foundation](https://haskell.foundation/) -As the intended source of funding, the Haskell Foundation would be invested in this proposal financially, and its effective success or failure would reflect respectively upon the Haskell Foundation. +As the intended source of funding, the Haskell Foundation would be invested in seeing that its funding is spent effectively. - [Haskell Cryptography Group ](https://github.com/haskell-cryptography) @@ -283,47 +297,97 @@ As the intended owner of the project, this project will be new maintenance burde - Users of existing cryptography libraries such as `crypton`, `cryptonite`, `libsodium / saltine`. -As the intended target audience of this project, this proposal aims to directly benefit developers by providing high-quality cryptography libraries at several appropriate levels of abstraction, but will also place the burden of migration to these new libraries on developers. +As the intended target audience of this project, this proposal aims to directly benefit developers by providing high-quality cryptography libraries with an appropriate level of abstraction, but will also place the burden of migration to these new libraries on developers. + +- Downstream users of Haskell software + +As the eventual user of the developed software, their safety and security will rest on the proper execution of this proposal. + +- [Botan / Randombit](https://botan.randombit.net/) + +As the developers of the `Botan` library, this project depends on their work, and some of this work may result in contributions back upstream to the `Botan` source code (such as any improvements / C++ shims). + +### Negative stakeholders -- Downstream users of Haskell software, developers with +- Supply-chain attacker -As the eventual user of the developed software, their safety and security will rest on the proper execution of this proposal. +As an attacker, every open-source project is a target for silently inserting compromised code into the codebase through PRs with the goal of penetrating downstream codebases. By relying on bindings to a public, well-scrutinized library, this project reduces the available attack surface comparative to implementing it ourselves. ## Risks There are a number of risks. -- Cryptography is hard and each algorithm may have individual nuances or uncommon requirements. +- There are many algorithm-specific nuances that are absent from library documentation - this makes securing each algorithm an individual albeit minor effort that requires often consulting external documentation - Botan FFI is incompletely documented and has gaps (stream ciphers, X509 stores, a few broken functions) which need to be fixed with C++ shims - Strictness, memory safety & scrubbing, C++ shims will take an unknown amount of time - A thorough / safe implementation will require consulting the C++ source of the FFI - Polynomial combinations of algorithms makes exhaustive testing complicated. -These risks are amortized in that while they are tedious and potentially time-consuming, they may be considered extended goals of high value, and there is still substantial value that can be delivered without them, and the completion of risk items may continue in a future proposal leg. +These risks are amortized in that while they are tedious and potentially time-consuming, they relate to extended goals of high value, and while there is still substantial value that can be delivered without them, the completion of risk-related items may be ensured by extending this proposal's timeline and funding if necessary. - Single point of failure -This is a proposal for one full-time engineer; this constitutes a single point of failure. To combat this, the official github repo will be owned by the Haskell Cryptography Group, and another member of the Haskell Cryptography Group will be selected to hold backup keys and permissions to the website, server, and official repositories as needed. In the case of an emergency for which I am unreachable, this will allow for others to take over as necessary. +This is a proposal for one full-time engineer; this constitutes a single point of failure. To combat this, the official github repo will be owned by the Haskell Cryptography Group, and another member of the Haskell Cryptography Group, or a member of the Haskell Foundation, will be selected to hold backup keys and permissions to any servers and official repositories as needed. In the case of an emergency for which I am unreachable, this will allow for others to take over as necessary. ## Success -This leg of the project will be considered a success if it results in the submission of a candidate or accepted package to hackage, with production-ready bindings, documentation, tutorials, and unit tests as appropriate for the following botan-related libraries: +This proposal will be considered a success if it results in the submission of a candidate or accepted package to hackage, with production-ready bindings, documentation, tutorials, and unit tests as appropriate for the following botan-related libraries: - botan-bindings - botan-low - botan -The following Botan library features are considered optional to the success of this leg, due to requirement of C++ / FFI shims or additional research which will take an unknown amount of work, but effort will be made to include them if possible within the timeframe. +The following Botan library features are goals, but are considered optional to the success of this proposal, due to requirement of C++ / FFI shims or additional research which will take an unknown amount of work. - Extended X509 support - Stream ciphers - Test vectors for algorithms, especially [CAVP / FIPS / NIST-approved](https://csrc.nist.gov/projects/cryptographic-algorithm-validation-program) -The following libraries are considered non-essential to the success of this leg, but may be worked on as they provide a valuable use case with which to test the lower libraries against, and may be subsumed into another at any time (eg, botanite could be merged into crypton). +Effort will be made to include them if possible within the proposed timeline, and the project may be extended at the discretion of the Haskell Foundation in order to ensure their completion. + + +# Appendix A - Commercial Incentive Misalignment + +There is no open, community-driven, community-owned cryptography solution, and it is unlikely that one will be developed without community investment. Commercial development is unlikely to provide us with a solution, due to the issue of ownership of the resulting ecosystem, and the resulting incentive misalignment: + +- Cryptography and security are preventative measures and a cost center for most companies +- Companies for which cryptography / security are a product have a strong incentive to marshal users into a walled garden in the name of both security and profit +- Developing an ecosystem is a significant up-front investment, and so investors may expect ownership of the ecosystem in order to recoup their cost +- There is an incentive to wait for someone else to develop an open ecosystem, instead of expending resources yourself to do it + +As such, commercial or for-profit approaches are incentivized to either invest in a private ecosystem, or invest in an established open ecosystem, but are perversely incentivized against building a new open ecosystem. + +# Appendix B - Lack of ByteString `Bits` instance + +Essentially, `Bits`` (and the fixed-width `FiniteBits``) is a class that combines two concepts: + + `Boolean / Heyting`` algebra, for which there is no concept of any 'internal bits' or structure and you are just operating on some object as a whole + Things that are contructed from / encoded into a set of 'internal bits' using some encoding, for which individual bits are indexable / addressible, and thus individually boolean-operable. There is also a link here to indexing / representable, but that is getting way out of scope. + +Notably, the documentation itself further states that "The `Bits` class defines bitwise operations over integral types.". Technically this means that only things that are integer numbers should be `Bits` and `FiniteBits`, but Godel numbering rears its ugly head, and that just adds the question of whether `Bits`'s boolean operations are intended to apply to an object itself or its encoding, and as a result `Bits` and `FiniteBits` are restricted to objects for which the boolean operations are isomorphic to boolean operations over its encoding. + +Given this understanding, `ByteString` does not have a canonical `Bits` instance because it has no canonical integer representation - a given bytestring could represent many such integers, whether it be big endian or little endian, it could have a sign bit, or be two's complement. Its bits might not be contiguous or even ordered. It could easily satisfy a `Boolean` instance, but the issue of indexing individual bits and integral type requirement interferes with a full instance of `Bits`. + +# Appendix C - NaCl / libsodium is not a cryptographic sink + +NaCl / libsodium is not a general purpose cryptography library; it is a highly opinionated and minimalist cryptography suite designed for safe and ergonomic use at the expense of control. + +- It makes all of the decisions for you +- It only implements a handful of common primitives +- It only implements a single algorithm variant for each primitive +- Interop with non- NaCl systems is difficult to impossible if NaCl doesn't support the algorithm or primitive. + +The idea behind NaCl is that the developer doesn't need to worry about the algorithms - they've chosen great defaults for most crypto use cases, and provided a high-level interface to help you implement it correctly. + +In fact, NaCl only provides: -- botanium -- botanite -- crypto-schemes -- crypto-schemes-botan +- SHA-2 (or sometimes Blake2b) hashing +- HMAC-SHA-512/256 message authentication +- AES-GCM cipher encryption +- Salsa20 stream encryption +- Ed25519 / X25519 public key signatures / agreement +- Poly1305-Wegman-Carter one-time authentication +- Salsa20-Poly1305 AEAD +- X25519-Salsa20-Poly1305 public key encryption -The next leg of this proposal is still undetermined, but tentatively will be focused on compatibility with `crypton` through `botanite`, and on the development of abstract cryptography classes in `crypto-schemes`. +Granted, NaCl does a good job of choosing a commonly used set of algorithms, but it is unsuitable as a general purpose cryptography library because it does not support anything except for these few select algorithms and primitives. If you need SHA3 hashing, or CBC mode for decryption, or RSA public / private keys, or arbitrary named elliptic curves, then NaCl can't really help. From a87c8285d8c3b4477702dc9b32186185ddd1c766 Mon Sep 17 00:00:00 2001 From: Leo Date: Mon, 9 Oct 2023 15:37:42 -0500 Subject: [PATCH 3/7] Formatting correction --- proposals/052-cryptography-leg-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/052-cryptography-leg-1.md b/proposals/052-cryptography-leg-1.md index fd74f759..ff906dc0 100644 --- a/proposals/052-cryptography-leg-1.md +++ b/proposals/052-cryptography-leg-1.md @@ -2,7 +2,7 @@ # Abstract -This community project proposal is for full-time funding for the development of a suite of Haskell cryptography libraries and tooling suitable a wide range of uses including data integrity, privacy, security, and networking. This is necessary to relieve the burden [of correctly implementing cryptography] from Haskell developers seeking to provide privacy and security to their users. We propose the development of a community-owned set of cryptography libraries of increasing capability, starting with bindings to a compatible open-source cryptography library. This is expected to require a considerable effort over time, and will be broken up into individual proposals, each of which will be accompanied by its own specific set of goals. +This community project proposal is for full-time funding for the development of a suite of Haskell cryptography libraries and tooling suitable a wide range of uses including data integrity, privacy, security, and networking. This is necessary to relieve the burden of correctly implementing cryptography from Haskell developers seeking to provide privacy and security to their users. We propose the development of a community-owned set of cryptography libraries of increasing capability, starting with bindings to a compatible open-source cryptography library. This is expected to require a considerable effort over time, and will be broken up into individual proposals, each of which will be accompanied by its own specific set of goals. This first proposal is for the development of high-quality, production-ready bindings to the Botan C++ cryptography library, and from acceptance to release on Hackage is estimated to take 3 months of full-time development. We see several challenges but no significant technical risk to this proposal; known challenges involve strictness and developing additional bindings to C++; these items are of high-value and may take significant effort to resolve, but are known to be solvable. Considerable progress has already been made, and the proposed funding will help ensure its completion. From 6cf84e50c3a41558a9e5af406e577d372989f5b1 Mon Sep 17 00:00:00 2001 From: Leo Date: Thu, 12 Oct 2023 10:34:54 -0500 Subject: [PATCH 4/7] Wording improvements, cryptohash-* variants mentioned, clarification of ownership, clarified bytestring-bits scope --- proposals/052-cryptography-leg-1.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/proposals/052-cryptography-leg-1.md b/proposals/052-cryptography-leg-1.md index ff906dc0..19cd8234 100644 --- a/proposals/052-cryptography-leg-1.md +++ b/proposals/052-cryptography-leg-1.md @@ -18,13 +18,13 @@ The collective properties of referential transparency, type systems, and monadic # Problem Statement -The Haskell cryptography ecosystem lacks significant capability beyond basic primitives. This places a significant burden on developers to properly implement various security techniques, and it exposes end users to significant risk in the event of a lapse in security. There are commercial solutions, but there is no community-driven, community-owned solution, and one is unlikely to be developed without deliberate community effort (see Appendix A - Commercial Incentive Misalignment). +The Haskell cryptography ecosystem lacks significant capability beyond basic primitives. This places a significant burden on developers to properly implement various security techniques, and it exposes end users to significant risk in the event of a lapse in security. There may be commercial solutions, but there is no community-driven, community-owned solution, and one is unlikely to be developed without deliberate community effort (see Appendix A - Commercial Incentive Misalignment). The Haskell cryptography ecosystem is also brittle and outdated. It is the product of considerable flux / churn over the years as various libraries have been developed and then deprecated or abandoned in favor of newer ones. The resulting libraries have highly-concrete interfaces and are strongly influenced by their implementation, both violating the abstraction barrier and preserving the design across successive libraries - as if their backend were to be changed, it would likely require changing or breaking the interface in turn. As a result, there are multiple cryptography libraries with similar interfaces, and it is not always clear which should be used preferentially. This promotes rot as downstream users unknownly rely on unmaintained libraries that slowly stop working. -Furthermore, a number of important libraries in the ecosystem (eg, `cryptonite`) have recently been abandoned by their author. Their backend is written in C, which presents a long-term maintenance burden with a high knowledge and skill requirement. NaCl bindings such as `saltine` are a high-quality alternative, but solve only a limited set of problems. +Furthermore, a number of important libraries in the cryptography ecosystem (eg, `cryptonite`, `asn1-encoding`, and `memory`) have recently been archived by their author and will no longer be receiving updates. Their backend is written in C, which presents a long-term maintenance burden with a high knowledge and skill requirement. NaCl bindings such as `saltine` are a high-quality alternative, but solve only a limited set of problems. The resulting brittleness has placed the long-term stability of important libraries such as `crypton`, `tls` and `x509` at risk of falling behind as compilers are upgraded, old standards are updated, and new standards are implemented. Additionally the instability creates an unattractive environment for developers, effectively driving them away to other ecosystems that can provide the necessary stability. @@ -140,6 +140,7 @@ Specific libraries worth noting are: - Predecessors to `cryptonite` - all deprecated - [cryptohash](https://hackage.haskell.org/package/cryptohash) + - `cryptohash-md5`, `-sha1`, `-sha256`, `-sha512` are reduced-footprint forks - [cryptocipher](https://hackage.haskell.org/package/cryptocipher) - [crypto-pubkey](https://hackage.haskell.org/package/crypto-pubkey) - [crypto-random](https://hackage.haskell.org/package/crypto-random) @@ -271,7 +272,7 @@ Each library should have: - Unit tests - CI tests for Linux, MacOS, and Windows -The official repo will be owned by the [Haskell Cryptography Group ](https://github.com/haskell-cryptography) and maintained by myself as a member of the group. Another member of the Haskell Cryptography Group, or a member of the Haskell Foundation, will be selected to hold backup keys and permissions to the website, server, and official repositories as needed, in order to give project members access and to reduce any conflicts about ownership or maintenance. +The official repo will be owned by the [Haskell Foundation](https://haskell.foundation/), and maintained by the [Haskell Cryptography Group](https://github.com/haskell-cryptography), with myself as the current maintainer. Another member of the Haskell Cryptography Group, or a member of the Haskell Foundation, will be selected to hold backup keys and permissions to any website, server, and official repositories as needed, in order to give project members access and to reduce any conflicts about ownership or maintenance. ## Budget @@ -359,14 +360,16 @@ As such, commercial or for-profit approaches are incentivized to either invest i # Appendix B - Lack of ByteString `Bits` instance -Essentially, `Bits`` (and the fixed-width `FiniteBits``) is a class that combines two concepts: +Essentially, `Bits` (and the fixed-width `FiniteBits`) is a class that combines two concepts: - `Boolean / Heyting`` algebra, for which there is no concept of any 'internal bits' or structure and you are just operating on some object as a whole + `Boolean / Heyting` algebra, for which there is no concept of any 'internal bits' or structure and you are just operating on some object as a whole Things that are contructed from / encoded into a set of 'internal bits' using some encoding, for which individual bits are indexable / addressible, and thus individually boolean-operable. There is also a link here to indexing / representable, but that is getting way out of scope. Notably, the documentation itself further states that "The `Bits` class defines bitwise operations over integral types.". Technically this means that only things that are integer numbers should be `Bits` and `FiniteBits`, but Godel numbering rears its ugly head, and that just adds the question of whether `Bits`'s boolean operations are intended to apply to an object itself or its encoding, and as a result `Bits` and `FiniteBits` are restricted to objects for which the boolean operations are isomorphic to boolean operations over its encoding. -Given this understanding, `ByteString` does not have a canonical `Bits` instance because it has no canonical integer representation - a given bytestring could represent many such integers, whether it be big endian or little endian, it could have a sign bit, or be two's complement. Its bits might not be contiguous or even ordered. It could easily satisfy a `Boolean` instance, but the issue of indexing individual bits and integral type requirement interferes with a full instance of `Bits`. +Given this understanding, `ByteString` does not have a canonical `Bits` instance because it has no canonical integer representation - a given bytestring could represent many such integers, whether it be big endian or little endian, it could have a sign bit, or be two's complement. Its bits might not be contiguous or even ordered. It could easily satisfy a hypothetical `Boolean` instance, but the issue of indexing individual bits and integral type requirement interferes with a full instance of `Bits`. + +Solving this issue is out of scope of this proposal, and the tangible impact is low as it is easily solved with a simple newtype wrapper. # Appendix C - NaCl / libsodium is not a cryptographic sink From b9110433f0a9785d96aebcbff1ce8f96899358f5 Mon Sep 17 00:00:00 2001 From: Leo Date: Thu, 12 Oct 2023 11:41:16 -0500 Subject: [PATCH 5/7] Added benchmarks --- proposals/052-cryptography-leg-1.md | 45 ++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/proposals/052-cryptography-leg-1.md b/proposals/052-cryptography-leg-1.md index 19cd8234..0d51e8bd 100644 --- a/proposals/052-cryptography-leg-1.md +++ b/proposals/052-cryptography-leg-1.md @@ -120,7 +120,50 @@ ghci> hashPassword 16 ("Fee fi fo fum!" :: ByteString) :: IO ByteString (8.84 secs, 2,138,266,840 bytes) ``` -It was quick and dirty testing in GHCi so your mileage may vary, but the results indicate that bindings to Botan may be significantly faster and consumes less memory than `crypton/ite`, if other modules / functions are similarly performant. Further implentation, testing, and benchmarking will be necessary to confirm this. +Further testing of `Bcrypt` and `SHA-3` with benchmarking using `tasty-bench` shows a considerable increase in speed, although `Botan` memory usage cannot be accurately measured due to the FFI. + +``` +$ cabal bench --benchmark-options '+RTS -T' botan-low-bench +Benchmark botan-low-bench: RUNNING... +All + Bcrypt work factor + Crypton + twelve: OK (1.65s) + 550 ms ± 27 ms, 109 MB allocated, 30 KB copied, 34 MB peak memory + fourteen: OK (6.57s) + 2.198 s ± 29 ms, 492 MB allocated, 21 KB copied, 34 MB peak memory + sixteen: OK (26.38s) + 8.797 s ± 34 ms, 2.0 GB allocated, 32 KB copied, 34 MB peak memory + Botan + twelve: OK (1.39s) + 232 ms ± 14 ms, 0 B allocated, 0 B copied, 34 MB peak memory + fourteen: OK (1.85s) + 925 ms ± 30 ms, 0 B allocated, 0 B copied, 34 MB peak memory + sixteen: OK (11.10s) + 3.700 s ± 40 ms, 0 B allocated, 0 B copied, 34 MB peak memory + Hash + Crypton + password: OK (1.17s) + 1.10 μs ± 106 ns, 3.0 KB allocated, 0 B copied, 45 MB peak memory + plaintext: OK (1.75s) + 1.66 μs ± 109 ns, 3.0 KB allocated, 0 B copied, 45 MB peak memory + longtext: OK (2.31s) + 9.04 ms ± 485 μs, 0 B allocated, 0 B copied, 45 MB peak memory + Botan + password: OK (2.30s) + 1.09 μs ± 61 ns, 1.1 KB allocated, 93 B copied, 48 MB peak memory + plaintext: OK (1.58s) + 1.49 μs ± 119 ns, 1.2 KB allocated, 93 B copied, 49 MB peak memory + longtext: OK (1.42s) + 5.55 ms ± 423 μs, 1.0 MB allocated, 117 B copied, 97 MB peak memory + +All 12 tests passed (60.66s) +Benchmark botan-low-bench: FINISH +``` + +Results indicate that bindings to Botan may be significantly more performant than `crypton/ite`, if other modules / functions are similarly performant. + +Note that the memory use increase in Hash.Botan.longtext is due to the rough state of initial bindings, as the bindings are not yet leak-free. Despite this, it is still approximately twice as fast. # Prior Art and Related Efforts From 84ee485c776f69610e6a48a88e55fd77811acddc Mon Sep 17 00:00:00 2001 From: Leo Date: Thu, 12 Oct 2023 14:57:29 -0500 Subject: [PATCH 6/7] Updated wording in response to notes --- proposals/052-cryptography-leg-1.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/052-cryptography-leg-1.md b/proposals/052-cryptography-leg-1.md index 0d51e8bd..62df274d 100644 --- a/proposals/052-cryptography-leg-1.md +++ b/proposals/052-cryptography-leg-1.md @@ -181,7 +181,7 @@ The wiki lists [several libraries](https://wiki.haskell.org/Applications_and_lib Specific libraries worth noting are: -- Predecessors to `cryptonite` - all deprecated +- Predecessors to `cryptonite` - [cryptohash](https://hackage.haskell.org/package/cryptohash) - `cryptohash-md5`, `-sha1`, `-sha256`, `-sha512` are reduced-footprint forks - [cryptocipher](https://hackage.haskell.org/package/cryptocipher) @@ -200,7 +200,7 @@ Specific libraries worth noting are: - [Crypto](https://hackage.haskell.org/package/Crypto) - a very old sink - [crypto-api](https://hackage.haskell.org/package/crypto-api) - cryptographic classes -These prior efforts can be considered to be somewhat partially successful in presenting Haskell developers with access to cryptographic primitives, but present a significant development challenge and a high refactoring cost if dependencies become deprecated, which happens often. Of these libraries, a considerable amount are deprecated or have been dead for many years, with `cryptonite`, `crypton`, and `saltine` being clear front-runners in terms of current usage and transitive dependencies (and `sel` being under active development). +These prior efforts can be considered to be somewhat partially successful in presenting Haskell developers with access to cryptographic primitives, but present a significant development challenge and a high refactoring cost if dependencies become deprecated, which happens often. Of these libraries, a considerable amount are deprecated or archived, with `cryptonite`, `crypton`, and `saltine` being clear front-runners in terms of current usage and transitive dependencies (and `sel` being under active development). Of these four, one is a fork of another (`crypton` and `cryptonite`), and the other two are bindings to `libsodium` which does not sufficiently capture our need for low-level cryptographic primitives in general. `crypton/ite` exposes a few highly- opinionated / specific classes for hashing, block and stream ciphers, but otherwise is focused on concrete implementations and functions. Both `crypton` and `cryptonite` rely on bindings to C implementations that have not undergone the same vetting as commonly-used C libraries; though the code is itself not suspect, the lack of provenance makes it ill-suited for recommendation in trusted environments, and maintenance of the C implementation poses burden of high technical skill. `libsodium` bindings are more trustworthy, but the limited options severely hamper interop with other systems unless they too are using `libsodium`, making them ill-suited for recommendation for some use cases. @@ -333,11 +333,11 @@ As the intended engineer and member of the Haskell Cryptography Group carrying o - [Haskell Foundation](https://haskell.foundation/) -As the intended source of funding, the Haskell Foundation would be invested in seeing that its funding is spent effectively. +As the intended owner and source of funding for this project, the Haskell Foundation would be invested in seeing that its funding is spent effectively. - [Haskell Cryptography Group ](https://github.com/haskell-cryptography) -As the intended owner of the project, this project will be new maintenance burden on the Haskell Cryptography Group. +As the intended maintainer of the project, this project will be new burden on the Haskell Cryptography Group. - Users of existing cryptography libraries such as `crypton`, `cryptonite`, `libsodium / saltine`. From 531ee75e4fb54e5885b3a71d97fa1bfb7ea43932 Mon Sep 17 00:00:00 2001 From: Leo Date: Mon, 16 Oct 2023 10:55:25 -0500 Subject: [PATCH 7/7] Improved NaCl description, improved cryptocurrency section in problem statement, improved quantum computing mention, improved botan vs botan-low description, added Future Work section --- proposals/052-cryptography-leg-1.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/proposals/052-cryptography-leg-1.md b/proposals/052-cryptography-leg-1.md index 62df274d..7dc9b093 100644 --- a/proposals/052-cryptography-leg-1.md +++ b/proposals/052-cryptography-leg-1.md @@ -24,7 +24,9 @@ The Haskell cryptography ecosystem is also brittle and outdated. It is the produ As a result, there are multiple cryptography libraries with similar interfaces, and it is not always clear which should be used preferentially. This promotes rot as downstream users unknownly rely on unmaintained libraries that slowly stop working. -Furthermore, a number of important libraries in the cryptography ecosystem (eg, `cryptonite`, `asn1-encoding`, and `memory`) have recently been archived by their author and will no longer be receiving updates. Their backend is written in C, which presents a long-term maintenance burden with a high knowledge and skill requirement. NaCl bindings such as `saltine` are a high-quality alternative, but solve only a limited set of problems. +Furthermore, a number of important libraries in the cryptography ecosystem (eg, `cryptonite`, `asn1-encoding`, and `memory`) have recently been archived by their author and will no longer be receiving updates. Their backend is written in C, which presents a long-term maintenance burden with a high knowledge and skill requirement. + +NaCl bindings such as `saltine` are a high-quality alternative, but solve only a limited set of problems, as they only provide a few algorithms for select basic primitives, and do not support more complex protocols such as TLS and X509 (see Appendix C - NaCl / libsodium is not a cryptographic sink). The resulting brittleness has placed the long-term stability of important libraries such as `crypton`, `tls` and `x509` at risk of falling behind as compilers are upgraded, old standards are updated, and new standards are implemented. Additionally the instability creates an unattractive environment for developers, effectively driving them away to other ecosystems that can provide the necessary stability. @@ -36,7 +38,7 @@ There are several additional reasons for this lack of suitable engineers: 3) There is a persistent notion that functional languages are too slow for cryptography, despite it being based on historical reasons that have been mostly since addressed. Well-written modern Haskell approaches C in terms of speed, and has a much higher ergonomic-use and type-safety factor, but the public image has not caught up yet, and so the ecosystem has not received the investment and attention necessary to attract the attention of skilled engineers for developing either pure implementations or foreign bindings. -4) A fair number of cryptographic projects in recent years has been specifically towards *cryptocurrencies*, rather than *cryptography in general*, and their results are at large fairly useless if they cannot be used outside of their ecosystems. +4) A fair number of cryptographic projects in recent years has been specifically towards *cryptocurrencies*, rather than *cryptography in general*, and their results are not always applicable outside of their specific ecosystems or use case. All of these reasons have contributed to the stagnant state of Haskell cryptography, and have make it an unfavorable environment for application developers, especially developers of cryptosystems and distributed systems. However, none of these are actual blockers, and each of these things can be corrected with effort. There is much that can be done to improve the Haskell cryptography ecosystem, ranging from better low-level support, to higher-level abstractions: @@ -101,7 +103,7 @@ The others were not selected for several reasons: - libgcrypt is C, and so the Haskell FFI can bind directly to it, but it has a GNU LGPL license, and provides less functionality than Botan - Crypto++ is C++, and has a Boost license (similar to BSD), but it does not export a C API, and provides less functionality than Botan -- libsodium / NaCL does not provide the necessary functionality, and there are several Haskell libsodium bindings already (see Appendix C - NaCl / libsodium) +- libsodium / NaCL does not provide the necessary functionality, and there are several Haskell libsodium bindings already (see Appendix C - NaCl / libsodium is not a cryptographic sink) - NSS has a copyleft license Bindings to Botan would solve the significant problem of dealing with common cryptography by providing much of the necessary 'cryptographic kitchen sink' via bindings to a suitably performant, suitably licensed, open-source library. The Haskell community would not be required to maintain the Botan cryptography library, only bindings to it, and this can be accomplished readily with some effort which has already begun. Furthermore, the Botan library supports several post-quantum cryptography algorithms, which is highly desirable. @@ -206,7 +208,7 @@ Of these four, one is a fork of another (`crypton` and `cryptonite`), and the ot Following that, there is also `Z-Botan`, which has been apparently unmaintained for several years. Reviving it was considered, but would require a comprehensive and near-total rewrite in order to divest itself of `Z-Haskell` as a dependency. New bindings could instead draw significantly from `Z-Botan`, while presenting a much smaller maintenance and dependency surface. -Furthermore, none of these libraries contain implementations of post-quantum cryptography schemes; although existing quantum computers are of insufficient capability to break commonly used algorithms such as Ed25519, this will not remain true forever, and it would be best to have quantum-resistant implementations ready for adoption long before such attacked become practical. +Furthermore, none of these libraries contain implementations of post-quantum cryptography schemes; although existing quantum computers are of insufficient capability to break commonly used algorithms such as Ed25519, this will not remain true forever, and it is necessary to have quantum-resistant implementations ready for adoption long before such attacks become practical, as it is sufficient to capture the encrypted traffic now and decode it later when quantum computers have advanced in power. Each of these efforts has one or more of these issues that preclude them from being considered a complete success: @@ -242,11 +244,11 @@ The `botan-bindings` library will contain raw bindings and is an almost direct, ### botan-low -The `botan-low` library will contain low-level bindings which wrap the FFI calls into IO actions. This library will handle the translation between buffers and ByteStrings, and throw exceptions in the case of errors, but will otherwise be a fairly faithful translation of the Botan interface. This library is suitable for use, but is not recommended for day-to-day programming due to statefulness and use of strings for algorithm selection. +The `botan-low` library will contain low-level bindings which wrap the FFI calls into IO actions. This library will handle the translation between buffers and ByteStrings, and throw exceptions in the case of errors, but will otherwise be a fairly faithful translation of the Botan interface. This library is suitable for use, but is not recommended for day-to-day programming due to its low level interface, statefulness, and use of strings for algorithm selection. ### botan -The `botan` library will contain high-level bindings which hide the stateful IO and expose appropriately idiomatic or pure interfaces. It will provide data types for algorithms, enumerations for constants, convenience methods for key and nonce generation, incremental processing of lazy bytestrings, and otherwise bring significant safety and ergonomics over `botan-low` and `botan-bindings`. +The `botan` library will contain high-level bindings which hide the stateful IO and expose appropriately idiomatic or pure interfaces. It will provide data types for algorithms, enumerations for constants, convenience methods for key and nonce generation, incremental processing of lazy bytestrings, and otherwise bring significant safety and ergonomics over `botan-low` and `botan-bindings`. This library is suitable for use and is recommended for production use due to idiomatic interfaces and improved safety. ## Related Technical Content @@ -389,6 +391,13 @@ The following Botan library features are goals, but are considered optional to t Effort will be made to include them if possible within the proposed timeline, and the project may be extended at the discretion of the Haskell Foundation in order to ensure their completion. +# Future Work + +There were several goals that were considered for this proposal, but were excluded from scope in order to keep the focus narrow. This represents future work that we would like to continue in the furtherance of development of the Haskell cryptography ecosystem. + +Assuming that this proposal is a success, we would then like to take the next logical step of integrating `botan` with one or more existing libraries as a backend. Candidates goals range from integrating with or providing a compatibility later as a drop-in replacement for `crypton`, to integrating with critical low-level libraries such as `tls` or `x509` directly, to integrating with more "consumer" facing libraries such as `hs-jose` or `clientsession`. + +These goals will be kept in mind as design factors during the implementation of this proposal, as the intended subject(s) of an potential future proposal, and efforts will be made to ensure compatibility with and prepare for integrating with the existing ecosystem. # Appendix A - Commercial Incentive Misalignment