From 79a8539d4edcaa2d050b02510f2ff8d8caa1211f Mon Sep 17 00:00:00 2001 From: Aaron Turon Date: Fri, 21 Oct 2016 16:01:20 -0700 Subject: [PATCH 01/10] Roadmap for 2017 --- text/0000-roadmap-2017.md | 582 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 582 insertions(+) create mode 100644 text/0000-roadmap-2017.md diff --git a/text/0000-roadmap-2017.md b/text/0000-roadmap-2017.md new file mode 100644 index 00000000000..476934b1377 --- /dev/null +++ b/text/0000-roadmap-2017.md @@ -0,0 +1,582 @@ +- Feature Name: N/A +- Start Date: 2016-10-04 +- RFC PR: (leave this empty) +- Rust Issue: (leave this empty) + +# Summary +[summary]: #summary + +This RFC proposes the *2017 Rust Roadmap*, in accordance with +[RFC 1728](https://github.com/rust-lang/rfcs/pull/1728). The goal of the roadmap +is to lay out a vision for where the Rust project should be in a year's +time. **This year's focus is improving Rust's *productivity*, while retaining its +emphasis on fast, reliable code**. At a high level, by the end of 2017: + +* Rust should have a lower learning curve +* Rust should have a pleasant edit-compile-debug cycle +* Rust should provide a solid, but basic IDE experience +* Rust should provide easy access to high quality crates +* Rust should be well-equipped for writing robust, high-scale servers +* Rust should have 1.0-level crates for essential tasks +* Rust should integrate easily into large build systems +* Rust should integrate easily with C++ code +* Rust's community should provide mentoring at all levels + +The proposal is based on the [2016 survey], systematic outreach, direct +conversations with individual Rust users, and an extensive +[internals thread]. Thanks to everyone who helped with this effort! + +[2016 survey]: https://blog.rust-lang.org/2016/06/30/State-of-Rust-Survey-2016.html +[internals thread]: https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/ + +# Motivation +[motivation]: #motivation + +There's no end of possible improvements to Rust—so what do we use to guide our +thinking? + +The core team has tended to view things not in terms of particular features or +aesthetic goals, but instead in terms of **making Rust successful while staying +true to its core values**. This basic sentiment underlies much of the proposed +roadmap, so let's unpack it a bit. + +## Making Rust successful + +### The measure of success + +What does it mean for Rust to be successful? There are a lot of good answers to +this question, a lot of different things that draw people to use or contribute +to Rust. But regardless of our *personal* values, there's at least one clear +measure for Rust's broad success: **people should be using Rust in +production and reaping clear benefits from doing so**. + +- Production use matters for the obvious reason: it grows the set of + stakeholders with potential to invest in the language and ecosystem. To + deliver on that potential, Rust needs to be part of the backbone of some major + products. + +- Production use measures our *design* success; it's the ultimate reality + check. Rust takes a unique stance on a number of tradeoffs, which we believe + to position it well for writing fast and reliable software. The real test of + those beliefs is people using Rust to build large, production systems, on + which they're betting time and money. + +- The *kind* of production use matters. For Rust to truly be a success, there + should be clear-cut reasons people are employing it rather than another + language. Rust needs to provide crisp, standout benefits to the organizations + using it. + +The idea here is *not* about "taking over the world" with Rust; it's not about +market share for the sake of market share. But if Rust is truly delivering a +valuable new way of programming, we should be seeing that benefit in "the real +world", in production uses that are significant enough to help sustain Rust's +development. + +### The obstacles to success + +At this point, we have a fair amount of data about how Rust is reaching its +audience, through the [2016 survey], informal conversations, and explicit +outreach to (pre-)production shops (writeup coming soon). The data from the +survey is generally corroborated by these other venues, so let's focus on that. + +[2016 survey]: https://blog.rust-lang.org/2016/06/30/State-of-Rust-Survey-2016.html + +We asked both current and potential users what most stands in the way of their +using Rust, and got some pretty clear answers: + +- 1 in 4: learning curve +- 1 in 7: lack of libraries +- 1 in 9: general “maturity” concerns +- 1 in 19: lack of IDEs (1 in 4 non-users) +- 1 in 20: compiler performance + +None of these obstacles is directly about the core language or `std`; people are +generally happy with what the language offers today. Instead, the connecting +theme is *productivity*—how quickly can I start writing real code? bring up a +team? prototype and iterate? debug my code? And so on. + +In other words, our primary challenge isn't making Rust "better" in the +abstract; it's making people *productive* with Rust. The need is most pronounced +in the early stages of Rust learning, where we risk losing a large pool of +interested people if we can't get them over the hump. Evidence from the survey +and elsewhere suggests that once people do get over the initial learning curve, +they tend to stick around. + +So how do we pull it off? + +### Core values + +Part of what makes Rust so exciting is that it attempts to eliminate some +seemingly fundamental tradeoffs. The central such tradeoff is between safety +and speed. Rust strives for + +- uncompromising reliability +- uncompromising performance + +and delivers on this goal largely thanks to its fundamental concept of +ownership. + +But there's a problem: at first glance, "productivity" and "learnability" may +seem at odds with Rust's core goals. It's common to hear the refrain that +"fighting with the borrow checker" is a rite of passage for Rustaceans. Or that +removing papercuts would mean glossing over safety holes or performance cliffs. + +To be sure, there are tradeoffs here. But as above, if there's one thing the +Rust community knows how to do, it's bending the curve around tradeoffs—memory +safety without garbage collection, concurrency without data races, and all the +rest. We have many examples in the language where we've managed to make a +feature pleasant to use, while also providing maximum performance and +safety—closures are a particularly good example, but there are +[others](https://internals.rust-lang.org/t/roadmap-2017-productivity-learning-curve-and-expressiveness/4097). + +And of course, beyond the core language, "productivity" also depends a lot on +tooling and the ecosystem. Cargo is one example where Rust's tooling provides a +huge productivity boost, and we've been working hard on other aspects of +tooling, like the +[compiler's error messages](https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-come.html), +that likewise have a big impact on productivity. There's so much more we can be +doing in this space. + +In short, **productivity should be a core value of Rust**. By the end of 2017, +let's try to earn the slogan: + +- Rust: fast, reliable, productive—pick three. + +# Detailed design +[design]: #detailed-design + +## Overall strategy + +In the abstract, reaching the kind of adoption we need means bringing +people along a series of distinct steps: + +- Public perception of Rust +- First contact +- Early play, toy projects +- Public projects +- Personal investment +- Professional investment + +We need to (1) provide "drivers", i.e. strong motivation to continue through the +stages and (2) avoid "blockers" that prevent people from progressing. + +At the moment, our most immediate adoption obstacles are mostly about blockers, +rather than a lack of drivers: there are people who see potential value in Rust, +but worry about issues like productivity, tooling, and maturity standing in the +way of use at scale The roadmap proposes a set of goals largely angled at +reducing these blockers. + +However, for Rust to make sense to use in a significant way in production, it +also needs to have a "complete story" for one or more domains of use. The goals +call out a specific domain where we are already seeing promising production use, +and where we have a relatively clear path toward a more complete story. + +Almost all of the goals focus squarely on "productivity" of one kind or another. + +## Goals + +Now to the meat of the roadmap: the goals. Each is phrased in terms of a +*qualitative vision*, trying to carve out what the *experience* of Rust should +be in one year's time. The details mention some possible avenues toward a +solution, but this shouldn't be taken as prescriptive. + +> These goals are partly informed from the [internals thread] about the +roadmap. That thread also posed a number of possible additional goals. Of +course, part of the work of the roadmap is to allocate our limited resources, +which fundamentally means not including some possible goals. Some of the most +promising suggestions that didn't make it into the roadmap proposal itself are +included at the end of the Goals section. + +### Rust should have a lower learning curve + +Rust offers a unique value proposition in part because it offers a unique +feature: its ownership model. Because the concept is not (yet!) a widespread one +in other languages, it is something most people have to learn from scratch +before hitting their stride with Rust. And that often comes on top of other +aspects of Rust that may be less familiar. A common refrain is "the first couple +of weeks are tough, but it's oh so worth it." How many people are bouncing off +of Rust before they get through those first couple of weeks? How many team leads +are reluctant to introduce Rust because of the training needed? (1 in 4 survey +respondents mentioned the learning curve.) + +Here are some strategies we might take to lower the learning curve: + +- **Improved docs**. While the existing Rust book has been successful, we've + learned a lot about teaching Rust, and there's a + [rewrite](http://words.steveklabnik.com/whats-new-with-the-rust-programming-language) + in the works. The effort is laser-focused on the key areas that trip people up + today (ownership, modules, strings, errors). + +- **Improved errors**. We've already made some + [big strides](https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-come.html) + here, particularly for ownership-related errors, but there's surely more room + for improvement. + +- **Improved language features**. There are a couple of ways that the language + design itself can be oriented toward learnability. First, we can introduce new + features with an explicit eye toward + [how they will be taught](https://github.com/rust-lang/rfcs/pull/1636). Second, + we can improve existing features to make them easier to understand and use -- + things like non-lexical lifetimes being a major example. There's already been + [some discussion on internals](https://internals.rust-lang.org/t/roadmap-2017-productivity-learning-curve-and-expressiveness/4097/) + +- **IDEs and other tooling**. IDEs provide a good opportunity for deeper + teaching. An IDE can visualize errors, for example *showing* you the lifetime + of a borrow. They can also provide deeper inspection of what's going on with + things like method dispatch, type inference, and so on. + +### Rust should have a pleasant edit-compile-debug cycle + +The edit-compile-debug cycle in Rust takes too long, and it's one of the +complaints we hear most often from production users. We've laid down a good +foundation with [MIR] (now turned on by default) and [incremental compilation] +(which recently hit alpha). But we need to continue pushing hard to actually +deliver the improvements. And to fully address the problem, **the improvement +needs to apply to large Rust projects, not just small or mid-sized benchmarks**. + +To get this done, we're also going to need further improvements to the +performance monitoring infrastructure, including more benchmarks. Note, though, +that the goal is stated *qualitatively*, and we need to be careful with what we +measure to ensure we don't lose sight of that goal. + +While the most obvious routes are direct improvements like incremental +compilation, since the focus here is primarily on development (including +debugging), another promising avenue is more usable debug builds. Production +users often say "debug binaries are too slow to run, but release binaries are +too slow to build". There may be a lot of room in the middle. + +Depending on how far we want to take IDE support (see below), pushing +incremental compilation up through the earliest stages of the compiler may also +be important. + +[MIR]: https://blog.rust-lang.org/2016/04/19/MIR.html +[incremental compilation]: https://blog.rust-lang.org/2016/09/08/incremental.html + +### Rust should provide a solid, but basic IDE experience + +For many people—even whole organizations—IDEs are an essential part of the +programming workflow. In the survey, 1 in 4 respondents mentioned requiring IDE +support before using Rust seriously. Tools like [Racer] and the [IntelliJ] Rust +plugin have made great progress this year, but [compiler integration] in its +infancy, which limits the kinds of tools that general IDE plugins can provide. + +The problem statement here says "solid, but basic" rather than "world-class" IDE +support to set realistic expectations for what we can get done this year. Of +course, the precise contours will need to be driven by implementation work, but +we can enumerate some basic constraints for such an IDE here: + +- It should be **reliable**: it shouldn't crash, destroy work, or give inaccurate + results in situations that demand precision (like refactorings). +- It should be **responsive**: the interface should never hang waiting on the + compiler or other computation. In places where waiting is required, the + interface should update as smoothly as possible, while providing + responsiveness throughout. +- It should provide **basic functionality**. At a minimum, that's: syntax + highlighting, basic code navigation (e.g. go-to-definition), code completion, + build support (with Cargo integration), error integration, and code + formatting. + +Note that while some of this functionality is available in existing IDE/plugin +efforts, a key part of this initiative is to (1) lay the foundation for plugins +based on compiler integration (2) pull together existing tools into a single +service that can integrate with multiple IDEs. + +[Racer]: https://github.com/phildawes/racer +[IntelliJ]: https://intellij-rust.github.io/ +[compiler integration]: https://internals.rust-lang.org/t/introducing-rust-language-server-source-release/4209/ + +### Rust should provide easy access to high quality crates + +Another major message from the survey and elsewhere is that Rust's ecosystem, +while growing, is still immature (1 in 9 survey respondents mentioned +this). Maturity is not something we can rush. But there are steps we can take +across the ecosystem to help improve the quality and discoverability of crates, +both of which will help increase the overall sense of maturity. + +Some avenues for quality improvement: + +- Provide stable, extensible test/bench frameworks. +- Provide more push-button CI setup, e.g. have `cargo new` set up Travis/Appveyor. +- Restart the [API guidelines](http://aturon.github.io/) project. +- Use badges on crates.io to signal various quality metrics. +- Perform API reviews on important crates. + +Some avenues for discoverability improvement: + +- Adding categories to crates.io, making it possible to browse lists like + "crates for parsing". +- More sophisticated ranking and/or curation. + +A number of ideas along these lines were discussed in the [Rust Platform thread]. + +[Rust Platform thread]: https://internals.rust-lang.org/t/proposal-the-rust-platform/3745 + +### Rust should be well-equipped for writing robust, high-scale servers + +The biggest area we've seen with interest in production Rust so far is the +server, particularly in cases where high-scale performance, control, and/or +reliability are paramount. At the moment, our ecosystem in this space is +nascent, and production users are having to build a lot from scratch. + +Of the specific domains we might target for having a more complete story, Rust +on the server is the place with the clearest direction and momentum. In a year's +time, it's within reach to drastically improve Rust's server ecosystem and the +overall experience of writing server code. The relevant pieces here include +foundations for async IO, language improvements for async code ergonomics, +shared infrastructure for writing services (including abstractions for +implementing protocols and middleware), and endless interfaces to existing +services/protocols. + +There are two reasons to focus on the robust, high-scale case. Most importantly, +it's the place where Rust has the clearest value proposition relative to other +languages, and hence the place where we're likeliest to achieve significant, +quality production usage (as discussed earlier in the RFC). More generally, the +overall server space is *huge*, so choosing a particular niche provides +essential focus for our efforts. + +### Rust should have 1.0-level crates for essential tasks + +Rust has taken a decidedly lean approach to its standard library, preferring for +much of the typical "batteries included" functionality to live externally in the +crates.io ecosystem. While there are a lot of benefits to that approach, it's +important that we do in fact provide the batteries somewhere: we need 1.0-level +functionality for essential tasks. To pick just one example, the `rand` crate +has suffered from a lack of vision and has effectively stalled before reaching +1.0 maturity, despite its central importance for a non-trivial part of the +ecosystem. + +There are two basic strategies we might take to close these gaps. + +The first is to identify a broad set of "essential tasks" by, for example, +finding the commonalities between large "batteries included" standard libraries, +and focus community efforts on bolstering crates in these areas. With sustained +and systematic effort, we can probably help push a number of these crates to 1.0 +maturity this year. + +A second strategy is to focus specifically on tasks that play to Rust's +strengths. For example, Rust's potential for [fearless concurrency] across a +range of paradigms is one of the most unique and exciting aspects of the +language. But we aren't fully delivering on this potential, due to the +immaturity of libraries in the space. The response to work in this space, like +the recent [futures library announcement], suggests that there is a lot of +pent-up demand and excitement, and that this kind of work can open a lot of +doors for Rust. So concurrency/asynchrony/parallelism is one segment of the +ecosystem that likely deserves particular focus (and feeds into the high-scale +server goal as well); there are likely others. + +[fearless concurrency]: http://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.html +[futures library announcement]: http://aturon.github.io/blog/2016/08/11/futures/ + +### Rust should integrate easily into large build systems + +When working with larger organizations interested in using Rust, one of the +first hurdles we tend to run into is fitting into an existing build +system. We've been exploring a number of different approaches, each of which +ends up using Cargo (and sometimes `rustc`) in different ways, with different +stories about how to incorporate crates from the broader crates.io ecosystem. +Part of the issue seems to be a perceived overlap between functionality in Cargo +(and its notion of compilation unit) and in ambient build systems, but we have +yet to truly get to the bottom of the issues—and it may be that the problem is +one of communication, rather than of some technical gap. + +By the end of 2017, this kind of integration should be *easy*: as a community, +we should have a strong understanding of best practices, and potentially build +tooling in support of those practices. And of course, we want to approach this +goal with Rust's values in mind, ensuring that first-class access to the +crates.io ecosystem is a cornerstone of our eventual story. + +### Rust should integrate easily with C++ code + +Rust's current support for interfacing with C is fairly strong, but wrapping a C +library still involves tedious work mirroring declarations and writing C shims +or other glue code. Moreover, many projects that are ripe for Rust integration +are currently using C++, and interfacing with those effectively requires +maintaining an alternative C wrapper for the C++ APIs. This is a problem both +for Rust code that wants to employ existing libraries and for those who want to +integrate Rust into existing C/C++ codebases. + +Why C++ rather than the myriad other languages we might focus on? Of any of the +possible language integrations, C++ came up most consistently across both the +general survey and the commercial outreach. And that's not too surprising: Rust +is well-positioned as a C++ replacement, so it's natural for people with +existing C++ codebases to be looking at Rust but needing a good integration +story for moving incrementally. In addition, many of the features needed for +high quality C/C++ binding support are prerequisites for high quality binding +support in other languages, since runtime internals will generally be accessible +through a C or C++ interface. So in general, C++ seems like the wisest place to +focus on the language integration front for the moment. + +**The goal should be that using a C++ library in Rust is not much harder than +using it in C++**. In other words, it should be possible to directly include C++ +headers (e.g., include! {myproject.hpp}) and have the extern declarations, glue +code, and so forth get generated automatically. This means (eventually) full +support for interfacing with C++ code that uses features like templates, +overloading, classes and virtual calls, and so forth. + +A number of the needed ingredients already exist. With the addition of unions, +Rust itself supports much of the representation infrastructure needed to +describe C and C++ data structures, though gaps (such as bitflags) +remain. Current work on the bindgen tool is extending its support for handling +the layout of more complex C++ data structures (such as classes). Moreover, a +large number of projects have paved the way in terms of identifying best +practices and gaps. What is needed now is to formulate a plan for enabling +better interop, including intermediate milestones that allow early adopters to +make immediate use of each feature as it is added. + +### Rust's community should provide mentoring at all levels + +The Rust community is awesome, in large part because of how welcoming it is. But +we could do a lot more to help grow people into roles in the project, including +pulling together important work items at all level of expertise to direct people +to, providing mentoring, and having a clearer on-ramp to the various official +Rust teams. Outreach and mentoring is also one of the best avenues for +increasing diversity in the project, which, as the survey demonstrates, has a +lot of room for improvement. + +While there's work here for *all* the teams, the community team in particular +will continue to focus on early-stage outreach, while other teams will focus on +leadership onboarding. + +## Areas of support + +The goals above represent the steps we think are most essential to Rust's +success in 2017, and where we want to commit leadership resources from the +various Rust teams. + +Beyond those goals, however, there are a number of areas with strong potential +for Rust that are in a more exploratory phase, with subcommunities already +exploring the frontiers. Some of these areas are important enough that we should +provide *support*, in two forms: + +- Highlighting the areas and trying to coordinate work within their + communities. For example, in the published form of the roadmap, these "areas + of interest" should be included, with links to ongoing work and points of + coordination. + +- Giving more consideration to feature requests that strongly impact these + areas. For example, numeric computation is a potential domain of interest, and + its one where "const generics" would be a great help—where the lack of that + feature is effectively a blocker. That's not to say we would necessarily + tackle such features; the primary goals have to come first. + +Here are two major areas we should consider supporting this way: + +- **Integration with higher-level languages like Javascript, Ruby, Python, Java + and C#**. Lack of integration in general can be a blocker for Rust adoption in + a given context, and so work on any of these fronts has the potential to + increase adoption. They came up many times in the internals thread. (However, + as explained above, C++ integration is the clear winner and hence primary + focus.) There's existing work on integration for these languages and many + others; we should raise awareness of this work, try to improve coordination + across the integration projects, and consider features that would aid + integration. + +- **Embedded devices**. Rust is a very natural fit for programming + resource-constrained devices, and there are some + [nascent efforts](https://github.com/rust-embedded/) to better organize work + in this area, as well as a + [thread](https://internals.rust-lang.org/t/roadmap-2017-needs-of-no-std-embedded-developers/4096) + on the current significant problems in the domain. Embedded devices likewise + came up repeatedly in the internals thread. It's also a potentially huge + market. At the moment, though, it's far from clear what it will take to + achieve significant production use in the embedded space. It would behoove us + to try to get a clearer picture of this space in 2017. + +## Non-goals + +Finally, it's important that the roadmap "have teeth": we should be focusing on +the goals, and avoid getting distracted by other improvements that, whatever +their appeal, could sap bandwidth and our ability to ship what we believe is +most important in 2017. + +To that end, it's worth making some explicit *non*-goals, to set expectations +and short-circuit discussions: + +- No major new language features, except in service of one of the goals. Cases + that have a very strong impact on the "areas of support" may be considered + case-by-case. + +- No major expansions to `std`, except in service of one of the goals. Cases + that have a very strong impact on the "areas of support" may be considered + case-by-case. + +- No Rust 2.0. In particular, no changes to the language or `std` that could be + perceived as "major breaking changes". We need to be doing everything we can + to foster maturity in Rust, both in reality and in perception, and ongoing + stability is an important part of that story. + +## Additional goals from the internals thread + +There were several strong contenders for additional goals from the +internals thread that we might consider: + +- A goal explicitly for + [systematic expansion of commercial use](https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/3958/68); + this proposal takes that as a kind of overarching idea for all of the goals. + +- A goal for Rust infrastructure, which came + [up](https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/3958/9) + [several](https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/3958/68) + [times](https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/3958/5). + While this goal seems quite worthwhile in terms of paying dividends across the + project, in terms of our current subteam makeup it's hard to see how to + allocate resources toward this goal without dropping other important goals. We + might consider forming a dedicated infrastructure team, or somehow organizing + and growing our bandwidth in this area. + +- A goal for progress in areas like + [scientific computing](https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/3958/52), + [HPC](https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/3958/48). + +After an exhaustive look at the thread, the remaining proposals are in one way +or another covered somewhere in the discussion above. + +# Drawbacks and alternatives +[drawbacks]: #drawbacks + +It's a bit difficult to enumerate the full design space here, given how much +there is we could potentially be doing. + +At a high level, though, the biggest alternatives (and potential for drawbacks) +are probably at the strategic level. This roadmap proposal takes the approach of +(1) focusing on reducing clear blockers to Rust adoption, particularly connected +with productivity and (2) choosing one particular "driver" for adoption to +invest in, namely high-scale servers. The balance between blocker/driver focus +could be shifted—it might be the case that by providing more incentive to use +Rust in a particular domain, people are willing to overlook some of its +shortcomings. + +Another possible blind spot is the conservative take on language expansion, +particularly when it comes to productivity. For example, we could put much +greater emphasis on "metaprogramming", and try to complete Plugins 2.0 +in 2017. That kind of investment *could* pay dividends, since libraries can do +amazing things with plugins that could draw people to Rust. But, as above, the +overall strategy of reducing blockers assumes that what's most needed isn't more +flashy examples of Rust's power, but rather more bread-and-butter work on +reducing friction, improving tooling, and just making Rust easier to use across +the board. + +The roadmap is pretty informed by the survey, systematic outreach, numerous +direct conversations, and general strategic thinking. But there could certainly +be blind spots and biases. It's worth double-checking our inputs. + +# Unresolved questions +[unresolved]: #unresolved-questions + +The main unresolved question is how to break the given goals into more +deliverable pieces of work, but that's a process that will happen after the +overall roadmap is approved. + +Are there other "areas of support" we should consider? Should any of these areas +be elevated to a top-level goal (which would likely involve cutting back on some +other goal)? + +Should we consider some loose way of organizing "special interest groups" to +focus on some of the priorities not part of the official goal set, but where +greater coordination would be helpful? This was suggested +[multiple](https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/3958/70) +[times](https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/3958/135). + +Finally, there were several strong contenders for additional goals from the +internals thread that we might consider, which are listed at the end of the +goals section. From e59d253ac87318b13c8f751aeb1ac875ed4c8ef2 Mon Sep 17 00:00:00 2001 From: Aaron Turon Date: Fri, 21 Oct 2016 16:08:15 -0700 Subject: [PATCH 02/10] Clarify Additional Goals section --- text/0000-roadmap-2017.md | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/text/0000-roadmap-2017.md b/text/0000-roadmap-2017.md index 476934b1377..88f9f81ef99 100644 --- a/text/0000-roadmap-2017.md +++ b/text/0000-roadmap-2017.md @@ -6,11 +6,7 @@ # Summary [summary]: #summary -This RFC proposes the *2017 Rust Roadmap*, in accordance with -[RFC 1728](https://github.com/rust-lang/rfcs/pull/1728). The goal of the roadmap -is to lay out a vision for where the Rust project should be in a year's -time. **This year's focus is improving Rust's *productivity*, while retaining its -emphasis on fast, reliable code**. At a high level, by the end of 2017: +This RFC proposes the *2017 Rust Roadmap*, in accordance with [RFC 1728](https://github.com/rust-lang/rfcs/pull/1728). The goal of the roadmap is to lay out a vision for where the Rust project should be in a year's time. **This year's focus is improving Rust's *productivity*, while retaining its emphasis on fast, reliable code**. At a high level, by the end of 2017: * Rust should have a lower learning curve * Rust should have a pleasant edit-compile-debug cycle @@ -22,9 +18,7 @@ emphasis on fast, reliable code**. At a high level, by the end of 2017: * Rust should integrate easily with C++ code * Rust's community should provide mentoring at all levels -The proposal is based on the [2016 survey], systematic outreach, direct -conversations with individual Rust users, and an extensive -[internals thread]. Thanks to everyone who helped with this effort! +The proposal is based on the [2016 survey], systematic outreach, direct conversations with individual Rust users, and an extensive [internals thread]. Thanks to everyone who helped with this effort! [2016 survey]: https://blog.rust-lang.org/2016/06/30/State-of-Rust-Survey-2016.html [internals thread]: https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/ @@ -505,10 +499,11 @@ and short-circuit discussions: to foster maturity in Rust, both in reality and in perception, and ongoing stability is an important part of that story. -## Additional goals from the internals thread +## Other ideas from the internals thread -There were several strong contenders for additional goals from the -internals thread that we might consider: +There were several strong contenders for additional goals from the internals +thread that we might consider. To be clear, these are not currently part of the +proposed goals, but we may want to consider elevating them: - A goal explicitly for [systematic expansion of commercial use](https://internals.rust-lang.org/t/setting-our-vision-for-the-2017-cycle/3958/68); From 31c4f92534651d24498d914b6d95173454becb5a Mon Sep 17 00:00:00 2001 From: Aaron Turon Date: Fri, 21 Oct 2016 16:22:23 -0700 Subject: [PATCH 03/10] Missing period --- text/0000-roadmap-2017.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-roadmap-2017.md b/text/0000-roadmap-2017.md index 88f9f81ef99..1179ddb40a6 100644 --- a/text/0000-roadmap-2017.md +++ b/text/0000-roadmap-2017.md @@ -157,7 +157,7 @@ stages and (2) avoid "blockers" that prevent people from progressing. At the moment, our most immediate adoption obstacles are mostly about blockers, rather than a lack of drivers: there are people who see potential value in Rust, but worry about issues like productivity, tooling, and maturity standing in the -way of use at scale The roadmap proposes a set of goals largely angled at +way of use at scale. The roadmap proposes a set of goals largely angled at reducing these blockers. However, for Rust to make sense to use in a significant way in production, it From 097fccdea6a148d0ffabcb2916f5ae1095a96d83 Mon Sep 17 00:00:00 2001 From: Aaron Turon Date: Fri, 21 Oct 2016 17:06:43 -0700 Subject: [PATCH 04/10] JavaScript --- text/0000-roadmap-2017.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-roadmap-2017.md b/text/0000-roadmap-2017.md index 1179ddb40a6..b4caefcb640 100644 --- a/text/0000-roadmap-2017.md +++ b/text/0000-roadmap-2017.md @@ -455,7 +455,7 @@ provide *support*, in two forms: Here are two major areas we should consider supporting this way: -- **Integration with higher-level languages like Javascript, Ruby, Python, Java +- **Integration with higher-level languages like JavaScript, Ruby, Python, Java and C#**. Lack of integration in general can be a blocker for Rust adoption in a given context, and so work on any of these fronts has the potential to increase adoption. They came up many times in the internals thread. (However, From a2cb43616195ff05cf636bc4ad664c6cd7bf63c9 Mon Sep 17 00:00:00 2001 From: Aaron Turon Date: Fri, 21 Oct 2016 17:07:49 -0700 Subject: [PATCH 05/10] clarity supporting areas --- text/0000-roadmap-2017.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-roadmap-2017.md b/text/0000-roadmap-2017.md index b4caefcb640..64acf1f1197 100644 --- a/text/0000-roadmap-2017.md +++ b/text/0000-roadmap-2017.md @@ -453,7 +453,7 @@ provide *support*, in two forms: feature is effectively a blocker. That's not to say we would necessarily tackle such features; the primary goals have to come first. -Here are two major areas we should consider supporting this way: +Here are two major areas we should support this way: - **Integration with higher-level languages like JavaScript, Ruby, Python, Java and C#**. Lack of integration in general can be a blocker for Rust adoption in From c403028cc4fbcbdd936d33571f757e676a97fad3 Mon Sep 17 00:00:00 2001 From: Aaron Turon Date: Fri, 21 Oct 2016 19:27:28 -0700 Subject: [PATCH 06/10] Clarify production framing --- text/0000-roadmap-2017.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/text/0000-roadmap-2017.md b/text/0000-roadmap-2017.md index 64acf1f1197..e31ba3df328 100644 --- a/text/0000-roadmap-2017.md +++ b/text/0000-roadmap-2017.md @@ -29,7 +29,7 @@ The proposal is based on the [2016 survey], systematic outreach, direct conversa There's no end of possible improvements to Rust—so what do we use to guide our thinking? -The core team has tended to view things not in terms of particular features or +The core team has tended to view our strategy not in terms of particular features or aesthetic goals, but instead in terms of **making Rust successful while staying true to its core values**. This basic sentiment underlies much of the proposed roadmap, so let's unpack it a bit. @@ -66,6 +66,12 @@ valuable new way of programming, we should be seeing that benefit in "the real world", in production uses that are significant enough to help sustain Rust's development. +That's not to say we should expect to see this usage *immediately*; there's a +long pipeline for technology adoption, so the effects of our work can take a +while to appear. The framing here is about our long-term aims. We should be +making investments in Rust today that will position it well for this kind of +success in the future. + ### The obstacles to success At this point, we have a fair amount of data about how Rust is reaching its From b7ac67c87c85ee24040df8271b8142245f8162e1 Mon Sep 17 00:00:00 2001 From: Aaron Turon Date: Fri, 21 Oct 2016 19:42:13 -0700 Subject: [PATCH 07/10] Fix awkward phrasing --- text/0000-roadmap-2017.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0000-roadmap-2017.md b/text/0000-roadmap-2017.md index e31ba3df328..2539ecc1a4f 100644 --- a/text/0000-roadmap-2017.md +++ b/text/0000-roadmap-2017.md @@ -195,9 +195,9 @@ in other languages, it is something most people have to learn from scratch before hitting their stride with Rust. And that often comes on top of other aspects of Rust that may be less familiar. A common refrain is "the first couple of weeks are tough, but it's oh so worth it." How many people are bouncing off -of Rust before they get through those first couple of weeks? How many team leads -are reluctant to introduce Rust because of the training needed? (1 in 4 survey -respondents mentioned the learning curve.) +of Rust in those first couple of weeks? How many team leads are reluctant to +introduce Rust because of the training needed? (1 in 4 survey respondents +mentioned the learning curve.) Here are some strategies we might take to lower the learning curve: From 1d10205dc908c638410ef65bfa3c54d9990a0f7c Mon Sep 17 00:00:00 2001 From: Aaron Turon Date: Fri, 21 Oct 2016 19:56:16 -0700 Subject: [PATCH 08/10] Clarify extra goals from internals --- text/0000-roadmap-2017.md | 68 +++++++++++++++++++++------------------ 1 file changed, 36 insertions(+), 32 deletions(-) diff --git a/text/0000-roadmap-2017.md b/text/0000-roadmap-2017.md index 2539ecc1a4f..5fbe4e19e07 100644 --- a/text/0000-roadmap-2017.md +++ b/text/0000-roadmap-2017.md @@ -180,12 +180,12 @@ Now to the meat of the roadmap: the goals. Each is phrased in terms of a be in one year's time. The details mention some possible avenues toward a solution, but this shouldn't be taken as prescriptive. -> These goals are partly informed from the [internals thread] about the +These goals are partly informed from the [internals thread] about the roadmap. That thread also posed a number of possible additional goals. Of course, part of the work of the roadmap is to allocate our limited resources, which fundamentally means not including some possible goals. Some of the most promising suggestions that didn't make it into the roadmap proposal itself are -included at the end of the Goals section. +included in the Alternatives section. ### Rust should have a lower learning curve @@ -505,9 +505,42 @@ and short-circuit discussions: to foster maturity in Rust, both in reality and in perception, and ongoing stability is an important part of that story. +# Drawbacks and alternatives +[drawbacks]: #drawbacks + +It's a bit difficult to enumerate the full design space here, given how much +there is we could potentially be doing. Instead, we'll take a look at some +alternative high-level strategies, and some additional goals from the internals +thread. + +## Overall strategy + +At a high level, though, the biggest alternatives (and potential for drawbacks) +are probably at the strategic level. This roadmap proposal takes the approach of +(1) focusing on reducing clear blockers to Rust adoption, particularly connected +with productivity and (2) choosing one particular "driver" for adoption to +invest in, namely high-scale servers. The balance between blocker/driver focus +could be shifted—it might be the case that by providing more incentive to use +Rust in a particular domain, people are willing to overlook some of its +shortcomings. + +Another possible blind spot is the conservative take on language expansion, +particularly when it comes to productivity. For example, we could put much +greater emphasis on "metaprogramming", and try to complete Plugins 2.0 +in 2017. That kind of investment *could* pay dividends, since libraries can do +amazing things with plugins that could draw people to Rust. But, as above, the +overall strategy of reducing blockers assumes that what's most needed isn't more +flashy examples of Rust's power, but rather more bread-and-butter work on +reducing friction, improving tooling, and just making Rust easier to use across +the board. + +The roadmap is informed by the survey, systematic outreach, numerous direct +conversations, and general strategic thinking. But there could certainly be +blind spots and biases. It's worth double-checking our inputs. + ## Other ideas from the internals thread -There were several strong contenders for additional goals from the internals +Finally, there were several strong contenders for additional goals from the internals thread that we might consider. To be clear, these are not currently part of the proposed goals, but we may want to consider elevating them: @@ -532,35 +565,6 @@ proposed goals, but we may want to consider elevating them: After an exhaustive look at the thread, the remaining proposals are in one way or another covered somewhere in the discussion above. -# Drawbacks and alternatives -[drawbacks]: #drawbacks - -It's a bit difficult to enumerate the full design space here, given how much -there is we could potentially be doing. - -At a high level, though, the biggest alternatives (and potential for drawbacks) -are probably at the strategic level. This roadmap proposal takes the approach of -(1) focusing on reducing clear blockers to Rust adoption, particularly connected -with productivity and (2) choosing one particular "driver" for adoption to -invest in, namely high-scale servers. The balance between blocker/driver focus -could be shifted—it might be the case that by providing more incentive to use -Rust in a particular domain, people are willing to overlook some of its -shortcomings. - -Another possible blind spot is the conservative take on language expansion, -particularly when it comes to productivity. For example, we could put much -greater emphasis on "metaprogramming", and try to complete Plugins 2.0 -in 2017. That kind of investment *could* pay dividends, since libraries can do -amazing things with plugins that could draw people to Rust. But, as above, the -overall strategy of reducing blockers assumes that what's most needed isn't more -flashy examples of Rust's power, but rather more bread-and-butter work on -reducing friction, improving tooling, and just making Rust easier to use across -the board. - -The roadmap is pretty informed by the survey, systematic outreach, numerous -direct conversations, and general strategic thinking. But there could certainly -be blind spots and biases. It's worth double-checking our inputs. - # Unresolved questions [unresolved]: #unresolved-questions From 4f40ba07f2a0730c188cb5db6b0b9c5887ae1801 Mon Sep 17 00:00:00 2001 From: Aaron Turon Date: Tue, 13 Dec 2016 17:20:06 -0800 Subject: [PATCH 09/10] Remove C++ goal, and refactor to 'areas of exploration' --- text/0000-roadmap-2017.md | 155 +++++++++++++++++++------------------- 1 file changed, 77 insertions(+), 78 deletions(-) diff --git a/text/0000-roadmap-2017.md b/text/0000-roadmap-2017.md index 5fbe4e19e07..f12aa6759f6 100644 --- a/text/0000-roadmap-2017.md +++ b/text/0000-roadmap-2017.md @@ -15,9 +15,14 @@ This RFC proposes the *2017 Rust Roadmap*, in accordance with [RFC 1728](https:/ * Rust should be well-equipped for writing robust, high-scale servers * Rust should have 1.0-level crates for essential tasks * Rust should integrate easily into large build systems -* Rust should integrate easily with C++ code * Rust's community should provide mentoring at all levels +In addition, we should make significant strides in *exploring* two areas where +we're not quite ready to set out specific goals: + +* Integration with other languages, running the gamut from C to JavaScript +* Usage in resource-constrained environments + The proposal is based on the [2016 survey], systematic outreach, direct conversations with individual Rust users, and an extensive [internals thread]. Thanks to everyone who helped with this effort! [2016 survey]: https://blog.rust-lang.org/2016/06/30/State-of-Rust-Survey-2016.html @@ -385,44 +390,6 @@ tooling in support of those practices. And of course, we want to approach this goal with Rust's values in mind, ensuring that first-class access to the crates.io ecosystem is a cornerstone of our eventual story. -### Rust should integrate easily with C++ code - -Rust's current support for interfacing with C is fairly strong, but wrapping a C -library still involves tedious work mirroring declarations and writing C shims -or other glue code. Moreover, many projects that are ripe for Rust integration -are currently using C++, and interfacing with those effectively requires -maintaining an alternative C wrapper for the C++ APIs. This is a problem both -for Rust code that wants to employ existing libraries and for those who want to -integrate Rust into existing C/C++ codebases. - -Why C++ rather than the myriad other languages we might focus on? Of any of the -possible language integrations, C++ came up most consistently across both the -general survey and the commercial outreach. And that's not too surprising: Rust -is well-positioned as a C++ replacement, so it's natural for people with -existing C++ codebases to be looking at Rust but needing a good integration -story for moving incrementally. In addition, many of the features needed for -high quality C/C++ binding support are prerequisites for high quality binding -support in other languages, since runtime internals will generally be accessible -through a C or C++ interface. So in general, C++ seems like the wisest place to -focus on the language integration front for the moment. - -**The goal should be that using a C++ library in Rust is not much harder than -using it in C++**. In other words, it should be possible to directly include C++ -headers (e.g., include! {myproject.hpp}) and have the extern declarations, glue -code, and so forth get generated automatically. This means (eventually) full -support for interfacing with C++ code that uses features like templates, -overloading, classes and virtual calls, and so forth. - -A number of the needed ingredients already exist. With the addition of unions, -Rust itself supports much of the representation infrastructure needed to -describe C and C++ data structures, though gaps (such as bitflags) -remain. Current work on the bindgen tool is extending its support for handling -the layout of more complex C++ data structures (such as classes). Moreover, a -large number of projects have paved the way in terms of identifying best -practices and gaps. What is needed now is to formulate a plan for enabling -better interop, including intermediate milestones that allow early adopters to -make immediate use of each feature as it is added. - ### Rust's community should provide mentoring at all levels The Rust community is awesome, in large part because of how welcoming it is. But @@ -437,50 +404,82 @@ While there's work here for *all* the teams, the community team in particular will continue to focus on early-stage outreach, while other teams will focus on leadership onboarding. -## Areas of support +## Areas of exploration The goals above represent the steps we think are most essential to Rust's -success in 2017, and where we want to commit leadership resources from the -various Rust teams. +success in 2017, and where we are in a position to lay out a fairly concrete vision. Beyond those goals, however, there are a number of areas with strong potential for Rust that are in a more exploratory phase, with subcommunities already -exploring the frontiers. Some of these areas are important enough that we should -provide *support*, in two forms: - -- Highlighting the areas and trying to coordinate work within their - communities. For example, in the published form of the roadmap, these "areas - of interest" should be included, with links to ongoing work and points of - coordination. - -- Giving more consideration to feature requests that strongly impact these - areas. For example, numeric computation is a potential domain of interest, and - its one where "const generics" would be a great help—where the lack of that - feature is effectively a blocker. That's not to say we would necessarily - tackle such features; the primary goals have to come first. - -Here are two major areas we should support this way: - -- **Integration with higher-level languages like JavaScript, Ruby, Python, Java - and C#**. Lack of integration in general can be a blocker for Rust adoption in - a given context, and so work on any of these fronts has the potential to - increase adoption. They came up many times in the internals thread. (However, - as explained above, C++ integration is the clear winner and hence primary - focus.) There's existing work on integration for these languages and many - others; we should raise awareness of this work, try to improve coordination - across the integration projects, and consider features that would aid - integration. - -- **Embedded devices**. Rust is a very natural fit for programming - resource-constrained devices, and there are some - [nascent efforts](https://github.com/rust-embedded/) to better organize work - in this area, as well as a - [thread](https://internals.rust-lang.org/t/roadmap-2017-needs-of-no-std-embedded-developers/4096) - on the current significant problems in the domain. Embedded devices likewise - came up repeatedly in the internals thread. It's also a potentially huge - market. At the moment, though, it's far from clear what it will take to - achieve significant production use in the embedded space. It would behoove us - to try to get a clearer picture of this space in 2017. +exploring the frontiers. Some of these areas are important enough that we want +to call them out explicitly, and will expect ongoing progress over the course of +the year. In particular, the subteams are expected to proactively help organize +and/or carry out explorations in these areas, and by the end of the year we +expect to have greater clarity around Rust's story for these areas, putting us +in a position to give more concrete goals in subsequent roadmaps. + +Here are the two proposed Areas of Exploration. + +### Integration with other languages + +Other languages here includes "low-level" cases like C/C++, and "high-level" +cases like JavaScript, Ruby, Python, Java and C#. Rust adoption often depends on +being able to start using it *incrementally*, and language integration is often +a key to doing so -- an intuition substantiated by data from the survey and +commercial outreach. + +Rust's core support for interfacing with C is fairly strong, but wrapping a C +library still involves tedious work mirroring declarations and writing C shims +or other glue code. Moreover, many projects that are ripe for Rust integration +are currently using C++, and interfacing with those effectively requires +maintaining an alternative C wrapper for the C++ APIs. This is a problem both +for Rust code that wants to employ existing libraries and for those who want to +integrate Rust into existing C/C++ codebases. + +For interfacing with "high-level" languages, there is the additional barrier of +working with a runtime system, which often involves integration with a garbage +collector and object system. There are ongoing projects on these fronts, but +it's early days and there are still a lot of open questions. + +Some potential avenues of exploration include: + +- Continuing work on bindgen, with focus on seamless C and eventually C++ + support. This may involve some FFI-related language extensions (like richer + `repr`). +- Other routes for C/C++ integration. +- Continued expansion of existing projects like + [Helix](https://github.com/rustbridge/helix) and + [Neon](https://github.com/dherman/neon), which may require some language + enhancements. +- Continued work on [GC integration hooks](http://manishearth.github.io/blog/2016/08/18/gc-support-in-rust-api-design/) +- Investigation of object system integrations, including DOM and + [GObject](https://internals.rust-lang.org/t/rust-and-gnome-meeting-notes/4339). + +### Usage in resource-constrained environments + +Rust is a natural fit for programming resource-constrained devices, and +there are some [ongoing efforts](https://github.com/rust-embedded/) to better +organize work in this area, as well as a +[thread](https://internals.rust-lang.org/t/roadmap-2017-needs-of-no-std-embedded-developers/4096) +on the current significant problems in the domain. Embedded devices likewise +came up repeatedly in the internals thread. It's also a potentially huge +market. At the moment, though, it's far from clear what it will take to achieve +significant production use in the embedded space. It would behoove us to try to +get a clearer picture of this space in 2017. + +Some potential avenues of exploration include: + +- Continuing work on [rustup](https://github.com/rust-lang-nursery/rustup.rs/), + [xargo](https://github.com/japaric/xargo) and similar tools for easing + embedded development. +- Land ["std-aware Cargo"](https://github.com/rust-lang/rfcs/pull/1133), making + it easier to experiment with ports of the standard library to new platforms. +- Work on + [scenarios](https://internals.rust-lang.org/t/fleshing-out-libstd-scenarios/4206) + or other techniques for cutting down `std` in various ways, depending on + platform capabilities. +- Develop a story for failable allocation in `std` (i.e., without aborting when + out of memory). ## Non-goals From c59aa270f02f37f908e55b664c82321efc7fbfbe Mon Sep 17 00:00:00 2001 From: Aaron Turon Date: Thu, 5 Jan 2017 09:37:29 -0800 Subject: [PATCH 10/10] Add cookbooks, examples and patterns --- text/0000-roadmap-2017.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/text/0000-roadmap-2017.md b/text/0000-roadmap-2017.md index f12aa6759f6..e39a835404e 100644 --- a/text/0000-roadmap-2017.md +++ b/text/0000-roadmap-2017.md @@ -212,6 +212,14 @@ Here are some strategies we might take to lower the learning curve: in the works. The effort is laser-focused on the key areas that trip people up today (ownership, modules, strings, errors). +- **Gathering cookbooks, examples, and patterns**. One way to quickly get + productive in a language is to work from a large set of examples and + known-good patterns that can guide your early work. As a community, we could + push crates to include more substantial example code snippets, and organize + efforts around design patterns and cookbooks. (See + [the commentary on the RFC thread](https://github.com/rust-lang/rfcs/pull/1774#issuecomment-269359228) + for much more detail.) + - **Improved errors**. We've already made some [big strides](https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-come.html) here, particularly for ownership-related errors, but there's surely more room