CRAN Cookbook Survey Results

Survey Details

• The survey was hosted on Google Forms

• 126 responses over 23 Days: Start: June 27, 2024 ▶️ End: July 19, 2024

• Data collection method | The survey link was shared broadly on:

  • Social media (LinkedIn, X/Twitter, Mastodon, Discord)
  • Relevant Slack Communities (i.e. pharmaverse, Data Science Learning Community, R-Ladies, R Contributors, useR! Organizers, rOpenSci, etc..)
  • Mailing Lists

• We have retained access to the raw data to contact those respondents that indicated they would like to contribute in some way

• For survey questions that were free-form text responses: we leveraged ChatGPT to summarize the responses to the Top-n takeaways

Summarized Survey Results (Text responses only)

Why did you decide to have your package(s) hosted on CRAN?

1. Ease of Access for Users: CRAN is the default repository for R, making it the easiest and most familiar place for users to find and install packages. This is especially important for novice users or those in teaching environments.
2. Visibility and Reach: Hosting on CRAN provides high visibility within the R community, ensuring that packages are easily discoverable and accessible to a wide audience.
3. Quality Assurance: CRAN's rigorous checks and standards for package submission ensure a level of quality and reliability, which can enhance the reputation of the package.
4. Integration and Compatibility: CRAN packages are easily integrated into the R ecosystem, including seamless installation via install.packages(), and often come with binaries for various operating systems.
5. Community Contribution: Many developers see hosting on CRAN as a way to contribute back to the R community, providing tools and functions that can benefit others.
6. Historical and Institutional Requirements: Some developers hosted their packages on CRAN due to historical reasons (pre-dating other repositories) or because it was a requirement from their institution or funding body.
7. Perception of Stability: CRAN is seen as the "gold standard" for R packages, and hosting there can be perceived as a mark of a package's maturity and stability.

What things do you like about your experiences with CRAN?

1. Rigorous Quality Checks: Users appreciate that CRAN enforces thorough checks on packages, ensuring robustness and reducing the likelihood of errors or bugs. This rigorous process enhances the reliability of the packages available on CRAN.
2. Ease of Use: CRAN is described as straightforward and simple to use, making it easy for developers to submit packages and for users to install them. The process is generally automated and efficient.
3. Responsiveness and Support: The CRAN team is known for being prompt, helpful, and clear in their communication. Users value the constructive feedback they receive, which helps improve the quality of their packages.
4. Reliability and Interoperability: Users trust that packages from CRAN will work well together and function correctly when installed. This reliability is a significant factor in why CRAN is favored over other package repositories.
5. Consistency and Standardization: CRAN's standardized rules and guidelines help maintain a consistent level of quality across packages. This consistency ensures that users and developers know what to expect when interacting with CRAN.

What things could be improved?

Communication and Transparency

1. Clearer Feedback: Many users feel the feedback from CRAN volunteers is often terse, abrupt, and lacking in actionable detail. More constructive and detailed responses could help developers understand and resolve issues more efficiently.
2. Consistency: Ensuring that the communication, especially regarding package rejection or removal, is consistent and provides clear reasons and guidelines for fixes.

Reproducibility and Testing Environment

1. Aligned Testing Platforms: Users face issues when packages pass tests on platforms like winbuilder but fail on CRAN due to differences in configuration or versions. Synchronizing these environments would reduce frustrations and streamline the submission process.
2. Pre-submission Testing: Providing tools or services that mimic the exact CRAN testing environments would help developers identify and fix issues before submission. This could include detailed instructions for setting up local environments that match CRAN’s configurations.

Flexibility and Policies

1. Grace Periods: Extending the grace periods for fixing errors, especially for issues arising from changes in R-devel or other dependencies, would be beneficial. The current 14-day notice period can be too short, particularly for complex or unexpected problems.
2. Package Size and Example Limits: Offering more flexibility in package size and example time limits could alleviate the need to split packages unnecessarily or remove valuable examples.

Automation and Tooling

1. Automated Handling of Exceptional Cases: Developing automated tools to manage exceptional cases, such as archived packages, and providing clear protocols for handling these scenarios would improve the process.
2. Soft-Dependency Policies: Making policies around soft dependencies (Suggests) more open and transparent would help maintainers manage their packages more effectively.

Volunteer and Support Model

1. Volunteer Workload and Attitude: Addressing the overwork and varying attitudes of volunteers is crucial. Some volunteers are seen as kind and helpful, while others are perceived as rude and abrasive. Streamlining the workload and possibly increasing the number of volunteers could help improve interactions.
2. Support and Documentation: Enhancing online support and making documentation more navigable and tutorial-style rather than reference-style would aid developers, especially those less familiar with R or CRAN policies.

Which parts of the CRAN processes need better documentation?

1. Reproducible Testing Environments: Detailed instructions on how to set up precise, reproducible environments for testing packages are needed. This includes both containerized solutions and step-by-step guides for setting up these environments manually on different operating systems.
2. Handling Notices and Responses: Clear guidelines on how to respond to CRAN maintainers' notices and how to seek clarification on notice emails are required. This includes examples of common issues and appropriate responses to avoid misunderstandings and expedite the submission process.
3. Comprehensive Coverage of CRAN Checks: Documentation should fully explain all CRAN incoming checks, including those not covered by standard R CMD check (e.g., URL resolution checks, specific malformed DESCRIPTION checks). It should also detail the configuration of the check machines, environmental variables at each step, and provide insights into the manual checks performed by volunteers.
4. Template for Package Setup: A template or a more detailed guide for setting up packages would be beneficial. This includes best practices for structuring the package, writing documentation, and including necessary files (e.g., tests, examples, README).
5. Handling Packages with Online Resources: Specific guidance on how to manage packages that interface with online resources (e.g., APIs) is essential. This includes best practices for writing tests and examples that rely on these resources, handling authentication securely, and managing dependencies on external services.