Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate blog to Docusaurus #2054

Merged
merged 34 commits into from
Jan 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
d07c524
Initial commit
jaimergp Dec 11, 2022
89c5540
Add linters and docs about the life cycle (#1)
jaimergp Jan 27, 2023
15fe8e9
add blog/2019-12-06-cfep09.md
jaimergp Nov 4, 2023
1da1414
add blog/2020-02-04-GSoC.md
jaimergp Nov 4, 2023
ec98499
add blog/2020-03-05-grayskull.md
jaimergp Nov 4, 2023
c49e927
add blog/2020-03-10-pypy.md
jaimergp Nov 4, 2023
b24c975
add blog/2020-07-02-op-risk.md
jaimergp Nov 4, 2023
535f206
add blog/2020-07-06-scipy-bof.md
jaimergp Nov 4, 2023
38216a5
add blog/2020-07-11-R-4.md
jaimergp Nov 4, 2023
f0163cf
add blog/2020-10-02-versions.md
jaimergp Nov 4, 2023
6e3664b
add blog/2020-10-29-macos-arm64.md
jaimergp Nov 4, 2023
7398dee
add blog/2020-11-20-anaconda-tos.md
jaimergp Nov 4, 2023
3e549e5
add blog/2020-12-26-year-in-review.md
jaimergp Nov 4, 2023
511dcd0
add blog/2021-02-02-Outreachy.md
jaimergp Nov 4, 2023
3f9aec4
add blog/2021-06-16-graykull-step-by-step.md
jaimergp Nov 4, 2023
769c5cb
add blog/2021-09-24-travis-security.md
jaimergp Nov 4, 2023
45d20ad
add blog/2021-11-03-tensorflow-gpu.md
jaimergp Nov 4, 2023
1149d73
add blog/2022-08-26-outreachy-wrap-up-blog-2022.md
jaimergp Nov 4, 2023
96efe68
add blog/2023-03-12-circle-ci-security-breach.md
jaimergp Nov 4, 2023
14f460f
add blog/2023-07-13-installer-security-fixes.md
jaimergp Nov 4, 2023
4a0b3e6
add authors
jaimergp Nov 4, 2023
9426697
remove demo blog post
jaimergp Nov 4, 2023
a19d1dc
pre-commit
jaimergp Nov 3, 2023
5f4fd36
rename to lowercase
jaimergp Nov 3, 2023
9ca74ec
upgrade to docusaurus v3
jaimergp Nov 4, 2023
757ddfe
fix image
jaimergp Nov 4, 2023
349febd
pre-commit
jaimergp Nov 4, 2023
eaa04c1
truncate here
jaimergp Nov 4, 2023
c7c6638
Merge pull request #13 from Quansight-Labs/migrate-blog
jaimergp Nov 5, 2023
8c5e697
Bring blog/ content to Docusaurus
jaimergp Jan 8, 2024
a69214b
enable blog plugin; add some extra redirects
jaimergp Jan 8, 2024
46180f7
show more blog posts in the sidebar
jaimergp Jan 8, 2024
5331517
ensure case is respected
jaimergp Jan 8, 2024
ad68b66
remove #readme anchor
jaimergp Jan 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,9 @@ The role responsible for each step is noted at the beginning.
mailing list.
- **Developer**: Merge the security fix PR
- **Developer**: Release the package and/or deploy the fix as appropriate
- **Developer & Coordinator**: Draft a [blog post](https://github.com/conda-forge/blog) and other
- **Developer & Coordinator**: Draft a [blog post](/blog) and other
announcement texts. This can be done in parallel with the previous steps, but consider using a
[private advisory](https://github.com/conda-forge/blog/security/advisories) for the text.
[private advisory](https://github.com/conda-forge/conda-forge.github.io/security/advisories) for the text.
- **Coordinator**: Publish the security advisory on the announcement date. If applicable, GitHub
will post the CVE to the MITRE database.
- **Coordinator**: Publish the blog post and other announcements (Element chat room, Twitter,
Expand Down
49 changes: 49 additions & 0 deletions blog/2019-12-06-cfep09.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
authors:
- cj-wright
tags: [autotick-bot]
---

# Automatically Deployed ABI Migrations

Handling application binary interface (ABI) migrations has always been a
hassle for Conda-Forge. Maintaining ABI consistency helps enable the
"just use conda-forge" experience for many of our users, making
certain that numpy's blas is the same as scipy's. As libraries update
their code, the new versions may be ABI incompatible, as function
signatures and other symbols may have changed, leading to the dreaded
`SegmentationFault` and other errors.

<!--truncate-->

Conda-Forge handles this by having a pinning file that tracks all the
currently supported ABIs. These pinned ABIs are then used to build the
downstream packages, making certain that all are consistent. As new
versions of pinned software are released the pins are updated, causing a
migration of the pin, and the rebuilding of all packages which rely on
the pinned package. In the past, this was handled by a change to the
global pinnings and a subsequent migration via the auto-tick bot. While
this worked, there were issues that this created. Firstly, this approach
could cause unsatisfiable build dependencies for new packages, as some
of the new package's dependencies had been compiled with the new pins,
but not all. Secondly, migrations happened in series, if a second pin
was moved while the first was being migrated then the migration could go
wrong as packages which were being rebuilt for the first pin got the
second pin before they were ready.

Conda-Forge Core has recently approved CFEP-9, a migration policy to fix
these issues. With CFEP-9 pinnings are proposed as small yaml snippets
which contain the new pins. The auto-tick bot then starts migrating the
packages in order, applying the yaml snippet to each package in turn. If
a second pinning change is issued then the bot starts the migration for
that package too, enabling the two migrations to work independently. If
a package needs a change in both pins then the maintainers can choose
the order in which they apply the pins by merging one pin before the
other.

This approach will yield much greater stability in migrations and will
enable more maintainers to issue migrations. Migrations can be issued by
putting a PR into
[conda-forge/conda-forge-pinning-feedstock](https://github.com/conda-forge/conda-forge-pinning-feedstock),
adding a file to the `migrations` folder, PRs into the auto-tick bot are
not needed anymore.
112 changes: 112 additions & 0 deletions blog/2020-02-04-gsoc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
---
authors:
- ocefpaf
tags: [autotick-bot, gsoc]
---

# Google Summer of Code 2020 improved automatic maintenance of `conda-forge`

The `conda-forge` "autotick" bot is a crucial part of `conda-forge`'s
infrastructure. It enables automatic maintenance of `conda-forge`
packages by pushing version updates to the underlying software and
enabling large migrations of packages from one dependency to another
(e.g., Python 3.7 to Python 3.8). As `conda-forg` grows in size, with
over 9,000 packages to date, automatic maintenance of the `conda-forge`
ecosystem will become even more important.

<!--truncate-->

We here at `conda-forge` have a large number of potential Google Sumer
of Code projects around maintenance and development of the autotick bot
infrastructure. These projects are high-impact, affecting the entire
`conda-forge` ecosystem. They also cover multiple systems including
databases, `conda`'s CDN provider, continuous integration providers,
and user interactions on GitHub.

Want to be a part of the team? Great! Take a look at the projects below
and get in touch with us on GitHub! You can check the [GSoC
label](https://github.com/regro/cf-scripts/labels/GSOC) for a
detailed listing of the issues that need work.

1. Maintenance and Refactoring

We have a large backlog of maintenance and refactoring issues that
are great for people with a range of experience from beginners to
true code ninjas.

- Integration Testing for the Autotick Bot

> Run true integration testing on a copy of the graph to better
> test code changes and improve our CI process.
>
> **Issue**: [regro/cf-scripts261](https://github.com/regro/cf-scripts/issues/261)
>
> **Experience Level**: advanced

- Work on the "code hardening"
[Milestone](https://github.com/regro/cf-scripts/milestone/4)

Address any of the issues in the milestone above related to code
refactoring and cleanup.

**Issues**: [regro/cf-scripts milestone #4](https://github.com/regro/cf-scripts/milestone/4)

**Experience Level**: beginner to advanced

2. Automated PRs for Globally Pinned Packages

`conda-forge` maintains a list of globally pinned packages. These
are typically dependencies whose version needs to be the same across
all of `conda-forge` (e.g., the compiler versions or packages like
`HDF5`). While we have infrastructure to run a migration of the
downstream packages from a given pinned package, we do not have
automated infrastructure to propose the migration of the pin itself.
The project here is to add this functionality to our infrastructure.

**Issue**: [regro/cf-scripts#665](https://github.com/regro/cf-scripts/issues/665)

**Experience Level**: advanced

3. Check the `conda` CDN for Updated Packages before Issuing PRs in a
Migration

`conda` relies on a CDN provider to serve the index of available
packages. There is a moderate ~30 minute delay between when a
package is uploaded to `anaconda.org` and when it will appear in the
`conda` index. We currently do not take this delay into account when
issuing PRs in a migration.

**Issue**: [regro/cf-scripts#595](https://github.com/regro/cf-scripts/issues/595)

**Experience Level**: beginner

4. Finish Migrations with PR into the `conda-forge` Pinnings File

Right now, when the migration of say package `ABC` to version `X`
from version `Y` is done, we do not automatically merge the change
in the globally pinned value of `ABC` into our listing of global
pinnings. We should be issuing a PR to the pinnings file once we
have determined that a suitable fraction of the packages effected by
a migration have been properly rebuilt.

**Issue**: [regro/cf-scripts#595](https://github.com/regro/cf-scripts/issues/595)

**Experience Level**: moderate

5. Fully Render `conda` Packages in order to Determine Migration
Dependencies

Determining the dependencies of a given package is actually a
computation expensive task due to the way `conda` recipes are
structured and parametrized through the use of `Jinja2` and
`conda-build-config.yaml` files. Currently, the autotick bot
examines the static metadata in the `meta.yaml` file and not the
fully rendered metadata. For this reason, we sometimes miss
dependencies of a given package that need to be migrated first.
Addressing this issue involves both calling the rendering process
and also scaling that process to the entire set of `conda-forge`
packages.

**Issue**: [regro/cf-scripts#664](https://github.com/regro/cf-scripts/issues/664)

**Experience Level**: moderate
Loading