Test/validate catalog-next launch plan in sandbox

[adborden]

User Story

In order to reduce risk and keep the catalog-next launch as simple and foolproof as possible, a data.gov operator wants a detailed launch plan validated in the sandbox environment.

Acceptance Criteria

• GIVEN I follow the launch steps in the launch plan
  WHEN when I visit https://catalog.datagov.us
  THEN I see CKAN 2.8 is running successfully with data
  AND the dataset count matches https://catalog-next.sandbox.datagov.us/dataset
• GIVEN I follow the rollback steps in the launch plan
  WHEN when I visit https://catalog.datagov.us
  THEN I see CKAN 2.3 is running successfully with data
  AND the dataset count matches https://catalog.sandbox.datagov.us/dataset

Details / tasks

• Create new CloudFront distribution in sandbox
• Add the existing catalog instance (catalog.sandbox.datagov.us) to the CloudFront distribution
• Update catalog.datagov.us DNS record to point to the CloudFront distribution
• Apply the steps in the launch plan
• Cleanup: Remove the DNS name and CloudFront distribution

[adborden]

Maybe we should do this in sandbox since we don't have a CloudFront distribution configured for staging and it would not be easy to add one.

[ghost]

@thejuliekramer Is there an estimate for this?

[thejuliekramer]

Linking #449

[thejuliekramer]

@cmugisha #449 was created to do load testing but we may need for the production environment to do real load testing

[thejuliekramer]

@cmugisha I think more information is needed for this - who is going to do this work and what work needs to be done

[thejuliekramer]

We need to understand more what CloudFront does so we know what to do with this ticket

[adborden]

Paired with @thejuliekramer on this today. We created a new CloudFront distribution in sandbox: https://d1xr5ix4dos50f.cloudfront.net/dataset Think of this URL as the equivalent of catalog.data.gov.

Right now, visiting https://d1xr5ix4dos50f.cloudfront.net/dataset will show catalog-classic (catalog.sandbox.datagov.us). After running through launch plan, https://d1xr5ix4dos50f.cloudfront.net/dataset will show catalog-next (catalog-next.sandbox.datagov.us). If it doesn't, it means there's a bug in our launch plan and we should update the launch plan to fix it :)

@thejuliekramer and I discussed some of the details of this. We added the catalog-next backend to CloudFront and discussed what the actual switchover looks like through the CloudFront UI in the AWS console.

There's a catch in that the CKAN config (production.ini) can only point to one URL at a time, right now catalog-next is using catalog-next.sandbox.datagov.us. Because of this some redirects don't work correctly on https://d1xr5ix4dos50f.cloudfront.net, but that is just a test issue we can ignore. It does mean that we'll want to update production.ini as pre-step to the launch plan.

[thejuliekramer]

@adborden the pull request has been deployed I changed the behavior to now point to the catalog-next url as seen here

When I go the url d1xr5ix4dos50f.cloudfront.net it is still pointing to old catalog... is there I step I missed?

[adborden]

@thejuliekramer we probably need to invalidate the cache... Under the "Invalidations" tab, enter * to invalidate everything.

We should add this to the launch plan steps.

[thejuliekramer]

@adborden this worked! d1xr5ix4dos50f.cloudfront.net

I updated the launch plan steps to be more specific