prepare 6.7.10 release (#196)

* Enable hourly Relay integration tests against Production (#223)

* Record response body when relay archive download test fails (#225)

There is some additional information provided in the response body for certain 504 error codes and this should allow us to see it.

* Fix the broken production integration tests (#226)

* improve metrics documentation and fix route strings in docs (#227)

* improve metrics documentation and fix route strings in docs

* clarify mobile + browser terminology

* use more efficient jsonstream encoding/decoding for stream data and evaluations (#228)

* recognize alias events from non-v3-schema payloads and forward them unchanged

* environment should still be usable even if the client timed out

* bump dependency versions for SDK fixes

* update go-server-sdk-dynamodb to get newer AWS SDK

* minor clarification about Relay.Close (#234)

* Use the Go releaser template (#233)

* [ch102248] big segment sync with redis (#235)

* fix makefile so it tries building all the test code first before running any of it (#239)

* (big segments #1) add basic big segments configuration for SDK clients (#237)

* (big segments #2) add more abstraction around big segments implementation for testability (#238)

* [102253] bigsegment status / config (#242)

* big segments integration test + misc fixes (#240)

* use latest URL paths for big segments endpoints

* add SDK DynamoDB integration for big segments (#241)

* fix broken link in Markdown docs (#246)

* make sure newly added credentials for existing environments are accepted in requests (#244)

* don't return 503 if SDK initialization has timed out

* add in-repo docs about error/503 behavior (#249)

* [ch102255] BigSegments DynamoDB (#245)

* add init timeout config option + better test coverage + misc refactoring (#250)

* fix example build command

* use public prerelease tags instead of private dependencies

* fix Go installation in CI

* update SDK dependencies for JSON number parsing bugfix

* update gorilla/mux to 1.8.0

* update OpenCensus packages

* add Go 1.16 CI + "latest Go" CI + use latest 1.15 patch for release

* cimg images use "current", not "latest"

* seems there isn't any cimg/go "latest" or "current"

* add daily package build test in CI

* job names

* bump SDK version for traffic allocation feature

* [ch113491] update alpine base image (#258)

* use latest prerelease SDK

* fix enabling of test tags in CI

* add DynamoDB docker image in CI

* set a polling base URI in end-to-end tests since big segments logic will use it

* fix initialization logic so SDK client creation errors aren't lost when big segments are enabled

* fix use of prefix key in DynamoDB + improve tests (#260)

* more debug logging, less info logging for big segments logic

* make logging of big segments patch version mismatch clearer and use Warn level

* fix log parameter

* fix DynamoDB updates for big segments metadata

* add test to make sure sync time and cursor can be updated independently

* only start big seg synchronizer if necessary

* use SDK GA releases

* change applyPatch to exit early on version mismatch; go back to restarting stream in this case

* add unit tests for version mismatch behavior + DRY tests

* add log assertion

* fix retry logic on big segments stream failure

* add more logging for big segments connection status

* fix logging assertion

* add more big segments integration tests

* fix overly-time-sensitive file data tests

* fix more flaky tests

* run big segments tests with DynamoDB too

* Migrate transitive dep (jwt-go) to use modern version without vulnerability.

* Edit doc

* move Relay release logic to .ldrelease script

* suppress SDK big segments status query if we've never synced big segments

* dump Relay logs including debug logs if integration test fails

* include environment prefix in BigSegmentSynchronizer logging

* increase big segment integration test timeout (#274)

* generate client-side stream pings if big segments have changed

* clear big segments cache as needed + simplify state management

* fix tests and simplify component creation

* use GA releases of SDK packages

* disable CI package-build-test in Go 1.16+

* Migrate Relay release to Releaser v2 and support dry run (#278)

* Adding degraded doc blurb for big segments (#280)

* respect Redis password & TLS options for big segments; add Redis password integration tests

* redact Redis URL password in logs and status resource

* update go-server-sdk-redis-redigo to 1.2.1 for Redis URL logging fix

* Part 1, add the config and the documentation for the new config

* Part 2, Add the configuration validation and test

* Part 3, the actual logic to include the headers in the CORS Access-Control-Allow-Headers

* Linter

* update Alpine version to 3.14.2 to fix openssl CVEs

* Fix the global variable modification

* Go format

* turn off unnecessary metrics integrations in config for Docker smoke test

* rename test.env to smoke-test.env to clarify what it's for

* fix setting of custom Access-Control-Allow-Origin and add test (#285)

* add more explanatory test output and more verbose debugging for big segments integration tests (#287)

* update to Go 1.16.10 + Alpine 3.14.3; add some docs about releases (#288)

* update go-server-sdk-consul version for Consul API version update

* override x/crypto dependency version for CVE-2020-29652

* bump Prometheus dependency to eliminate jwt-go vulnerability

* drop support for Go 1.14 & 1.15

* make sure defaults are always applied for base URL properties

* rm unused

* rm unnecessary linter directive

* add separate configuration for server-side/client-side SDK base URLs & update the defaults

* remove Whitesource CI job + remove obsolete dependency issue note

* don't include any big segment status info in status resource unless that feature is active (#296)

* don't include any big segment status info in status resource unless that feature is active

* fix Big Segments staleness logic in status resource

* documentation

* update x/text package for vulnerability GO-2021-0113

* add Trivy security scan to CI (#297)

* add daily re-scan with Trivy

* use long timeout when awaiting changes related to file mod watching

* update Go version to 1.17.6 (#301)

* always terminate if auto-config stream fails with a fatal error

* pass along tags header when proxying events

* comments, rm debugging

* fix auth header logic

* fix auth header logic some more

* comments

* add tags header to CORS header whitelist (#304)

* update to Alpine 3.14.4 for CVE-2022-0778 fix

* force upgrade of openssl in Alpine

* also upgrade libretls

* fix it in both files

* update to Alpine 3.14.5 for CVE-2022-0778/CVE-2018-25032 (#308)

* update to Alpine 3.14.5 for CVE-2022-0778

* revert patches that are now included in Alpine 3.14.5

* add scripts for checking and updating Go/Alpine versions (#309)

* update to Alpine 3.14.5 for CVE-2022-0778

* add scripts for checking and updating Go/Alpine versions

* also make sure the Docker images really exist

* update CONTRIBUTING.md

* fix file rename

* revert patches that are now included in Alpine 3.14.5

* update Alpine to 3.14.6 for CVE-2022-28391

* update SDK packages (includes sc-136333 fix)

* don't include "v" prefix in Docker image version

* update go-server-sdk-dynamodb for data size error fix & add docs (#316)

* update builds to use Go 1.17.9 and fix the update script

* update go-server-sdk-consul to latest release

* update remote Docker version

* update golang.org/x/crypto for CVE-2022-27191 (#321)

* update golang.org/x/crypto for CVE-2022-27191

* fix go.sum

* update eventsource for SSE output efficiency fix (#322)

* Cache the replay event in case we get multiple new client connections (#189)

* Cache the replay event in case we get multiple new client connections

* Use singleflight to ensure only one replay event is generated at a time

Co-authored-by: Moshe Good <moshe@squareup.com>

* don't install curl in Docker images

* fix makefile logic for lint step

* remove indirect curl-based request logic in integration tests

* fix linter installation

* update Go to 1.17.11, Alpine to 3.16.0

* improve concurrency test to verify that the data is or isn't from a separate query

* fix lint warnings and remove unnecessary error return

* update libssl & libcrypto versions for CVE-2022-2097

* add security scan of already-published Docker image (#328)

Co-authored-by: Ben Woskow <48036130+bwoskow-ld@users.noreply.github.com>
Co-authored-by: Andrew Shannon Brown <ashanbrown@users.noreply.github.com>
Co-authored-by: Eli Bishop <eli@launchdarkly.com>
Co-authored-by: LaunchDarklyCI <dev@launchdarkly.com>
Co-authored-by: hroederld <hroeder@launchdarkly.com>
Co-authored-by: LaunchDarklyReleaseBot <launchdarklyreleasebot@launchdarkly.com>
Co-authored-by: Dan Richelson <drichelson@launchdarkly.com>
Co-authored-by: Dan Richelson <drichelson@users.noreply.github.com>
Co-authored-by: Ben Woskow <bwoskow@launchdarkly.com>
Co-authored-by: Louis Chan <lchan@launchdarkly.com>
Co-authored-by: Louis Chan <91093020+louis-launchdarkly@users.noreply.github.com>
Co-authored-by: Moshe Good <moshegood@users.noreply.github.com>
Co-authored-by: Moshe Good <moshe@squareup.com>

.circleci/config.yml

@@ -106,7 +106,8 @@ workflows:
           branches:
             only: v6
     jobs:
-      - security-scan-only
+      - security-scan-of-current-build
+      - security-scan-of-published-image
 
 jobs:
   go-test:
@@ -309,7 +310,7 @@ jobs:
       - store_artifacts:
           path: dist/
 
-  security-scan-only:
+  security-scan-of-current-build:
     docker:
       - image: cimg/go:<<pipeline.parameters.go-release-version>>
         environment:
@@ -342,3 +343,37 @@ jobs:
           command: trivy image --exit-code 1 launchdarkly/ld-relay:latest
           # Note that the "latest" here is not the latest published image in DockerHub;
           # it's the one that was just built locally in the previous step.
+
+  security-scan-of-published-image:
+    docker:
+      - image: cimg/go:<<pipeline.parameters.go-release-version>>
+        environment:
+          <<: *environment
+
+    steps:
+      - checkout
+      - setup_remote_docker:  # start docker engine
+          version: <<pipeline.parameters.remote-docker-version>>
+
+      - run:
+          name: add package source for Trivy
+          command: |
+            sudo apt-get install wget apt-transport-https gnupg lsb-release
+            wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
+            echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list
+
+      - run:
+          name: install packages
+          command: |
+            sudo apt-get update
+            sudo apt-get install rpm trivy
+
+      - run:
+          name: Trivy security scan
+          command: trivy image --exit-code 1 launchdarkly/ld-relay:latest
+          # Here, because we did not locally rebuild the image, it is pulling the latest release
+          # from DockerHub and re-scanning that (to detect any vulnerabilities that were reported
+          # after the time of the release). This is important because, if our build installs the
+          # latest versions of things at the time that the build happens, a fresh build might be
+          # getting patches that the released version did not have.
+

Dockerfile

@@ -30,6 +30,7 @@ RUN addgroup -g 1000 -S ldr-user && \
 
 RUN apk add --no-cache \
     ca-certificates \
+ && apk add --upgrade libcrypto1.1 libssl1.1 \
  && update-ca-certificates \
  && rm -rf /var/cache/apk/*
 

Dockerfile.goreleaser

@@ -7,6 +7,7 @@ FROM alpine:3.16.0
 
 RUN apk add --no-cache \
     ca-certificates \
+ && apk add --upgrade libcrypto1.1 libssl1.1 \
  && update-ca-certificates \
  && rm -rf /var/cache/apk/*
 

internal/filedata/archive_manager_test_base_test.go

@@ -123,7 +123,7 @@ func (p archiveManagerTestParams) requireMessage() testMessage {
 	select {
 	case m := <-p.messageHandler.received:
 		return m
-	case <-time.After(500 * time.Millisecond):
+	case <-time.After(2 * time.Second):
 		require.Fail(p.t, "timed out waiting for message")
 		return testMessage{}
 	}