Releases: google/exposure-notifications-verification-server
v0.26.1
Changes since v0.26.0
This patch release fixes an issue when there are more than 50 realms.
Operations
- Handle gorm log at debug level instead of error. (#2023, @sethvargo)
- Handle string or log in gorm log type. (#2027, @sethvargo)
Infrastructure
- Lower Cloud KMS database-encrypter rotation to 90d (#2019, @sethvargo)
- Use a wildcard for redirect domains. This fixes an issue for installations with more than 50 realms exceeding the limit on the URL map. (#2029, @sethvargo)
Dependencies
Added
Nothing has changed.
Changed
Nothing has changed.
Removed
Nothing has changed.
v0.26.0
Changes since v0.25.0
Upgrade notes
- If you have not done so already, you need to configure and enable authenticated SMS on the e2e-runner realm.
Enhancements
- Enable Authenticated SMS by default. To disable this functionality, set
ENABLE_AUTHENTICATED_SMS
tofalse
. The ability to disable this feature may be removed in a future release. (#1983, @mikehelmick) - Add Arabic translations for case worker UI. (#1969, @mikehelmick)
- Add translations for email verify pages. (#1999, @sethvargo)
- Add translations for signout and realm selection pages. (#1997, @sethvargo)
- Add translations for the login and password reset pages (#1978, @sethvargo)
- Add translations to change/select password pages. (#1996, @sethvargo)
- Add a warning on realm settings page that when user report is enabled, authenticated SMS should be enabled too. (#1991, @mikehelmick)
- Add guidance for user report SMS template (#1971, @mikehelmick)
- Add middleware to make OS present in request context (#1973, @mikehelmick)
- Don't let greyed out fields be changed in signing keys. Alert when creating a user that already exists. Fix ENX upgrade SMS template text when user-report is enabled. (#1989, @mikehelmick)
- Drop UPDATE privileges on audits table. This makes audit entries immutable (but still deletable). (#1966, @sethvargo)
- In the mobile apps screen, provides a user hint that AppStore redirect should be disabled if there is a custom app and iOS ENX in the same region. (#1977, @mikehelmick)
- Lower data-layer TTLs on background jobs (#1962, @sethvargo)
- Mention links should be clicked on mobile devices and internationalize 404 pages. (#2002, @sethvargo)
- Minify js and css assets on build. (#1992, @sethvargo)
- Move CSRF implementation into session. (#1963, @sethvargo)
- Remove XSS header. This is deprecated by most modern browsers and can cause client-side security issues. (#1961, @sethvargo)
- Return gzipped responses if the client supports gzipped responses. (#1998, @sethvargo)
- Set cache headers on static assets, do not apply middlewares. (#1990, @sethvargo)
- Switch to loading javascript using
defer
to minimize load times. (#1988, @sethvargo) - Switch to using
dir="rtl"
for right-to-left languages. (#1979, @sethvargo) - Use min versions of intl-tel-input script. (#2000, @sethvargo)
- When invalid codes are attempted to be used, the OS is captured in the realm stats [unknown, ios, android] (#1975, @mikehelmick)
Infrastructure
- Change load balancer https redirect rules to only match known host names, otherwise redirect to UI server. (#1982, @mikehelmick)
- Generates specific host name matches for all ENX redirect subdomains. For anything not matching on the http port, the user set sent to an information page at g.co/ens (#1984, @mikehelmick)
Statistics
- New chart on the realm admin stats page showing the invalid codes entered by operating system. (#1976, @mikehelmick)
Operations
- Centralize X-Forwarded-For processing to get real client IPs (#1964, @sethvargo)
- Cleanup job will run every 15 minutes (#1965, @mikehelmick)
- Do not mark PhoneNumberHMAC as required (#1960, @sethvargo)
- Upgrade to gcloud 335.0.0 in deployment scripts. (#1994, @sethvargo)
Misc
- Check password validity on page load to reduce code duplication. (#1980, @sethvargo)
- Experimental: Adds an optional Web UI that can be launched as an embedded Webview for requesting user initiated verification codes (#2011, @mikehelmick)
Dependencies
Added
Changed
- cloud.google.com/go/firestore: v1.4.0 → v1.5.0
- cloud.google.com/go: v0.79.0 → v0.81.0
- github.com/Azure/azure-sdk-for-go: v52.4.0+incompatible → v53.1.0+incompatible
- github.com/aws/aws-sdk-go: v1.37.30 → v1.38.17
- github.com/chromedp/cdproto: bf465a4 → 0942afb
- github.com/chromedp/chromedp: v0.6.5 → v0.6.10
- github.com/envoyproxy/go-control-plane: fd9021f → 668b12f
- github.com/fatih/color: v1.10.0 → v1.9.0
- github.com/fsnotify/fsnotify: v1.4.9 → v1.4.7
- github.com/golang/groupcache: 8c9f03a → 41bb18b
- github.com/golang/protobuf: v1.4.3 → v1.5.2
- github.com/google/exposure-notifications-server: ecf9dee → v0.26.0
- github.com/hashicorp/vault/api: 38d91b7 → v1.1.0
- github.com/hashicorp/vault/sdk: 8477cfe → v0.2.0
- github.com/jackc/pgconn: v1.8.0 → v1.8.1
- github.com/jackc/pgtype: v1.6.2 → v1.7.0
- github.com/jackc/pgx/v4: v4.10.1 → v4.11.0
- github.com/mattn/go-colorable: v0.1.8 → v0.1.7
- github.com/microcosm-cc/bluemonday: v1.0.4 → v1.0.6
- github.com/mitchellh/go-wordwrap: v1.0.1 → v1.0.0
- github.com/onsi/ginkgo: v1.14.2 → v1.12.0
- github.com/onsi/gomega: [v1.10.4 → v1.9.0](https://github.com/onsi/gomega/compare/v1.1...
v0.25.0
Changes since v0.24.0
Upgrade notes
-
This release improves the way metrics and alerts are handled to reduce the number of false alerts in favor of forward-progress alerting. When you run Terraform for the first time, you may see errors that an alert cannot be created due to a missing metric. These errors (and only these errors) can be safely ignored for now. Continue with the deployment steps. After all services are deployed, manually invoke each of the services via Cloud Scheduler. Finally, re-run Terraform to create the alerts based on the new metrics.
-
This release contains new services. Run
terraform taint module.en.null_resource.build
to ensure the new services are built during the Terraform run. This is a one-time operation.
Reliability
- Add authorization header to API call in database backup service. (#1936, @sethvargo)
- Add special "_all" key to apply to all service environments. The special key
_all
will apply to all services. This is useful for common configuration like log-levels. A service-specific configuration overrides a value in_all
. There are no default values for_all
, so the default behavior is unchanged. (#1951, @sethvargo) - Set alignment to 1m (#1946, @sethvargo)
- Simplify background job error handling and response consistency. (#1950, @sethvargo)
- Switch e2e-runner to forward progress alerting. (#1952, @sethvargo)
- Switch to forward-progress alerting for most background jobs. See the updated ForwardProgressFailed.md documentation for more information. (#1929, @sethvargo)
- Switch to standard alerts away from MQL (#1949, @sethvargo)
- Allow overriding default fp alerts in Terraform (#1939, @sethvargo)
- Increment metric on success for background jobs (#1926, @sethvargo)
- Introduce a new service:
backup
. Be sure to taint the null_resource builder during the Terraform apply to get the new service version. (#1932, @sethvargo) - Lower in-memory cache time and return a warning about cached values (#1925, @sethvargo)
- Re-add delta aligner (#1940, @sethvargo)
Enhancements
- All admin actions to issue user-report level verification codes, that can result in SELF_REPORT (#1955, @mikehelmick)
- Add support for self-report initiation and certification. This feature is disabled by default, enable by setting
ENABLE_USER_REPORT
totrue
(#1930, @mikehelmick) - User report can use signed SMS and relax template restrictions. (#1944, @mikehelmick)
- Ensure users retain system admin when being added to a realm (#1954, @sethvargo)
- Mongolian translations for case worker UI. (#1956, @mikehelmick)
- Remove warnings about unused environment variables. (#1935, @sethvargo)
Statistics
- Add charts for user-reported codes and claim stats. (#1941, @mikehelmick)
- Add sliders to graphs for publish requests and TEKs published (#1947, @mikehelmick)
- Composite stats are now available on the adminapi,
realm/composite.csv
andrealm/composite.json
(#1937, @mikehelmick) - Number of revision request are shown on key server stats if user initiated reporting is enabled. This is the only EN Express scenario where key revision could occur. (#1945, @mikehelmick)
Misc
- Rename CleanupStatus to LockStatus and fix an issue where modeler would return a non-200 when the lock was already held, causing Cloud Scheduler to fail. (#1927, @sethvargo)
- Use an in-memory key server for end-to-end tests. (#1922, @sethvargo)
Dependencies
Added
- github.com/OneOfOne/xxhash: v1.2.2
- github.com/apparentlymart/go-textseg/v13: v13.0.0
- github.com/armon/consul-api: eb2c6b5
- github.com/cespare/xxhash: v1.1.0
- github.com/checkpoint-restore/go-criu/v4: v4.1.0
- github.com/cilium/ebpf: v0.2.0
- github.com/coreos/bbolt: v1.3.2
- github.com/coreos/etcd: v3.3.10+incompatible
- github.com/coreos/go-systemd/v22: v22.1.0
- github.com/cyphar/filepath-securejoin: v0.2.2
- github.com/dgryski/go-sip13: e10d5fe
- github.com/godbus/dbus/v5: v5.0.3
- github.com/magiconair/properties: v1.8.0
- github.com/moby/sys/mountinfo: v0.4.0
- github.com/mrunalp/fileutils: v0.5.0
- github.com/oklog/ulid: v1.3.1
- github.com/opencontainers/selinux: v1.8.0
- github.com/pelletier/go-toml: v1.2.0
- github.com/prometheus/tsdb: v0.7.1
- github.com/seccomp/libseccomp-golang: v0.9.1
- github.com/spaolacci/murmur3: f09979e
- github.com/spf13/afero: v1.1.2
- github.com/spf13/cast: v1.3.0
- github.com/spf13/jwalterweatherman: v1.0.0
- github.com/spf13/viper: v1.4.0
- github.com/syndtr/gocapability: 42c35b4
- github.com/vishvananda/netlink: v1.1.0
- github.com/vishvananda/netns: 0a2b9b5
- github.com/vmihailenco/msgpack/v4: v4.3.12
- github.com/vmihailenco/tagparser: v0.1.1
- github.com/willf/bitset: v1.1.11
- github.com/xordataexchange/crypt: b2862e3
- github.com/zclconf/go-cty-debug: b22d67c
Changed
- cloud.google.com/go/storage: v1.13.0 → v1.14.0
- cloud.google.com/go: v0.77.0 → v0.79.0
- contrib.go.opencensus.io/exporter/prometheus: 6bcf6f8 → v0.3.0
- github.com/Azure/azure-sdk-for-go: v51.2.0+incompatible → v52.4.0+incompatible
- github.com/agext/levenshtein: v1.2.1 → v1.2.3
- github.com/aws/aws-sdk-go: v1.37.12 → v1.37.30
- github.com/containerd/console: [c12b1e7 → v1.0.1](https://github.com/containerd/conso...
v0.24.0
Changes since v0.23.0
Security
-
Fix a security vulnerability where, with a carefully crafted request or malicious proxy, a user with UserWrite permissions could create another user with higher privileges than their own due to insufficient checks on the allowed set of permissions.
-
Re-auth login redirect uses allowlist for post auth redirects. (#1919, @mikehelmick)
Bug fixes
- Fixes nilptr error in stats puller (not in a previous labeled release) (#1911, @mikehelmick)
System admin
- Add chaff reporting to system admin page. This will show whether a realm has issue any chaff requests in the past 7 days. (#1903, @sethvargo)
Operations
-
Add client-side retry logic and parallelize stats puller. The default parallelize is 5, but it can be customized with
STATS_PULLER_MAX_WORKERS
. There is also a behavior change. The stats-puller previously always returned success (but logged errors on failure). This changes the puller to return a non-200 response code if there are still failures after all retries have executed. (#1905, @sethvargo) -
Allow customizing global log retention period for all services in the project. The default value is 14 days. Note: this differs from the unconfigured value of 30 days!. To retain the existing behavior, set
log_retention_period
to30
in the Terraform configuration. However, we strongly recommend using a 14-day retention period instead. (#1902, @sethvargo) -
Improve service timeouts. In-request services have a timeout of 10 seconds while background jobs have a timeout of 900s. The Cloud Scheduler timeout (which invokes the background jobs) has a 60s buffer to reduce timeout races. (#1916, @sethvargo)
-
Remove modeler backend service (it is not public-facing). (#1917, @sethvargo)
-
Set Binary Authorization service annotations on Cloud Run services. (#1909, @sethvargo)
Misc
- Indonesia (id) language translation (#1890, @dwisiswant0)
Dependencies
Added
Nothing has changed.
Changed
- github.com/google/exposure-notifications-server: v0.23.0 → v0.24.0
Removed
Nothing has changed.
v0.23.1
Changes since v0.23.0
[SECURITY FIX] This release fixes a security vulnerability where, with a carefully crafted request or malicious proxy, a user with UserWrite permissions could create another user with higher privileges than their own due to insufficient checks on the allowed set of permissions.
v0.23.0
Release notes for v0.23.0
Changelog since v0.22.0
Notice
- Release contains breaking changes for server operators, please see tagged release notes below.
- This release upgrade to golang version 1.6
Changes by Kind
New Features
- Allow system administrators to set system notices. See the system admin guide for more information. (#1877, @sethvargo)
- Introduce configurable Authenticated SMS failure modes. The default behavior is to "fail open" (continue on error). Operators can configure the system to "fail closed" (halt on error) by setting the
SMS_FAIL_CLOSED
environment variable on theserver
andadminapi
services. We recommend leaving the default configuration. Regardless of the configuration, this also introduces a new non-paging alert to inform operators when an out-of-threshold number of failures occur while signing SMS messages, per realm. See the new playbook for more information. (#1834, @sethvargo)
Documentation
- Document authenticated sms in realm admin guide (#1848, @sethvargo)
Statistics
- Clarify naming on code claim charts. Change units on mean claim graph from seconds to minutes. (#1867, @mikehelmick)
- The tokens claimed/invalid chart has been removed (data is still in exports)
- Add total publish requests to the codes/issued claim chart (if key server stats are enabled)
- Separate out revisions and missing onset from TEKs published
- Add total publish requests and missing onset as overlays on the publish bar chart (#1886, @mikehelmick)
Bug Fixes and Improvements
- BREAKING Remove backwards-compatibility for
TOKEN_SIGNING_KEY
. Specifying multiple values has been deprecated since 0.21 and token signing keys have moved into the database. If you are on an older version, you **MUST upgrade to v0.22 for at least 24h before applying this update.*- You should ensure that the value ofTOKEN_SIGNING_KEY
points to a key (not a key version) in the service environment before applying this upgrade. Since v0.22,TOKEN_SIGNING_KEY
accepted a key version or a parent key. This release only accepts a parent key. (#1872, @sethvargo) - Link out to Twilio splits calculator for advanced calculations (#1858, @sethvargo)
- Add and connect recovery middleware (#1827, @sethvargo)
- Drop support for crypto/dsa in public key parsing. The EN system only supports ecdsa keys, so this will not affect the system. (#1866, @sethvargo)
- Make per-test timeout a project function (#1826, @sethvargo)
- Merge e2e tests and integration tests (#1856, @sethvargo)
- Only return a signer if Authenticated SMS is enabled (#1843, @sethvargo)
- Only update SMS preview when the textarea is updated (#1840, @sethvargo)
- Reduce arbitrary sleeps in tests (#1859, @sethvargo)
- Refactor user tests to use less chromedp (#1852, @sethvargo)
- Remove shared state from tests (#1888, @sethvargo)
- Remove unneeded chromedp from realmadmin and realmkeys (#1871, @sethvargo)
- Simulate sending SMS in e2e-runner and enable authenticated SMS for the e2e test realm. Operators will either need to opt out of the e2e-runner service testing SMS paths by setting
E2E_SKIP_SMS
(not recommended) or configure the e2e test realm with Twilio Test Credentials. See the updated production guide for screenshots and examples. We recommend configuring the e2e-test-realm before upgrading the service to minimize alerts. (#1839, @sethvargo) - Upgrade e2e-runner's HTTP client. This changes the required configuration for
KEY_SERVER
from the full URL to the/publish
endpoint to just the URL to the key server. Where previously you may have configuredKEY_SERVER=https://foo.bar/v1/publish
, please re-configure withKEY_SERVER=https://foo.bar
. The system attempts to maintain backwards compatibility by parsing the URL, but this may be removed at a later date. (#1850, @sethvargo) - Upgrade to Firebase Auth version 8.2.9. (#1881, @sethvargo)
- Upgrade to Go 1.16 (#1865, @sethvargo)
- Use append instead of inserting at index in e2e test (#1822, @sethvargo)
- Use less chromedp and less database churn in API key tests (#1857, @sethvargo)
- Use less chromedp in login controller (#1863, @sethvargo)
- Use less chromedp on admin controller tests (#1861, @sethvargo)
- Use less chromedp on codes tests (#1862, @sethvargo)
- Wrap returned error from decoding form (#1828, @sethvargo)
Infrastructure
- Add structured logger for gorm (#1845, @sethvargo)
- Admin API gets signing config in terraform (#1831, @mikehelmick)
- Also run tests on pushes to the main branch (#1854, @sethvargo)
- Change rate limiting defaults from 60/min to 120/min (#1860, @sethvargo)
- Change to tmpdir when installing linters (#1882, @sethvargo)
- Disable authenticated sms in tests when the creds are missing (#1855, @sethvargo)
- Do not remove ASSETS_PATH and LOCALES_PATH from Terraform just yet. (#1869, @sethvargo)
- Do not use embedded fs in dev mode (#1878, @sethvargo)
- Enable query insights (#1879, @sethvargo)
- Fixing path of buildinfo package for build flags (#1835, @bschlaman)
- Get Cloud SQL Proxy from new link for migrate (#1842, @sethvargo)
- Ignore more annotations in Terraform diff (#1821, @sethvargo)
- Set blobstore, key manager, and secret manager in migrate (#1818, @sethvargo)
- Set blobstore, key manager, and secret manager on services...
v0.22.2
Changes since v0.22.1
Misc
- Ignore more annotations in Terraform diff (#1821, @sethvargo)
- Use append instead of inserting at index in e2e test (#1822, @sethvargo)
See also: changes since v0.21
Dependencies
Added
Nothing has changed.
Changed
Nothing has changed.
Removed
Nothing has changed.
v0.22.1
Changes since v0.22.0
Operations
- Ignore more annotations in Terraform diff (#1379, @sethvargo)
- Set blobstore, key manager, and secret manager in migrate (#1377, @sethvargo)
- Set blobstore, key manager, and secret manager on services (#1378, @sethvargo)
See also: changes since v0.21.0.
Dependencies
Added
Nothing has changed.
Changed
Nothing has changed.
Removed
Nothing has changed.
has changed._
v0.22.0
Changes since v0.21.0
Upgrade notes
- Cloud-specific dependencies are now a compile-time dependency. When building the binary, you must specify the build tag for your target environments to compile the appropriate Key Manager, Secret Manager, and Blobstore support for that target platform.
Build & CI/CD
- Allow parallel linting, fix missing newline in tabcheck (#1806, @sethvargo)
- Move tabcheck into GitHub Actions (#1803, @sethvargo)
- Run go tests on GitHub Actions (#1809, @sethvargo)
- Switch linter CI to GitHub Actions (#1801, @sethvargo)
Observability and reliability
- Add more admin tests (#1754, @sethvargo)
- Add more rotation tests (#1759, @sethvargo)
- Add more tests for appsync (#1760, @sethvargo)
- Add more tests for rbac (#1757, @sethvargo)
- Add test coverage to activate keys and keys index (#1807, @whaught)
- Basic coverage for /codes pages (#1746, @whaught)
- Do not produce debug logs for rate limiting facets. Do log info when a key has been rate limited. The key will be HMACed and non-identifiable, but correlatable. (#1774, @sethvargo)
- Update probers to check database for uptime (#1739, @sethvargo)
SMS
- Highlight when a message crosses an SMS boundary (#1773, @sethvargo)
- Include multiline and whitespace in search in SMS preview to fix a bug where an SMS message contained newline characters. This did not affect delivered SMSes, only the preview window. (#1745, @sethvargo)
Stats
- Clarify numbering of ticks on stats charts (#1788, @whaught)
- Custom chart tooltips with % of total (#1782, @whaught)
- Document example key server statistics (#1790, @sethvargo)
- Minor wording fix in TEK publish metrics chart description (#1765, @bschlaman)
Terraform
- Allow for redis auth to be disabled and terraform to apply successfully (#1751, @mikehelmick)
- Fix broken Terraform envvars (#1753, @sethvargo)
- Tune ignore_changes to prevent recurring diff in Terraform (#1775, @sethvargo)
Misc
- Ensure error from retry is bubbled to the caller when generating a code (#1800, @sethvargo)
- Ensure the login page doesn't refresh session freshness (#1772, @whaught)
- Upgrade to gcloud 324.0.0 in builds (#1768, @sethvargo)
Dependencies
Added
Nothing has changed.
Changed
- cloud.google.com/go/storage: v1.12.0 → v1.13.0
- cloud.google.com/go: v0.74.0 → v0.76.0
- github.com/Azure/azure-sdk-for-go: v49.2.0+incompatible → v51.2.0+incompatible
- github.com/Azure/azure-storage-blob-go: v0.12.0 → v0.13.0
- github.com/Azure/go-autorest/autorest/adal: v0.9.10 → v0.9.13
- github.com/Azure/go-autorest/autorest/azure/auth: v0.5.5 → v0.5.7
- github.com/Azure/go-autorest/autorest: v0.11.15 → v0.11.18
- github.com/Azure/go-autorest/logger: v0.2.0 → v0.2.1
- github.com/DataDog/datadog-go: v3.7.1+incompatible → v3.2.0+incompatible
- github.com/alecthomas/units: 1786d5e → ff826a3
- github.com/armon/go-metrics: v0.3.4 → v0.3.3
- github.com/aws/aws-sdk-go: v1.36.16 → v1.37.10
- github.com/bitly/go-hostpool: v0.1.0 → a3a6125
- github.com/chromedp/cdproto: be40c82 → bf465a4
- github.com/chromedp/chromedp: v0.5.4 → v0.6.5
- github.com/circonus-labs/circonusllhist: v0.1.4 → v0.1.3
- github.com/cncf/udpa/go: efcf912 → 5459f2c
- github.com/containerd/continuity: 1805252 → 50096c9
- github.com/coreos/pkg: 399ea9e → 3ac0863
- github.com/envoyproxy/go-control-plane: v0.9.7 → fd9021f
- github.com/fatih/color: v1.9.0 → v1.10.0
- github.com/frankban/quicktest: v1.10.0 → v1.11.3
- github.com/ghodss/yaml: 25d852a → v1.0.0
- github.com/gocql/gocql: 34081ed → f6df828
- github.com/google/exposure-notifications-server: v0.21.0 → v0.22.0
- github.com/google/gofuzz: v1.1.0 → v1.0.0
- github.com/google/pprof: 1bf35d6 → d980be6
- github.com/google/uuid: v1.1.2 → v1.2.0
- github.com/gorilla/websocket: v1.4.2 → 4201258
- github.com/gostaticanalysis/analysisutil: v0.6.1 → 4088753
- github.com/hashicorp/consul/api: v1.4.0 → v1.3.0
- github.com/hashicorp/consul/sdk: v0.4.0 → v0.3.0
- github.com/hashicorp/go-cleanhttp: v0.5.1 → v0.5.2
- github.com/hashicorp/go-immutable-radix: v1.2.0 → v1.1.0
- github.com/hashicorp/go-msgpack: v1.1.5 → v0.5.3
- github.com/hashicorp/go-plugin: v1.3.0 → v1.0.1
- github.com/hashicorp/go-version: v1.2.1 → v1.2.0
- github.com/hashicorp/hcl/v2: v2.8.1 → v2.8.2
- github.com/hashicorp/mdns: v1.0.1 → v1.0.0
- github.com/hashicorp/memberlist: v0.1.4 → v0.1.3
- github.com/hashicorp/serf: v0.8.3 → v0.8.2
- gi...