Skip to content
This repository has been archived by the owner on Mar 28, 2023. It is now read-only.

4x400: Early pass at a relay maintainer initcontainer #575

Merged
merged 41 commits into from
Apr 21, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
0ea3c0a
Early pass at a relay maintainer initcontainer
Shadowfiend Apr 10, 2020
5e85d0c
Add deployment of relay contract to migrations
Shadowfiend Apr 13, 2020
943f336
Flesh out relay maintainer initcontainer some more
Shadowfiend Apr 13, 2020
b15a22b
WIP relay-maintainer Deployment
Apr 13, 2020
f5536cd
Use env vars for ETH account material
Apr 15, 2020
ae311e6
Set Bitcoin host
Apr 15, 2020
ed17179
Merge branch 'master' into four-by-four-hundred
Apr 15, 2020
c2b4159
Build and publish InitContainer
Apr 15, 2020
e621410
Execute initcontainer jobs in keep-dev workflow for now
Apr 15, 2020
a5717bb
squash: comment filters for test deploy
Apr 15, 2020
7b54a54
Publish requires the build step to finish first
Apr 15, 2020
ac52304
Use existing dir path
Apr 16, 2020
fd4488d
Create image dir
Apr 16, 2020
f7d2da8
Deploy Summa relay to keep-dev
Apr 16, 2020
aa9d157
Update tbtc to get latest pre version
Apr 16, 2020
68c10b1
Use docker-node executor for relay maintainer
Shadowfiend Apr 17, 2020
a7924c7
Bump initcontainer to 0.14.0-pre release
Shadowfiend Apr 17, 2020
418c6e2
Bump relay initcontrainer to Node 13, use Alpine
Shadowfiend Apr 17, 2020
188eca3
Pull URL class out of Node url library
Shadowfiend Apr 17, 2020
02aa78f
Pull operator key and address to top level
Shadowfiend Apr 17, 2020
0b75d3d
Include network name + Bitcoin info from env
Shadowfiend Apr 17, 2020
dec7b72
Fix maintainer env template find/replace
Shadowfiend Apr 17, 2020
2b19680
WIP Don't do contract migration for testing build
Shadowfiend Apr 17, 2020
b4de285
Set SUMMA_RELAY_BCOIN_API_KEY with BITCOIN_API_KEY
Apr 17, 2020
ea8f738
Revert "Set SUMMA_RELAY_BCOIN_API_KEY with BITCOIN_API_KEY"
Apr 17, 2020
0b0d895
Fiddling with getting Bitcoin auth working
Apr 17, 2020
08d33df
Minimum viable Kube config for Bcoin
Apr 18, 2020
009f9a3
Set BITCOIN_API_KEY
Apr 19, 2020
a5a7777
Call it what it is
Apr 19, 2020
80f4aea
Update env vars
Apr 19, 2020
b7f91ac
Build/publish relay InitContainer in keep-test workflow
Apr 19, 2020
25d7034
Tag InitContainer image.
Apr 19, 2020
8856fac
fix: clear dirty lower-order bits in functionSignature
liamzebedee Apr 20, 2020
6391c6c
Rename createNewDepositFeeEstimate -> getNewDepositFeeEstimate
liamzebedee Apr 20, 2020
3d07ece
Remove unnecessary require assertions
liamzebedee Apr 20, 2020
db763e8
Ignore kube secrets dir
Apr 20, 2020
bc5e151
Include bcoin / relayer keep-test deploy
Apr 20, 2020
deaa64a
Fix serviceName for bcoin StatefulSets
Shadowfiend Apr 21, 2020
a5804d5
Merge pull request #586 from keep-network/rename-deposit-factory-getter
Shadowfiend Apr 21, 2020
0f585af
Merge pull request #585 from keep-network/a-bit-sale
Shadowfiend Apr 21, 2020
d10c78a
Merge branch 'master' into four-by-four-hundred
Shadowfiend Apr 21, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 92 additions & 19 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,47 @@ jobs:
command: |
cd /tmp/tbtc/contracts
gsutil -m cp * gs://${CONTRACT_DATA_BUCKET}/tbtc
build_relay_maintainer_initcontainer:
executor: docker-node
steps:
- setup_remote_docker:
docker_layer_caching: true
- checkout
- run:
name: Run Docker build
command: |
mkdir -p /tmp/relay-maintainer/docker-images
cd infrastructure/kube/templates/relay-maintainer/initcontainer
npm upgrade @keep-network/tbtc
docker build \
-t $GCR_REGISTRY_URL/$GOOGLE_PROJECT_ID/initcontainer-provision-relay-maintainer:$DOCKER_IMAGE_TAG .
- run:
name: Save initcontainer-provision-relay-maintainer image
command: |
docker save -o /tmp/relay-maintainer/docker-images/initcontainer-provision-relay-maintainer.tar $GCR_REGISTRY_URL/$GOOGLE_PROJECT_ID/initcontainer-provision-relay-maintainer:$DOCKER_IMAGE_TAG
- persist_to_workspace:
root: /tmp/relay-maintainer
paths:
- docker-images
publish_relay_maintainer_initcontainer:
executor: gcp-gcr/default
steps:
- attach_workspace:
at: /tmp/relay-maintainer
- run:
name: Load Docker images
command: |
docker load -i /tmp/relay-maintainer/docker-images/initcontainer-provision-relay-maintainer.tar
- gcp-gcr/gcr-auth:
google-project-id: GOOGLE_PROJECT_ID
google-compute-zone: GOOGLE_COMPUTE_ZONE_A
# This param doesn't actually set anything, leaving here as a reminder to check when they fix it.
gcloud-service-key: GCLOUD_SERVICE_KEY
- gcp-gcr/push-image:
google-project-id: GOOGLE_PROJECT_ID
registry-url: $GCR_REGISTRY_URL
image: initcontainer-provision-relay-maintainer
tag: $DOCKER_IMAGE_TAG

workflows:
version: 2
Expand Down Expand Up @@ -286,28 +327,42 @@ workflows:
- generate_docs_asciidoctor
migrate_build_publish_keep_dev:
jobs:
- compile_contracts
- migrate_contracts:
filters:
branches:
only: master
context: keep-dev
requires:
- compile_contracts
- publish_npm_package:
filters:
branches:
only: master
#- compile_contracts
#- migrate_contracts:
# # filters:
# # branches:
# # only: master
# context: keep-dev
# requires:
# - compile_contracts
#- publish_npm_package:
# # filters:
# # branches:
# # only: master
# context: keep-dev
# requires:
# - migrate_contracts
#- publish_contract_data:
# # filters:
# # branches:
# # only: master
# context: keep-dev
# requires:
# - migrate_contracts
- build_relay_maintainer_initcontainer:
# filters:
# branches:
# only: master
context: keep-dev
requires:
- migrate_contracts
- publish_contract_data:
filters:
branches:
only: master
#requires:
# - publish_npm_package
- publish_relay_maintainer_initcontainer:
# filters:
# branches:
# only: master
context: keep-dev
requires:
- migrate_contracts
- build_relay_maintainer_initcontainer
migrate_build_publish_keep_test:
jobs:
- keep_test_approval:
Expand Down Expand Up @@ -352,3 +407,21 @@ workflows:
context: keep-test
requires:
- migrate_contracts
- build_relay_maintainer_initcontainer:
filters:
tags:
only: /^v.*/
branches:
ignore: /.*/
context: keep-test
requires:
- publish_npm_package
- publish_relay_maintainer_initcontainer:
filters:
tags:
only: /^v.*/
branches:
ignore: /.*/
context: keep-test
requires:
- build_relay_maintainer_initcontainer
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,6 @@ node_modules/
.mypy_cache
# Visual Studio Code
*.code-workspace

# infrastructure
.secrets
20 changes: 20 additions & 0 deletions infrastructure/kube/keep-dev/bcoin-service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
apiVersion: v1
kind: Service
metadata:
name: bcoin
namespace: default
labels:
app: bitcoin-node
type: bcoin
spec:
ports:
- port: 18332
targetPort: 18332
name: testnet-rpc
- port: 18333
targetPort: 18333
name: testnet
selector:
app: bitcoin-node
type: bcoin
63 changes: 63 additions & 0 deletions infrastructure/kube/keep-dev/bcoin-statefulset.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: bcoin
namespace: default
labels:
app: bitcoin-node
type: bcoin
spec:
replicas: 1
serviceName: bcoin
volumeClaimTemplates:
- metadata:
name: bcoin-data
spec:
accessModes: [ReadWriteOnce]
resources:
requests:
storage: 40Gi
selector:
matchLabels:
app: bitcoin-node
type: bcoin
template:
metadata:
labels:
app: bitcoin-node
type: bcoin
spec:
containers:
- name: bcoin
image: gcr.io/keep-dev-fe24/bcoin
ports:
- containerPort: 18332
- containerPort: 18333
env:
- name: BCOIN_NETWORK
value: testnet
- name: BCOIN_HTTP_HOST
value: '0.0.0.0'
- name: BCOIN_HTTP_PORT
value: '18332'
- name: BCOIN_PORT
value: '18333'
- name: BCOIN_LOG_LEVEL
value: info
- name: BCOIN_PREFIX
value: /mnt/.bcoin
- name: BCOIN_API_KEY
valueFrom:
secretKeyRef:
name: bcoin
key: http-api-key
volumeMounts:
- name: bcoin-data
mountPath: /mnt/.bcoin
command:
["bcoin"]
volumes:
- name: bcoin-data
persistentVolumeClaim:
claimName: bcoin-data
110 changes: 110 additions & 0 deletions infrastructure/kube/keep-dev/relay-maintainer-deployment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
---
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put some space between a few values for readability while we work through this.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: relay-maintainer
namespace: default
labels:
keel.sh/policy: all
app: maintainer
type: relay
spec:
replicas: 1
selector:
matchLabels:
app: maintainer
type: relay
template:
metadata:
labels:
app: maintainer
type: relay
spec:

initContainers:
- name: initcontainer-provision-relay-maintainer
image: gcr.io/keep-dev-fe24/initcontainer-provision-relay-maintainer
imagePullPolicy: Always
env:

- name: ETH_RPC_URL
valueFrom:
configMapKeyRef:
name: eth-network-internal
key: rpc-url

- name: ETH_NETWORK_ID
valueFrom:
configMapKeyRef:
name: eth-network-internal
key: network-id

- name: ETH_NETWORK_NAME
value: internal

- name: RELAY_MAINTAINER_ETH_ACCOUNT_ADDRESS
valueFrom:
configMapKeyRef:
name: eth-account-info
key: account-0-address

- name: RELAY_MAINTAINER_ETH_ACCOUNT_PRIVATE_KEY
valueFrom:
secretKeyRef:
name: eth-account-privatekeys
key: account-0

- name: CONTRACT_OWNER_ETH_ACCOUNT_ADDRESS
valueFrom:
configMapKeyRef:
name: eth-network-internal
key: contract-owner-eth-account-address

- name: CONTRACT_OWNER_ETH_ACCOUNT_PRIVATE_KEY
valueFrom:
secretKeyRef:
name: eth-network-internal
key: contract-owner-eth-account-private-key

- name: BCOIN_HOST
value: bcoin.default.svc.cluster.local

- name: BCOIN_PORT
value: '18332'

- name: BCOIN_API_KEY
valueFrom:
secretKeyRef:
name: bcoin
key: http-api-key

volumeMounts:
- name: relay-maintainer-env
mountPath: /mnt/relay-maintainer
command: ['node', '/tmp/provision-relay-maintainer.js']
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Dockerfile already sets an appropriate entry point, yah?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I go back and forth on this, it feels good to be explicit on the app config with what command is being run for a container, even if it's the default entry point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this is probably true for external containers, since they can change out from under you, but for containers we control it feels like we're just doubling our workload whenever we need to change something… Either way, it's fine for this PR, especially if it's what we've been doing elsewhere.


containers:
- name: relay-maintainer
image: gcr.io/keep-dev-fe24/relay-maintainer
imagePullPolicy: Always
volumeMounts:
- name: relay-maintainer-env
mountPath: /mnt/relay-maintainer

volumes:
- name: relay-maintainer-env
persistentVolumeClaim:
claimName: relay-maintainer-env

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: relay-maintainer-env
namespace: default
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Mi
20 changes: 20 additions & 0 deletions infrastructure/kube/keep-test/bcoin-service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
apiVersion: v1
kind: Service
metadata:
name: bcoin
namespace: default
labels:
app: bitcoin-node
type: bcoin
spec:
ports:
- port: 18332
targetPort: 18332
name: testnet-rpc
- port: 18333
targetPort: 18333
name: testnet
selector:
app: bitcoin-node
type: bcoin
Loading