Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mojaloop/#2092): upgrade nodeJS version for core services #49

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
b48a4d4
feat(mojaloop/#2092): upgrade nodeJS version for core services
mdebarros Jul 12, 2022
d03eae3
chore: fixes for xunit and test-coverage
mdebarros Jul 13, 2022
9445aaf
added release and snapshot scripts to package.json
mdebarros Jul 13, 2022
8c00ce1
chore(snapshot): 12.0.0-snapshot.0
mdebarros Jul 13, 2022
f74fcf5
added release and snapshot scripts to package.json
mdebarros Jul 13, 2022
ee55f20
added release and snapshot scripts to package.json
mdebarros Jul 13, 2022
db8715a
chore(snapshot): 12.0.0-snapshot.1
mdebarros Jul 13, 2022
f31e51a
chore: fixes for upgraded ava and unit tests
mdebarros Jul 13, 2022
ef1e257
chore: lint fixes
mdebarros Jul 13, 2022
67a12b7
chore: fixes for code-coverage
mdebarros Jul 13, 2022
845384d
chore: removed ncruc dependencies + upgraded all @hapy/* depdenencies…
mdebarros Jul 13, 2022
6ffcc2d
chore(snapshot): 12.0.0-snapshot.2
mdebarros Jul 13, 2022
af96145
added missing license header from new js test/util helper
mdebarros Jul 13, 2022
2f168ac
chore: fixes for dockerfile
mdebarros Jul 13, 2022
d5309ca
chore(snapshot): 12.0.0-snapshot.3
mdebarros Jul 13, 2022
2a87e38
chore: removed npm pub jobs from circleci config
mdebarros Jul 13, 2022
1311409
chore(snapshot): 12.0.0-snapshot.4
mdebarros Jul 13, 2022
b8afea4
fixed missing publish job for docker-snapshots and added codeowners
mdebarros Jul 13, 2022
45357bb
chore(snapshot): 12.0.0-snapshot.5
mdebarros Jul 13, 2022
d371a14
updated dependencies
mdebarros Jul 18, 2022
ac55336
chore(snapshot): 12.0.0-snapshot.6
mdebarros Jul 18, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
635 changes: 510 additions & 125 deletions .circleci/config.yml

Large diffs are not rendered by default.

12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,15 @@ typings/

# VSCode directory
.vscode

# Ignore test results
**/test/results/*

# https://devspace.sh/
devspace*
.devspace/**.*

# Add ignores
*IGNORE*
*ignore*
!.npmignore
11 changes: 0 additions & 11 deletions .ncurc.json

This file was deleted.

5 changes: 5 additions & 0 deletions .ncurc.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
## Add a TODO comment indicating the reason for each rejected dependency upgrade added to this list, and what should be done to resolve it (i.e. handle it through a story, etc).
reject: [
# TODO: v6+ (ref: https://github.com/sindresorhus/get-port/releases/tag/v6.0.0) is an ESM library and thus not compatible with CommonJS. Future story needed to resolve.
"get-port"
]
3 changes: 2 additions & 1 deletion .npmignore
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,5 @@ Thumbs.db
.Spotlight-V100
.Trashes

test/temp
test/*
.circleci/*
1 change: 1 addition & 0 deletions .nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
16.15.0
5 changes: 5 additions & 0 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# These owners will be the default owners for everything in
# the repo. Unless a later match takes precedence,
# The below users will be requested for
# review when someone opens a pull request.
* @mdebarros @elnyry-sam-k @vijayg10
29 changes: 16 additions & 13 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,32 +1,35 @@
FROM node:12.16.1-alpine as builder
WORKDIR /opt/event-sidecar
FROM node:16.15.0-alpine as builder
WORKDIR /opt/app

RUN apk add --no-cache -t build-dependencies git make gcc g++ python libtool autoconf automake \
RUN apk --no-cache add git
RUN apk add --no-cache -t build-dependencies make gcc g++ python3 libtool libressl-dev openssl-dev autoconf automake \
&& cd $(npm root -g)/npm \
&& npm config set unsafe-perm true \
&& npm install -g node-gyp

COPY package.json package-lock.json* /opt/event-sidecar/
RUN npm install
COPY package.json package-lock.json* /opt/app/

RUN apk del build-dependencies
RUN npm ci --production

COPY config /opt/event-sidecar/config
COPY src /opt/event-sidecar/src

FROM node:12.16.1-alpine
WORKDIR /opt/event-sidecar
FROM node:16.15.0-alpine
WORKDIR /opt/app

# Create empty log file & link stdout to the application log file
RUN mkdir ./logs && touch ./logs/combined.log
RUN ln -sf /dev/stdout ./logs/combined.log

# Create a non-root user: ml-user
RUN adduser -D ml-user

# Copy over filesfrom builder
COPY --chown=ml-user --from=builder /opt/app .

# Set non-root user
USER ml-user

COPY --chown=ml-user --from=builder /opt/event-sidecar .
RUN npm prune --production
# Copy actual code
COPY config /opt/app/config
COPY src /opt/app/src

EXPOSE 4001
CMD ["npm", "run", "start"]
63 changes: 61 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,71 @@
# Event-Sidecar

[![Git Commit](https://img.shields.io/github/last-commit/mojaloop/event-sidecar.svg?style=flat)](https://github.com/mojaloop/event-sidecar/commits/master)
[![Git Releases](https://img.shields.io/github/release/mojaloop/event-sidecar.svg?style=flat)](https://github.com/mojaloop/event-sidecar/releases)
[![Docker pulls](https://img.shields.io/docker/pulls/mojaloop/event-sidecar.svg?style=flat)](https://hub.docker.com/r/mojaloop/event-sidecar)
[![CircleCI](https://circleci.com/gh/mojaloop/event-sidecar.svg?style=svg)](https://circleci.com/gh/mojaloop/event-sidecar)

Swagger api [location](src/interface/swagger.json)

Swagger api [src/interface/swagger.json](src/interface/swagger.json)

- The Event-Sidecar will publish events to a singular Kafka topic that will allow for multiple handlers to consume and process the events as required.
- Kafka partitions will be determined by the event-type (e.g. log, audit, trace, errors etc).
- Each Mojaloop component will have its own tightly coupled Sidecar.

## Auditing Dependencies

We use `npm-audit-resolver` along with `npm audit` to check dependencies for node vulnerabilities, and keep track of resolved dependencies with an `audit-resolve.json` file.

To start a new resolution process, run:

```bash
npm run audit:resolve
```

You can then check to see if the CI will pass based on the current dependencies with:

```bash
npm run audit:check
```

And commit the changed `audit-resolve.json` to ensure that CircleCI will build correctly.

## Container Scans

As part of our CI/CD process, we use anchore-cli to scan our built docker container for vulnerabilities upon release.

If you find your release builds are failing, refer to the [container scanning](https://github.com/mojaloop/ci-config#container-scanning) in our shared Mojaloop CI config repo. There is a good chance you simply need to update the `mojaloop-policy-generator.js` file and re-run the circleci workflow.

For more information on anchore and anchore-cli, refer to:

- [Anchore CLI](https://github.com/anchore/anchore-cli)
- [Circle Orb Registry](https://circleci.com/orbs/registry/orb/anchore/anchore-engine)

## Automated Releases

As part of our CI/CD process, we use a combination of CircleCI, standard-version
npm package and github-release CircleCI orb to automatically trigger our releases
and image builds. This process essentially mimics a manual tag and release.

On a merge to master, CircleCI is configured to use the mojaloopci github account
to push the latest generated CHANGELOG and package version number.

Once those changes are pushed, CircleCI will pull the updated master, tag and
push a release triggering another subsequent build that also publishes a docker image.

### Potential problems

- There is a case where the merge to master workflow will resolve successfully, triggering
a release. Then that tagged release workflow subsequently failing due to the image scan,
audit check, vulnerability check or other "live" checks.

This will leave master without an associated published build. Fixes that require
a new merge will essentially cause a skip in version number or require a clean up
of the master branch to the commit before the CHANGELOG and bump.

This may be resolved by relying solely on the previous checks of the
merge to master workflow to assume that our tagged release is of sound quality.
We are still mulling over this solution since catching bugs/vulnerabilities/etc earlier
is a boon.

- It is unknown if a race condition might occur with multiple merges with master in
quick succession, but this is a suspected edge case.
45 changes: 45 additions & 0 deletions audit-resolve.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,51 @@
"1670|hapi-swagger>handlebars": {
"decision": "fix",
"madeAt": 1624007811893
},
"1075703|@mojaloop/event-sdk>grpc>protobufjs": {
"decision": "ignore",
"madeAt": 1657643692131,
"expiresAt": 1660235687152
},
"1075704|@mojaloop/event-sdk>grpc>protobufjs": {
"decision": "ignore",
"madeAt": 1657643693651,
"expiresAt": 1660235687152
},
"1081008|@mojaloop/event-sdk>grpc>protobufjs>moment": {
"decision": "ignore",
"madeAt": 1657643694628,
"expiresAt": 1660235687152
},
"1081346|ansi-regex": {
"decision": "ignore",
"madeAt": 1657643695584,
"expiresAt": 1660235687152
},
"1067560|hapi-openapi>swagger-parser>z-schema>validator": {
"decision": "ignore",
"madeAt": 1657643696618,
"expiresAt": 1660235687152
},
"1067560|hapi-swagger>swagger-parser>z-schema>validator": {
"decision": "ignore",
"madeAt": 1657643696618,
"expiresAt": 1660235687152
},
"1069994|hapi-swagger>swagger-parser>z-schema>validator>swagger-ui-dist": {
"decision": "ignore",
"madeAt": 1657643697701,
"expiresAt": 1660235687152
},
"1070422|hapi-swagger>swagger-parser>z-schema>validator>swagger-ui-dist": {
"decision": "ignore",
"madeAt": 1657643698698,
"expiresAt": 1660235687152
},
"1070022|node-fetch": {
"decision": "ignore",
"madeAt": 1657643699756,
"expiresAt": 1660235687152
}
},
"rules": {},
Expand Down
Loading