Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing Redis DB support for ballerina persist feature #2

Closed
wants to merge 71 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
5fae984
basic native project structure
dinukaamarasinghe817 Jan 23, 2024
d7b2457
added native module-info.java, Utils.java and Constants.java files
dinukaamarasinghe817 Jan 24, 2024
f6267e3
added content for Utils.java file
dinukaamarasinghe817 Jan 24, 2024
f2a54f3
added the initial query method in RedisProcessor.java
dinukaamarasinghe817 Jan 24, 2024
8fe49ec
added build-config basic content
dinukaamarasinghe817 Jan 24, 2024
fcca68b
added initial files and content for the redis generic client library …
dinukaamarasinghe817 Jan 26, 2024
d279676
added init constructor for the RedisClient
dinukaamarasinghe817 Jan 26, 2024
e83c7fb
added the runReadByKeyQuery method and it's helper methods
dinukaamarasinghe817 Jan 26, 2024
d41297c
added the runReadQuery for batch read
dinukaamarasinghe817 Jan 26, 2024
8753d36
added the runBatchInsertQuery for batch reads
dinukaamarasinghe817 Jan 26, 2024
1aead8d
added update and delete methods for the redis client
dinukaamarasinghe817 Jan 26, 2024
db52ca0
added getKeyFields public method and PersistRedisStream class
dinukaamarasinghe817 Jan 26, 2024
8ca4a58
added and tested native query method in redisProcessor for entities w…
dinukaamarasinghe817 Jan 30, 2024
4f5dce2
added queryOne method in RedisProcessor for records without associations
dinukaamarasinghe817 Jan 30, 2024
456e26d
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Jan 31, 2024
4c7c00b
bug fixes in one to many relationships in getManyRelations method
dinukaamarasinghe817 Jan 31, 2024
379cdf9
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Jan 31, 2024
b2edbee
added one to many relationship
dinukaamarasinghe817 Jan 31, 2024
8325105
added minor changes to redis_client.bal
dinukaamarasinghe817 Feb 1, 2024
769a2cf
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 1, 2024
80c373c
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 1, 2024
b138268
updated constraint fail and already exist error types
dinukaamarasinghe817 Feb 1, 2024
d85d7f6
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 2, 2024
94f42bf
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 2, 2024
9edb544
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 2, 2024
48156e9
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 2, 2024
0ba124d
added one to one relationship constraints on non owner of the relatio…
dinukaamarasinghe817 Feb 2, 2024
c045571
Merge pull request #1 from dinukaamarasinghe817/one-to-one-relation-c…
dinukaamarasinghe817 Feb 2, 2024
637e8c4
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 6, 2024
68a6502
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 6, 2024
fe0d235
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 6, 2024
39a7dbd
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 6, 2024
3a7997f
fixed the issues related to licensing and code formatting
dinukaamarasinghe817 Feb 6, 2024
c713119
added document formatting
dinukaamarasinghe817 Feb 6, 2024
307b626
Merge pull request #2 from dinukaamarasinghe817/one-to-one-relation-c…
dinukaamarasinghe817 Feb 6, 2024
b736764
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 7, 2024
c3ed704
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 8, 2024
781fc17
added time:Date and time:TimeOfDay data types as supported datatypes
dinukaamarasinghe817 Feb 8, 2024
4a402b3
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 11, 2024
0ced4d5
added test cases
dinukaamarasinghe817 Feb 13, 2024
8d57aa5
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 14, 2024
96b3fd6
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 14, 2024
bfda1aa
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 14, 2024
158930c
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 14, 2024
3f8d177
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 14, 2024
bb5c31e
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 15, 2024
35e9697
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 15, 2024
def18eb
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 15, 2024
18b5fcf
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 16, 2024
9856cb9
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 18, 2024
bb72b13
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 18, 2024
b83125e
completed integrating testing
dinukaamarasinghe817 Feb 18, 2024
db16d1b
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 19, 2024
9d7a36c
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Feb 19, 2024
ac71fb0
fixed a bug in runUpdateQuery method
dinukaamarasinghe817 Feb 19, 2024
3dcaa98
Merge pull request #3 from dinukaamarasinghe817/test-cases
dinukaamarasinghe817 Feb 19, 2024
1d0ad5c
added github workflow
dinukaamarasinghe817 Feb 20, 2024
45dbbb4
Merge pull request #4 from dinukaamarasinghe817/test-cases
dinukaamarasinghe817 Feb 20, 2024
2c0ba0f
un comment the release task in build.gradle
dinukaamarasinghe817 Feb 20, 2024
3ed4a1b
changed end of line to lf for java files
dinukaamarasinghe817 Feb 21, 2024
be8b87b
added dependencies in build.gradle
dinukaamarasinghe817 Feb 21, 2024
b5af2d1
added execution permission for gradle build in ubuntu
dinukaamarasinghe817 Feb 21, 2024
230a521
removed an unnecessary empty dependency
dinukaamarasinghe817 Feb 21, 2024
4e69c07
pull redis dependency on gradle build
dinukaamarasinghe817 Feb 21, 2024
0b33d64
make build depend on pulling redis
dinukaamarasinghe817 Feb 21, 2024
ca0722c
pull ballerinax/redis v.2.5.1 on build
dinukaamarasinghe817 Feb 21, 2024
74cdbd8
added execution permission for graalVM
dinukaamarasinghe817 Feb 21, 2024
42d97ff
added read permission to config.toml
dinukaamarasinghe817 Feb 21, 2024
8d36248
removed Config.toml file from gitignore
dinukaamarasinghe817 Feb 21, 2024
e07b64b
Merge pull request #5 from dinukaamarasinghe817/test-cases
dinukaamarasinghe817 Feb 21, 2024
3ded106
Merge pull request #1 from dinukaamarasinghe817/main
daneshk Feb 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#
# https://help.github.com/articles/dealing-with-line-endings/
#
# Linux start script should use lf
/gradlew text eol=lf

# These are Windows script files and should use crlf
*.bat text eol=crlf

# These are Windows java files and should use lf
*.java text eol=lf
12 changes: 12 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
## Purpose

Fixes:

## Examples

## Checklist
- [ ] Linked to an issue
- [ ] Updated the specification
- [ ] Updated the changelog
- [ ] Added tests
- [ ] Checked native-image compatibility
66 changes: 66 additions & 0 deletions .github/workflows/build-timestamped-master.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
name: Build

on:
workflow_dispatch:
push:
branches:
- main
- 2201.[0-9]+.x
paths-ignore:
- 'load-tests/**'
- '*.md'
- 'docs/**'

jobs:
build:
runs-on: ubuntu-latest
if: github.repository_owner == 'ballerina-platform'
steps:
- uses: actions/checkout@v2
- name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
java-version: 17.0.7
- name: Change to Timestamped Version
run: |
initialVersion=$((grep -w 'version' | cut -d= -f2) < gradle.properties )
echo "Initial_Version=$initialVersion" >> $GITHUB_ENV
startTime=$(TZ="Asia/Kolkata" date +'%Y%m%d-%H%M00')
latestCommit=$(git log -n 1 --pretty=format:"%h")
VERSION=$((grep -w 'version' | cut -d= -f2) < gradle.properties | rev | cut --complement -d- -f1 | rev)
updatedVersion=$VERSION-$startTime-$latestCommit
echo $updatedVersion
sed -i "s/version=\(.*\)/version=$updatedVersion/g" gradle.properties
- name: Give execute permission to gradlew
run: chmod +x ./gradlew
- name: Build with Gradle
env:
packageUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
packagePAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
publishUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
publishPAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
CLIENT_ID: ${{ secrets.CLIENT_ID }}
CLIENT_SECRET: ${{ secrets.CLIENT_SECRET }}
REFRESH_TOKEN: ${{ secrets.REFRESH_TOKEN }}
run: |
./gradlew clean build publishAllPublicationsToGitHubPackagesRepository --scan --no-daemon
- name: Generate CodeCov Report
uses: codecov/codecov-action@v2
- name: Upload Artifact
uses: actions/upload-artifact@v2
with:
name: ballerina-runtime
path: target/ballerina-runtime/
- name: Revert to SNAPSHOT Version
run: |
echo "version=${{ env.Initial_Version }}"
sed -i "s/version=\(.*\)/version=${{ env.Initial_Version }}/g" gradle.properties
- name: Publish to Nexus
env:
packageUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
packagePAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
nexusUser: ${{ secrets.NEXUS_USERNAME }}
nexusPassword: ${{ secrets.NEXUS_PASSWORD }}
run: |
./gradlew clean publishMavenJavaPublicationToWSO2NexusRepository -x test --scan --no-daemon
41 changes: 41 additions & 0 deletions .github/workflows/build-with-bal-test-graalvm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: GraalVM Check

on:
workflow_dispatch:
inputs:
lang_tag:
description: Branch/Release Tag of the Ballerina Lang
required: true
default: master
lang_version:
description: Ballerina Lang Version (If given ballerina lang buid will be skipped)
required: false
default: ''
native_image_options:
description: Default native-image options
required: false
default: ''
schedule:
- cron: '30 18 * * *'
pull_request:
branches:
- main
types: [ opened, synchronize, reopened, labeled, unlabeled ]

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true

jobs:
call_stdlib_workflow:
name: Run StdLib Workflow
if: ${{ github.event_name != 'schedule' || (github.event_name == 'schedule' && github.repository_owner == 'ballerina-platform') }}
uses: ballerina-platform/ballerina-standard-library/.github/workflows/build-with-bal-test-graalvm-template.yml@main
with:
lang_tag: ${{ inputs.lang_tag }}
lang_version: ${{ inputs.lang_version }}
native_image_options: '-J-Xmx7G ${{ inputs.native_image_options }}'
additional_windows_build_flags: '-x test'
steps:
- name: Give execute permission to gradlew
run: chmod +x ./gradlew
89 changes: 89 additions & 0 deletions .github/workflows/central-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
name: Publish to the Ballerina central

on:
workflow_dispatch:
inputs:
environment:
type: choice
description: Select environment
required: true
options:
- CENTRAL
- DEV CENTRAL
- STAGE CENTRAL

jobs:
publish-release:
runs-on: ubuntu-latest
if: github.repository_owner == 'ballerina-platform'
steps:
- uses: actions/checkout@v2
- name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
java-version: 17.0.7
- name: Give execute permission to gradlew
run: chmod +x ./gradlew
- name: Build with Gradle
env:
packageUser: ${{ github.actor }}
packagePAT: ${{ secrets.GITHUB_TOKEN }}
run: ./gradlew build -x check -x test
- name: Create lib directory if not exists
run: mkdir -p ballerina/lib
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'rootfs'
scan-ref: '/github/workspace/ballerina/lib'
format: 'table'
timeout: '10m0s'
exit-code: '1'

- name: Ballerina Central Push
if: ${{ github.event.inputs.environment == 'CENTRAL' }}
env:
BALLERINA_DEV_CENTRAL: false
BALLERINA_STAGE_CENTRAL: false
BALLERINA_CENTRAL_ACCESS_TOKEN: ${{ secrets.BALLERINA_CENTRAL_ACCESS_TOKEN }}
packageUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
packagePAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
GITHUB_TOKEN: ${{ secrets.BALLERINA_BOT_TOKEN }}
CLIENT_ID: ${{ secrets.CLIENT_ID }}
CLIENT_SECRET: ${{ secrets.CLIENT_SECRET }}
REFRESH_TOKEN: ${{ secrets.REFRESH_TOKEN }}
run: |
./gradlew clean build -PpublishToCentral=true

- name: Ballerina Central Dev Push
if: ${{ github.event.inputs.environment == 'DEV CENTRAL' }}
env:
BALLERINA_DEV_CENTRAL: true
BALLERINA_STAGE_CENTRAL: false
BALLERINA_CENTRAL_ACCESS_TOKEN: ${{ secrets.BALLERINA_CENTRAL_DEV_ACCESS_TOKEN }}
packageUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
packagePAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
GITHUB_TOKEN: ${{ secrets.BALLERINA_BOT_TOKEN }}
CLIENT_ID: ${{ secrets.CLIENT_ID }}
CLIENT_SECRET: ${{ secrets.CLIENT_SECRET }}
REFRESH_TOKEN: ${{ secrets.REFRESH_TOKEN }}
run: |
sed -i 's/version=\(.*\)-SNAPSHOT/version=\1/g' gradle.properties
./gradlew clean build -PpublishToCentral=true

- name: Ballerina Central Stage Push
if: ${{ github.event.inputs.environment == 'STAGE CENTRAL' }}
env:
BALLERINA_DEV_CENTRAL: false
BALLERINA_STAGE_CENTRAL: true
BALLERINA_CENTRAL_ACCESS_TOKEN: ${{ secrets.BALLERINA_CENTRAL_STAGE_ACCESS_TOKEN }}
packageUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
packagePAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
GITHUB_TOKEN: ${{ secrets.BALLERINA_BOT_TOKEN }}
CLIENT_ID: ${{ secrets.CLIENT_ID }}
CLIENT_SECRET: ${{ secrets.CLIENT_SECRET }}
REFRESH_TOKEN: ${{ secrets.REFRESH_TOKEN }}
run: |
sed -i 's/version=\(.*\)-SNAPSHOT/version=\1/g' gradle.properties
./gradlew clean build -PpublishToCentral=true
78 changes: 78 additions & 0 deletions .github/workflows/publish-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: Publish release

on:
workflow_dispatch:
repository_dispatch:
types: [ stdlib-release-pipeline ]

jobs:
publish-release:
runs-on: ubuntu-latest
if: github.repository_owner == 'ballerina-platform'
steps:
- uses: actions/checkout@v2
- name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
java-version: 17.0.7
- name: Give execute permission to gradlew
run: chmod +x ./gradlew
- name: Build with Gradle
env:
packageUser: ${{ github.actor }}
packagePAT: ${{ secrets.GITHUB_TOKEN }}
run: |
git config --global user.name ${{ secrets.BALLERINA_BOT_USERNAME }}
git config --global user.email ${{ secrets.BALLERINA_BOT_EMAIL }}
./gradlew build -x check -x test
- name: Create lib directory if not exists
run: mkdir -p ballerina/lib
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'rootfs'
scan-ref: '/github/workspace/ballerina/lib'
format: 'table'
timeout: '10m0s'
exit-code: '1'
- name: Set version env variable
run: echo "VERSION=$((grep -w 'version' | cut -d= -f2) < gradle.properties | rev | cut --complement -d- -f1 | rev)" >> $GITHUB_ENV
- name: Pre release dependency version update
env:
GITHUB_TOKEN: ${{ secrets.BALLERINA_BOT_TOKEN }}
run: |
echo "Version: ${VERSION}"
git checkout -b release-${VERSION}
sed -i 's/ballerinaLangVersion=\(.*\)-SNAPSHOT/ballerinaLangVersion=\1/g' gradle.properties
sed -i 's/ballerinaLangVersion=\(.*\)-[0-9]\{8\}-[0-9]\{6\}-.*$/ballerinaLangVersion=\1/g' gradle.properties
sed -i 's/stdlib\(.*\)=\(.*\)-SNAPSHOT/stdlib\1=\2/g' gradle.properties
sed -i 's/stdlib\(.*\)=\(.*\)-[0-9]\{8\}-[0-9]\{6\}-.*$/stdlib\1=\2/g' gradle.properties
sed -i 's/observe\(.*\)=\(.*\)-SNAPSHOT/observe\1=\2/g' gradle.properties
sed -i 's/observe\(.*\)=\(.*\)-[0-9]\{8\}-[0-9]\{6\}-.*$/observe\1=\2/g' gradle.properties
git add gradle.properties
git commit -m "Move dependencies to stable version" || echo "No changes to commit"
- name: Grant execute permission for gradlew
run: chmod +x gradlew
- name: Publish artifact
env:
BALLERINA_CENTRAL_ACCESS_TOKEN: ${{ secrets.BALLERINA_CENTRAL_ACCESS_TOKEN }}
GITHUB_TOKEN: ${{ secrets.BALLERINA_BOT_TOKEN }}
packageUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
packagePAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
publishUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
publishPAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
nexusUser: ${{ secrets.NEXUS_USERNAME }}
nexusPassword: ${{ secrets.NEXUS_PASSWORD }}
CLIENT_ID: ${{ secrets.CLIENT_ID }}
CLIENT_SECRET: ${{ secrets.CLIENT_SECRET }}
REFRESH_TOKEN: ${{ secrets.REFRESH_TOKEN }}
run: |
./gradlew clean release -Prelease.useAutomaticVersion=true
./gradlew -Pversion=${VERSION} publish -x test -PpublishToCentral=true
- name: GitHub Release and Release Sync PR
env:
GITHUB_TOKEN: ${{ secrets.BALLERINA_BOT_TOKEN }}
run: |
gh release create v$VERSION --title "module-ballerinax-persist.redis-v$VERSION"
gh pr create --title "[Automated] Sync main after $VERSION release" --body "Sync main after $VERSION release"
29 changes: 29 additions & 0 deletions .github/workflows/publish-snapshot-nexus.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Publish Snapshot to Nexus

on:
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest
if: github.repository_owner == 'ballerina-platform'
steps:
- uses: actions/checkout@v2
- name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
java-version: 17.0.7
- name: Give execute permission to gradlew
run: chmod +x ./gradlew
- name: Build with Gradle
env:
packageUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
packagePAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
nexusUser: ${{ secrets.NEXUS_USERNAME }}
nexusPassword: ${{ secrets.NEXUS_PASSWORD }}
CLIENT_ID: ${{ secrets.CLIENT_ID }}
CLIENT_SECRET: ${{ secrets.CLIENT_SECRET }}
REFRESH_TOKEN: ${{ secrets.REFRESH_TOKEN }}
run: |
./gradlew build publishMavenJavaPublicationToWSO2NexusRepository --scan --no-daemon
50 changes: 50 additions & 0 deletions .github/workflows/pull-request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: PR build

on:
pull_request:
branches:
- main
- 2201.[0-9]+.x

jobs:
ubuntu-build:
name: Build on Ubuntu
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
java-version: 17.0.7
- name: Give execute permission to gradlew
run: chmod +x ./gradlew
- name: Build with Gradle
env:
packageUser: ${{ github.actor }}
packagePAT: ${{ secrets.GITHUB_TOKEN }}
CLIENT_ID: ${{ secrets.CLIENT_ID }}
CLIENT_SECRET: ${{ secrets.CLIENT_SECRET }}
REFRESH_TOKEN: ${{ secrets.REFRESH_TOKEN }}
run: ./gradlew build
- name: Generate Codecov Report
uses: codecov/codecov-action@v2
with:
token: ${{ secrets.CODECOV_TOKEN }}

windows-build:
name: Build on Windows
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
java-version: 17.0.7
- name: Build with Gradle
env:
packageUser: ${{ github.actor }}
packagePAT: ${{ secrets.GITHUB_TOKEN }}
run: ./gradlew.bat build -x test
# Disabling tests because no docker in git-action windows
Loading
Loading