Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Redis DB support for bal persist #4

Merged
merged 47 commits into from
Mar 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
91174ce
Add native project structure
dinukaamarasinghe817 Jan 23, 2024
0d867b7
Add native module info and Utils
dinukaamarasinghe817 Jan 24, 2024
59915fb
Add content for Utils
dinukaamarasinghe817 Jan 24, 2024
2c9de58
Add the query method template in RedisProcessor
dinukaamarasinghe817 Jan 24, 2024
98f6e64
Add build configs
dinukaamarasinghe817 Jan 24, 2024
30cd950
Add initial content in redis generic client
dinukaamarasinghe817 Jan 26, 2024
98325a6
Add init constructor in the RedisClient
dinukaamarasinghe817 Jan 26, 2024
2b8f131
Add runReadByKeyQuery method and helper methods
dinukaamarasinghe817 Jan 26, 2024
dc19ec5
Add runReadQuery method
dinukaamarasinghe817 Jan 26, 2024
8938cb0
Add runBatchInsertQuery method
dinukaamarasinghe817 Jan 26, 2024
c6c04ad
Add update and delete methods in redis client
dinukaamarasinghe817 Jan 26, 2024
be5091c
Add PersistRedisStream class
dinukaamarasinghe817 Jan 26, 2024
34a791a
Add native query method in redisProcessor
dinukaamarasinghe817 Jan 30, 2024
f6ac563
Add queryOne method in RedisProcessor
dinukaamarasinghe817 Jan 30, 2024
f26cb91
Add one to many relationship
dinukaamarasinghe817 Jan 31, 2024
02ddf2d
Add one to one relationship constraints
dinukaamarasinghe817 Feb 2, 2024
4609f79
Add license headers
dinukaamarasinghe817 Feb 6, 2024
c5e94de
Fix document formatting issues
dinukaamarasinghe817 Feb 6, 2024
ad1a133
Add time to supported data types
dinukaamarasinghe817 Feb 7, 2024
6aa1bc8
Add test cases
dinukaamarasinghe817 Feb 13, 2024
d0103bf
Add github workflow
dinukaamarasinghe817 Feb 20, 2024
a4eda29
Pull redis dependency on gradle build
dinukaamarasinghe817 Feb 21, 2024
0d2c03f
Add logging
dinukaamarasinghe817 Feb 27, 2024
3aa4745
Update license headers
dinukaamarasinghe817 Feb 29, 2024
c77e778
Remove extra colon from set elements
dinukaamarasinghe817 Feb 29, 2024
3b4ae2d
Migrate workflows to centralized workflows
dinukaamarasinghe817 Mar 1, 2024
b44b9a0
Address sugesstions
dinukaamarasinghe817 Mar 5, 2024
6892e34
Merge pull request #7 from dinukaamarasinghe817/test-cases
dinukaamarasinghe817 Mar 5, 2024
9fc6284
Update PR workflow
dinukaamarasinghe817 Mar 5, 2024
e203342
Fix PR workflow
dinukaamarasinghe817 Mar 5, 2024
f1ee4ef
Merge pull request #8 from dinukaamarasinghe817/test-cases
dinukaamarasinghe817 Mar 5, 2024
23f9eeb
Fix formatting issues
dinukaamarasinghe817 Mar 6, 2024
0c1dc8d
Merge pull request #9 from dinukaamarasinghe817/test-cases
dinukaamarasinghe817 Mar 6, 2024
bce6193
Fix issues with metadata visibility
dinukaamarasinghe817 Mar 7, 2024
3882e93
Merge pull request #10 from dinukaamarasinghe817/test-cases
dinukaamarasinghe817 Mar 7, 2024
d6ead18
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Mar 9, 2024
e057f3f
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Mar 9, 2024
7342a0e
Add connection URI option in Redis configuration
dinukaamarasinghe817 Mar 9, 2024
89957b1
[Automated] Update native jar versions in toml files
dinukaamarasinghe817 Mar 11, 2024
ffcc09f
Update configs to use Redis connector v3
dinukaamarasinghe817 Mar 11, 2024
1f4179e
Merge pull request #11 from dinukaamarasinghe817/connection-uri
dinukaamarasinghe817 Mar 11, 2024
f995aea
Update git ignore to omit IDEA generated files
dinukaamarasinghe817 Mar 11, 2024
8ab4b77
Merge pull request #12 from dinukaamarasinghe817/connection-uri
dinukaamarasinghe817 Mar 11, 2024
35fdebb
Update git ignore to omit IDEA generated files
dinukaamarasinghe817 Mar 11, 2024
b910285
Merge pull request #13 from dinukaamarasinghe817/connection-uri
dinukaamarasinghe817 Mar 11, 2024
514020a
Add a new line in Config.toml file
dinukaamarasinghe817 Mar 11, 2024
47e4945
Merge pull request #14 from dinukaamarasinghe817/connection-uri
dinukaamarasinghe817 Mar 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#
# https://help.github.com/articles/dealing-with-line-endings/
#
# Linux start script should use lf
/gradlew text eol=lf

# These are Windows script files and should use crlf
*.bat text eol=crlf

# These are Windows java files and should use lf
*.java text eol=lf
12 changes: 12 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
## Purpose

Fixes:

## Examples

## Checklist
- [ ] Linked to an issue
- [ ] Updated the specification
- [ ] Updated the changelog
- [ ] Added tests
- [ ] Checked native-image compatibility
18 changes: 18 additions & 0 deletions .github/workflows/build-timestamped-master.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Build

on:
push:
branches:
- main
paths-ignore:
- '*.md'
- 'docs/**'

workflow_dispatch:

jobs:
call_workflow:
name: Run Build Workflow
if: ${{ github.repository_owner == 'ballerina-platform' }}
uses: ballerina-platform/ballerina-library/.github/workflows/build-timestamp-master-template.yml@main
secrets: inherit
41 changes: 41 additions & 0 deletions .github/workflows/build-with-bal-test-graalvm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: GraalVM Check

on:
workflow_dispatch:
inputs:
lang_tag:
description: Branch/Release Tag of the Ballerina Lang
required: true
default: master
lang_version:
description: Ballerina Lang Version (If given ballerina lang buid will be skipped)
required: false
default: ''
native_image_options:
description: Default native-image options
required: false
default: ''
schedule:
- cron: '30 18 * * *'
pull_request:
branches:
- main
types: [ opened, synchronize, reopened, labeled, unlabeled ]

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true

jobs:
call_stdlib_workflow:
name: Run StdLib Workflow
if: ${{ github.event_name != 'schedule' || (github.event_name == 'schedule' && github.repository_owner == 'ballerina-platform') }}
uses: ballerina-platform/ballerina-library/.github/workflows/build-with-bal-test-graalvm-template.yml@main
with:
lang_tag: ${{ inputs.lang_tag }}
lang_version: ${{ inputs.lang_version }}
native_image_options: '-J-Xmx7G ${{ inputs.native_image_options }}'
additional_windows_build_flags: '-x test'
steps:
- name: Give execute permission to gradlew
run: chmod +x ./gradlew
21 changes: 21 additions & 0 deletions .github/workflows/central-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: Publish to the Ballerina central

on:
workflow_dispatch:
inputs:
environment:
type: choice
description: Select Environment
required: true
options:
- DEV CENTRAL
- STAGE CENTRAL

jobs:
call_workflow:
name: Run Central Publish Workflow
if: ${{ github.repository_owner == 'ballerina-platform' }}
uses: ballerina-platform/ballerina-library/.github/workflows/central-publish-template.yml@main
secrets: inherit
with:
environment: ${{ github.event.inputs.environment }}
78 changes: 78 additions & 0 deletions .github/workflows/publish-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: Publish release

on:
workflow_dispatch:
repository_dispatch:
types: [ stdlib-release-pipeline ]

jobs:
publish-release:
runs-on: ubuntu-latest
if: github.repository_owner == 'ballerina-platform'
steps:
- uses: actions/checkout@v2
- name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
java-version: 17.0.7
- name: Give execute permission to gradlew
run: chmod +x ./gradlew
- name: Build with Gradle
env:
packageUser: ${{ github.actor }}
packagePAT: ${{ secrets.GITHUB_TOKEN }}
run: |
git config --global user.name ${{ secrets.BALLERINA_BOT_USERNAME }}
git config --global user.email ${{ secrets.BALLERINA_BOT_EMAIL }}
./gradlew build -x check -x test
- name: Create lib directory if not exists
run: mkdir -p ballerina/lib
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'rootfs'
scan-ref: '/github/workspace/ballerina/lib'
format: 'table'
timeout: '10m0s'
exit-code: '1'
- name: Set version env variable
run: echo "VERSION=$((grep -w 'version' | cut -d= -f2) < gradle.properties | rev | cut --complement -d- -f1 | rev)" >> $GITHUB_ENV
- name: Pre release dependency version update
env:
GITHUB_TOKEN: ${{ secrets.BALLERINA_BOT_TOKEN }}
run: |
echo "Version: ${VERSION}"
git checkout -b release-${VERSION}
sed -i 's/ballerinaLangVersion=\(.*\)-SNAPSHOT/ballerinaLangVersion=\1/g' gradle.properties
sed -i 's/ballerinaLangVersion=\(.*\)-[0-9]\{8\}-[0-9]\{6\}-.*$/ballerinaLangVersion=\1/g' gradle.properties
sed -i 's/stdlib\(.*\)=\(.*\)-SNAPSHOT/stdlib\1=\2/g' gradle.properties
sed -i 's/stdlib\(.*\)=\(.*\)-[0-9]\{8\}-[0-9]\{6\}-.*$/stdlib\1=\2/g' gradle.properties
sed -i 's/observe\(.*\)=\(.*\)-SNAPSHOT/observe\1=\2/g' gradle.properties
sed -i 's/observe\(.*\)=\(.*\)-[0-9]\{8\}-[0-9]\{6\}-.*$/observe\1=\2/g' gradle.properties
git add gradle.properties
git commit -m "Move dependencies to stable version" || echo "No changes to commit"
- name: Grant execute permission for gradlew
run: chmod +x gradlew
- name: Publish artifact
env:
BALLERINA_CENTRAL_ACCESS_TOKEN: ${{ secrets.BALLERINA_CENTRAL_ACCESS_TOKEN }}
GITHUB_TOKEN: ${{ secrets.BALLERINA_BOT_TOKEN }}
packageUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
packagePAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
publishUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
publishPAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
nexusUser: ${{ secrets.NEXUS_USERNAME }}
nexusPassword: ${{ secrets.NEXUS_PASSWORD }}
CLIENT_ID: ${{ secrets.CLIENT_ID }}
CLIENT_SECRET: ${{ secrets.CLIENT_SECRET }}
REFRESH_TOKEN: ${{ secrets.REFRESH_TOKEN }}
run: |
./gradlew clean release -Prelease.useAutomaticVersion=true
./gradlew -Pversion=${VERSION} publish -x test -PpublishToCentral=true
- name: GitHub Release and Release Sync PR
env:
GITHUB_TOKEN: ${{ secrets.BALLERINA_BOT_TOKEN }}
run: |
gh release create v$VERSION --title "module-ballerinax-persist.redis-v$VERSION"
gh pr create --title "[Automated] Sync main after $VERSION release" --body "Sync main after $VERSION release"
29 changes: 29 additions & 0 deletions .github/workflows/publish-snapshot-nexus.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Publish Snapshot to Nexus

on:
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest
if: github.repository_owner == 'ballerina-platform'
steps:
- uses: actions/checkout@v2
- name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
java-version: 17.0.7
- name: Give execute permission to gradlew
run: chmod +x ./gradlew
- name: Build with Gradle
env:
packageUser: ${{ secrets.BALLERINA_BOT_USERNAME }}
packagePAT: ${{ secrets.BALLERINA_BOT_TOKEN }}
nexusUser: ${{ secrets.NEXUS_USERNAME }}
nexusPassword: ${{ secrets.NEXUS_PASSWORD }}
CLIENT_ID: ${{ secrets.CLIENT_ID }}
CLIENT_SECRET: ${{ secrets.CLIENT_SECRET }}
REFRESH_TOKEN: ${{ secrets.REFRESH_TOKEN }}
run: |
./gradlew build publishMavenJavaPublicationToWSO2NexusRepository --scan --no-daemon
17 changes: 17 additions & 0 deletions .github/workflows/pull-request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: PR Build

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true

on:
pull_request:
types: [opened, reopened, synchronize]

jobs:
call_workflow:
name: Run PR Build Workflow
if: ${{ github.repository_owner == 'ballerina-platform' }}
uses: ballerina-platform/ballerina-library/.github/workflows/pull-request-build-template.yml@main
with:
additional-windows-test-flags: "-x test"
19 changes: 19 additions & 0 deletions .github/workflows/stale_check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: 'Close stale pull requests'

on:
schedule:
- cron: '30 19 * * *'
workflow_dispatch:

jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v3
with:
stale-pr-message: 'This PR has been open for more than 15 days with no activity. This will be closed in 3 days unless the `stale` label is removed or commented.'
close-pr-message: 'Closed PR due to inactivity for more than 18 days.'
days-before-pr-stale: 15
days-before-pr-close: 3
days-before-issue-stale: -1
days-before-issue-close: -1
13 changes: 13 additions & 0 deletions .github/workflows/trivy-scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: Trivy

on:
workflow_dispatch:
schedule:
- cron: "30 20 * * *"

jobs:
call_workflow:
name: Run Trivy Scan Workflow
if: ${{ github.repository_owner == 'ballerina-platform' }}
uses: ballerina-platform/ballerina-library/.github/workflows/trivy-scan-template.yml@main
secrets: inherit
26 changes: 26 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,29 @@
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*
replay_pid*

# Ballerina
Ballerina.lock

# VS code files
.vscode

# IDEA code files
.idea

# Generated files
target
.ballerina
ballerina/target
ballerina/.devcontainer.json
ballerina/build

# gradle
.gradle
build/
gradle-app.setting
!gradle-wrapper.jar
.gradletasknamecache

# Ignore Gradle build output directory
build
4 changes: 4 additions & 0 deletions ballerina/.devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"image": "ballerina/ballerina-devcontainer:2201.8.2",
"extensions": ["WSO2.ballerina"],
}
24 changes: 24 additions & 0 deletions ballerina/Ballerina.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
[package]
org = "ballerinax"
name = "persist.redis"
version = "0.1.0"
authors = ["Ballerina"]
keywords = ["persist", "redis", "experimental"]
repository = "https://github.com/ballerina-platform/module-ballerinax-persist.redis"
license = ["Apache-2.0"]
distribution = "2201.9.0"

[platform.java17]
graalvmCompatible = true

[[platform.java17.dependency]]
groupId = "io.ballerina.persist"
artifactId = "persist-redis-native"
version = "0.1.0"
path = "../native/build/libs/persist.redis-native-0.1.0-SNAPSHOT.jar"

[[platform.java17.dependency]]
groupId = "io.ballerina.stdlib"
artifactId = "persist-native"
version = "1.2.0"
path = "./lib/persist-native-1.2.0.jar"
Loading
Loading