Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Redis DB support for bal persist #4

Merged
merged 47 commits into from
Mar 11, 2024

Conversation

dinukaamarasinghe817
Copy link
Contributor

@dinukaamarasinghe817 dinukaamarasinghe817 commented Mar 1, 2024

Purpose

To make data persistence easier for bal persist with Redis DB.
Addressing : https://github.com/ballerina-platform/persist-tools/issues/328

Goals

To support Redis as a database for the bal persist feature which allows users to manage data persistence without writing database queries inside the code. We need to generate the persistence client code for Redis when the ‘bal persist generate’ command gets executed.

Architecture

Persist Redis - Design-High Level Architecture

Approach

Implements a generic client for Redis which contains resource methods to get, insert, update and delete records. This utilizes main data types supported by Redis which are Hashes and Sets. This also includes a Java native processor to support dependent types in get resource methods.

Documentation

Design Documentation : Link

Related PRs

Related to : #3
Related to : #2

Future Improvements

Revisit after : ballerina-platform/ballerina-library#6136
Revisit after : ballerina-platform/ballerina-library#6137

Copy link

codecov bot commented Mar 1, 2024

Codecov Report

Attention: Patch coverage is 86.59476% with 87 lines in your changes are missing coverage. Please review.

❗ No coverage uploaded for pull request base (main@3605494). Click here to learn what that means.

Files Patch % Lines
ballerina/redis_client.bal 88.07% 62 Missing ⚠️
ballerina/stream_types.bal 71.42% 12 Missing ⚠️
...stdlib/persist/redis/datastore/RedisProcessor.java 83.33% 6 Missing and 3 partials ⚠️
.../java/io/ballerina/stdlib/persist/redis/Utils.java 80.00% 4 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main       #4   +/-   ##
=======================================
  Coverage        ?   86.59%           
  Complexity      ?        9           
=======================================
  Files           ?        7           
  Lines           ?      649           
  Branches        ?      381           
=======================================
  Hits            ?      562           
  Misses          ?       84           
  Partials        ?        3           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

name: Run Build Workflow
if: ${{ github.repository_owner == 'ballerina-platform' }}
uses: ballerina-platform/ballerina-library/.github/workflows/build-timestamp-master-template.yml@main
secrets: inherit
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
secrets: inherit
secrets: inherit

name: Run Trivy Scan Workflow
if: ${{ github.repository_owner == 'ballerina-platform' }}
uses: ballerina-platform/ballerina-library/.github/workflows/trivy-scan-template.yml@main
secrets: inherit
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
secrets: inherit
secrets: inherit

@@ -0,0 +1,50 @@
# Module Overview
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Module Overview
# Overview

import ballerina/test;
import ballerina/persist;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
import ballerina/test;
import ballerina/persist;
import ballerina/persist;
import ballerina/test;

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check for other places

ballerina/tests/rainier_generated_types.bal Show resolved Hide resolved
ballerina/stream_types.bal Outdated Show resolved Hide resolved
private typedesc<record {}> targetType;
private map<anydata> typeMap;

public isolated function init(stream<record {}, error?>? anydataStream, typedesc<record {}> targetType, map<anydata> typeMap, string[] fields, string[] include, any[] typeDescriptions, RedisClient persistClient, persist:Error? err = ()) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check for the lines which are exceeding the maximum length

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dinukaamarasinghe817 can you break it down to multiple lines?

ballerina/Module.md Outdated Show resolved Hide resolved
ballerina/redis_client.bal Outdated Show resolved Hide resolved
ballerina/redis_client.bal Outdated Show resolved Hide resolved
ballerina/redis_client.bal Outdated Show resolved Hide resolved
ballerina/redis_client.bal Outdated Show resolved Hide resolved
ballerina/stream_types.bal Outdated Show resolved Hide resolved
gradle.properties Outdated Show resolved Hide resolved
daneshk
daneshk previously approved these changes Mar 10, 2024
Copy link
Member

@daneshk daneshk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add minor comments. Overall LGTM

@dinukaamarasinghe817
Copy link
Contributor Author

Please note that I have updated the configurations to use the new Redis connector version 3.0.0 and now we support both connection parameters and connection URI in configurations.

@daneshk daneshk merged commit d8e1fd5 into ballerina-platform:main Mar 11, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants