Skip to content

Commit

Permalink
[CLN] Refactor sysdb and log service (#2893)
Browse files Browse the repository at this point in the history
## Description of changes

*Summarize the changes made by this PR.*
 - This PR cleans out the sysdb and needless abstractions.
 - It removes the notification processor, and notifier since they are not used and don't intend to be used in the future
 - It removes the interfaces for the catalog and the coordinator since there is only one implementation and there's no future plans of having multiple implementations
 - It changes the directory structure to locate all the sysdb code together and similarly all the log service code together.
 - The directory structure has one directory per abstraction layer. Data models that interface two layer are present in the directory of the lower layer because the higher layer is technically dependent on the lower layer

## Test plan
- [x] Tests pass locally with `pytest` for python, `yarn test` for js, `cargo test` for rust

## Documentation Changes
None
  • Loading branch information
sanketkedia authored Oct 16, 2024
1 parent 3286f77 commit 70f1e74
Show file tree
Hide file tree
Showing 129 changed files with 156 additions and 6,754 deletions.
8 changes: 4 additions & 4 deletions go/Dockerfile.migration
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ RUN apt upgrade -y
RUN apt install -y curl jq
RUN curl -sSf https://atlasgo.sh | sh -s -- --community

COPY ./go/migrations migrations
COPY ./go/atlas.hcl atlas.hcl
COPY ./go/pkg/sysdb/metastore/db/migrations migrations
COPY ./go/pkg/sysdb/metastore/db/atlas.hcl atlas.hcl

FROM arigaio/atlas:latest as logservice-migration
COPY ./go/database/log/migrations migrations
COPY ./go/database/log/atlas.hcl atlas.hcl
COPY ./go/pkg/log/store/migrations migrations
COPY ./go/pkg/log/store/atlas.hcl atlas.hcl
2 changes: 1 addition & 1 deletion go/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ docker:
docker build -t chroma-coordinator:latest .

### LOG SERVICE
DATABABASE_LOG_DIR := database/log
DATABABASE_LOG_DIR := pkg/log/store

log_db_clean:
rm -rf $(DATABABASE_LOG_DIR)/db
Expand Down
7 changes: 1 addition & 6 deletions go/cmd/coordinator/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ import (
"io"
"time"

"github.com/chroma-core/chroma/go/pkg/coordinator/grpc"
"github.com/chroma-core/chroma/go/pkg/grpcutils"
"github.com/chroma-core/chroma/go/pkg/sysdb/grpc"

"github.com/chroma-core/chroma/go/cmd/flag"
"github.com/chroma-core/chroma/go/pkg/utils"
Expand Down Expand Up @@ -41,11 +41,6 @@ func init() {
Cmd.Flags().IntVar(&conf.DBConfig.MaxOpenConns, "max-open-conns", 10, "MetaTable max open connections")
Cmd.Flags().StringVar(&conf.DBConfig.SslMode, "ssl-mode", "disable", "SSL mode for database connection")

// Notification
Cmd.Flags().StringVar(&conf.NotificationStoreProvider, "notification-store-provider", "memory", "Notification store provider")
Cmd.Flags().StringVar(&conf.NotifierProvider, "notifier-provider", "memory", "Notifier provider")
Cmd.Flags().StringVar(&conf.NotificationTopic, "notification-topic", "chroma-notification", "Notification topic")

// Memberlist
Cmd.Flags().StringVar(&conf.KubernetesNamespace, "kubernetes-namespace", "chroma", "Kubernetes namespace")
Cmd.Flags().DurationVar(&conf.ReconcileInterval, "reconcile-interval", 100*time.Millisecond, "Reconcile interval")
Expand Down
Loading

0 comments on commit 70f1e74

Please sign in to comment.