Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mlflow.set_experiment doesn't find newly created experiment #119

Closed
nojaf opened this issue Jan 23, 2025 · 0 comments · Fixed by #120
Closed

mlflow.set_experiment doesn't find newly created experiment #119

nojaf opened this issue Jan 23, 2025 · 0 comments · Fixed by #120

Comments

@nojaf
Copy link
Collaborator

nojaf commented Jan 23, 2025

When running mlflow.set_experiment(f"demo_at_{time.time()}") I got

ERRO[0000] SQL error                                     app_file="github.com/mlflow/mlflow-go-backend/pkg/tracking/store/sql/experiments.go:255" app_func="github.com/mlflow/mlflow-go-backend/pkg/tracking/store/sql.TrackingSQLStore.GetExperimentByName()" elapsed=0.631ms error="record not found" rows=0 sql="SELECT * FROM \"experiments\" WHERE name = 'demo_at_1737635915.371468' ORDER BY \"experiments\".\"experiment_id\" LIMIT 1"
2025/01/23 13:38:35 INFO mlflow.tracking.fluent: Experiment with name 'demo_at_1737635915.371468' does not exist. Creating a new experiment.

Tests to reproduce:

Run postgres docker container:

docker run --name mlflow-postgres \
  -e POSTGRES_PASSWORD=mysecretpassword \
  -p 5432:5432 \
  -d postgres

Install uv add mlflow-go

Script:

# script.py
import time
import mlflow
import mlflow.entities
import mlflow_go_backend

mlflow_go_backend.enable_go()

mlflow.set_tracking_uri("postgresql://postgres:mysecretpassword@localhost:5432/postgres")
mlflow.set_experiment(f"demo_at_{time.time()}")

uv run ./script.py

I believe the problem is here that our Go implementation returns an error if didn't find anything.
While Python returns None instead:

https://github.com/mlflow/mlflow/blob/d4bccf76dfb79165731a3643a1397085060e6593/mlflow/store/tracking/sqlalchemy_store.py#L402-L417

While we do:

func (s TrackingSQLStore) GetExperimentByName(
ctx context.Context, name string,
) (*entities.Experiment, *contract.Error) {
var experiment models.Experiment
err := s.db.WithContext(ctx).Preload("Tags").Where("name = ?", name).First(&experiment).Error
if err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
return nil, contract.NewError(
protos.ErrorCode_RESOURCE_DOES_NOT_EXIST,
fmt.Sprintf("Could not find experiment with name %s", name),
)
}
return nil, contract.NewErrorWith(
protos.ErrorCode_INTERNAL_ERROR,
fmt.Sprintf("failed to get experiment by name %s", name),
err,
)
}
return experiment.ToEntity(), nil
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant