update from master #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* [UI Server] Pod info handler * [UI] Pod info tab in run details page * Change pod info preview to use yaml editor * Fix namespace * Adds error handling for PodInfo * Adjust to warning message * [UI] Pod events in RunDetails page * Adjust error message * Refactor k8s helper to get rid of in cluster limit * Tests for pod info handler * Tests for pod event list handler * Move pod yaml viewer related components to separate file. * Unit tests for PodYaml component * Fix react unit tests * Fix error message * Address CR comments * Add permission to ui role
* Initial execution cache This commit adds initial execution cache service. Including http service and execution key generation. * Add initial server logic * Add const * Change folder name * Change execution key name * Fix unit test * Add Dockerfile and OWNERS file This commit adds Dockerfile for building source code and OWNERS file for easy review. This commit also renames some functions. * fix go.sum This PR fixes changes on go.sum * Add local deployment scripts This commit adds local deployment scripts which can deploy cache service to an existing cluster with KFP installed. * refactor src code * Add standalone deployment scripts and yamls This commit adds execution cache deployment scripts and yaml files in KFP standalone deployment. Including a deployer which will generate the certification and mutatingwebhookconfiguration and execution cache deployment. * Minor fix * Add execution cache image build in test folder * fix test cloudbuild * Fix cloudbuild * Add execution cache deployer image to test folder * Add copyright * Fix deployer build * Add license for execution cache and cloudbuild for execution cache images This commit adds licenses for execution cache source code. Also adds cloud build step for building cache image and cache deployer image. Change the manifest name based on changed image. * Refactor license intermediate data * Fix execution cache image manifest * Typo fix for cache and cache deployer images * Add arguments in ca generation scripts and change deployer base image to google/cloud * minor fix * fix arg * Mirror source code with MPL in execution_cache image * Minor fix * minor refactor on error handling * Refactor cache source code, Docker image and manifest * Fix variable names * Add images in .release.cloudbuild.yaml * Change execution_cache to generic name * revice readme * Move deployer job out of upgrade script * fix tests * fix tests * Seperate cache service and cache deployer job * mysql set up * wip * WIP * WIP * work mysql connection * initial cache logic * watcher * WIP pod watching with mysql * worked crud * Add sql unit test * fix manifest * Add copyright * Add watcher check and update cache key generation logic * test replace container images * work cache service * Add configmap for cache service * refactor * fix manifest * Add unit tests * Remove delete table * Fix sql dialect * Add cached step log * Add metadata execution id * minor fix * revert go.mod and go.sum * revert go.sum and go.mod * revert go.sum and go.mod * revert go.mod and go.sum
* SDK - Added support for maxCacheStaleness * Added the vendor prefix to the annotation
* update watson components with output path args to support tekton * fix store bug and stop batch logs * update pipeline with explicit helper function * add missing commit
* SDK - Moved the @python_component decorator test to dsl tests * Deprecate @python_component
* Bump version to 0.3.0 * Fix formatting * More formatting fixes * More formatting fixes * update requirements.txt * update version
* Reduce steps for release cloud build yaml * Update .release.cloudbuild.yaml
* try generate MKP binary for each submit * try run * fix format * fix format * fix format * it works, gcloud builds submit --config test/cloudbuild/mkp_verify.yaml --project ml-pipeline-test * test commit trigger * backup codes * test * fix * pass manual test before submit * 0.3.0 Co-authored-by: Renmin Gu <renming@google.com>
… artifacts in a consistent manner. (#2992)
* Fix README formatting. * more fixes
* [UI Server] Blocks reportMetrics KFP api * Also reject report workflow endpoint * Also block report swf endpoint
…#3326) Added test_fail_on_handling_list_arguments_containing_python_objects Added test_handling_list_arguments_containing_serializable_python_objects Moved test_handling_list_arguments_containing_pipelineparam to component_bridge_tests
* [UI] Stops experiment list from leaking previous error message * Move the fix to Page component so it's more generic * Update ExperimentList.test.tsx
* [UI] Stops experiment list from leaking previous error message * Move the fix to Page component so it's more generic * [UI] Add namespace to AllRunsList api request * [UI] Add namespace to archived run page * Fix snapshot
I introduced a bug when parsing the image for Tensorboard in #3235. This fixes it.
* try generate MKP binary for each submit * try run * fix format * fix format * fix format * it works, gcloud builds submit --config test/cloudbuild/mkp_verify.yaml --project ml-pipeline-test * test commit trigger * backup codes * test * fix * pass manual test before submit * 0.3.0 * quick fix for test path Co-authored-by: Renmin Gu <renming@google.com>
* Initial execution cache This commit adds initial execution cache service. Including http service and execution key generation. * fix master * Change cache deployer job to stateful set * Delete cache deployer job * Delete cache deployer job after it completes * minor fix * fix indention * Change cache deployer job to statefulset * Remove extra cluster role for cache deployer * remove cache in base kustomize file for upgrade test * minor fix
* [UI Server] Add namespace argument for tensorboard endpoints * Allow local node server to talk to minio in cluster * Use tensorboard namespace in UI * Add unit tests for tensorboard UI server * Fix tests * Fix tensorboard proxy url * Fix tensorboard proxy failure * Fix tests * Remove unecessary encodeURIComponent * Add old comment back
…#3108) * Manifests: Rename metadata gRPC server's resources to metadata-grpc-* The metadata service deployed is a gRPC server. Proper KF installation deploys both an HTTP server, naming the required resources as 'metadata-deployment' and 'metadata-service', as well as a gRPC server, naming the corresponding resources 'metadata-grpc-deployment' and 'metadata-grpc-service'. KFP standalone installation manifests deploy solely the gRPC server, but use naming identical to the KF's HTTP server one. Applying them on top of an existing KF cluster breaks Metadata service. In this PR we change the naming making it not diverge from a proper KF installation. We also make MetadataWriter aware of that change. Closes #2889. Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com> * Fix ConfigMaps' label * metadata-configmap * metadata-mysql-configmap * README: Link to KF installation & reference KFP version
but MKP mpdev:latest has an issue block our tests
…ent_spec annotation (#3751) Removing the component spec from component_ref (since it would be a duplicate), but making sure the whole spec if available in component_spec.
* AWS Sagemaker Components - enhance integration test coverage - Add tests for create endpoint, hpo job and batch transform - Minor bug fixes and documentation * rev2: Address comments and clean up generated artifacts * rev3: address more comments * rev4: add canary test marker * Trigger Build
…_image and set_default_base_image functions (#3773)
This makes switching from code to tests easier
…3776) * [Manifest] Use kustomize native image transformer to override image * Revert unintended changes * Fix kustomization.yaml location * Fix inverse proxy image
This makes testing easier to run in local dev scenarios.
* Make user identity header configurable * use constants in UT.
Co-authored-by: Thi Nguyen <duongnt@users.noreply.github.com>
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
* # This is a combination of 5 commits. # This is the 1st commit message: Add initial scripts # This is the commit message #2: Add working pytest script # This is the commit message #3: Add initial scripts # This is the commit message #4: Add environment variable files # This is the commit message #5: Remove old cluster script * Add initial scripts Add working pytest script Add initial scripts Add environment variable files Remove old cluster script Update pipeline credentials to OIDC Add initial scripts Add working pytest script Add initial scripts Add working pytest script * Remove debugging mark * Update example EKS cluster name * Remove quiet from Docker build * Manually pass env * Update env list vars as string * Update use array directly * Update variable array to export * Update to using read for splitting * Move to helper script * Update export from CodeBuild * Add wait for minio * Update kubectl wait timeout * Update minor changes for PR * Update integration test buildspec to quiet build * Add region to delete EKS * Add wait for pods * Updated README * Add fixed interval wait * Fix CodeBuild step order * Add file lock for experiment ID * Fix missing pytest parameter * Update run create only once * Add filelock to conda env * Update experiment name ensuring creation each time * Add try/catch with create experiment * Remove caching from KFP deployment * Remove disable KFP caching * Move .gitignore changes to inside component * Add blank line to default .gitignore
* Add the 'kfp experiment list' command * Add the 'kfp experiment get' command * Add the 'kfp experiment create' command * Add the 'kfp experiment delete' command * Add a caution to 'kfp experiment delete' * Use directly the backend api method to list experiments * Update a message based on the suggestion #3705 (comment)
* don't use aws-secret and update readme for sample pipelines * Addressed comments on PR and few more readme changes * small changes to readme * nit change * Address comments
* [UI] Fix confusion matrix wrong axes * Fix confusion matrix background opacity
* Adding HPO unit test * Adding best training job * Addressing comment
* Client - Allow specifying pipeline description when uploading Fixes #3825 * Implemented review feedback
…avis tests (#3833) TFX package is has inconsistent dependencies wwhich causes the installation to be flaky and install different numpy version every time leading to failures.
…ent (#3830) * Integration Test for AWS SageMaker GroundTruth Component * Unfix already fixed bug * Fix the README I overwrote by mistake * Remove use of aws-secret for OIDC * Rev 2: Fix linting errors
…3824) * Create README.md * Added README Updated page to include information on Amazon SageMaker components * Update README.md * Integrated feedback
…on/experiment names (#3846) * a more accurate grpc error code * remove accidentally checked in file
* 5 runs * 50 runs * (1) add labels (2) instead of plotting kde, plotting histogram and rug
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.