-
Notifications
You must be signed in to change notification settings - Fork 204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Capability to support loading terminated jobs after leader election #595
Conversation
Test Results487 tests - 62 481 ✔️ - 60 7m 15s ⏱️ -10s Results for commit f381695. ± Comparison against base commit 1dfc742. This pull request removes 68 and adds 6 tests. Note that renamed tests count towards both.
♻️ This comment has been updated with latest results. |
Uploaded ArtifactsTo use these artifacts in your Gradle project, paste the following lines in your build.gradle.
|
// let's load 1 page of completed jobs from DB and then let the rest be loaded lazily | ||
// todo(sundaram): Use a clock here | ||
Instant end = Instant.now(); | ||
List<CompletedJob> completedJobs = jobStore.loadCompletedJobsForCluster(name, end.minus(Duration.ofDays(7)), end); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this a blocking IO call? can it be converted to a future?
805b66a
to
962a49d
Compare
|
||
interface ICompletedJobsStore { | ||
|
||
void initialize() throws IOException; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add comments
|
||
void initialize() { | ||
try { | ||
logger.info("Loading completed jobs for cluster {}", name); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change this to debug instead.
List<CompletedJob> getCompletedJobs(int limit) throws IOException; | ||
|
||
/** | ||
* Gets a list of completed jobs | ||
* @param limit number of jobs to return | ||
* @param endExclusive end job id | ||
* @return list of completed jobs | ||
* @throws IOException if there is an error | ||
*/ | ||
List<CompletedJob> getCompletedJobs(int limit, JobId endExclusive) throws IOException; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So this only supports reading first x jobs? Is there a way to read from the list paginated?
while (!jobs.isEmpty()) { | ||
for (CompletedJob completedJob : jobs) { | ||
try { | ||
// todo(sundaram): Clean this up. This is a hack to get around the fact that the job store |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
incomplete comment?
Context
^^^
Checklist
./gradlew build
compiles code correctly./gradlew test
passes all tests