Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filters to Running Compactions table in monitor #4986

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,110 @@ $(document).ready(function () {
]
});

function handleFilterKeyup(input, feedbackElement, columnIndex) {
if (isValidRegex(input) || input === '') { // if valid, apply the filter
feedbackElement.hide();
$(this).removeClass('is-invalid');
const isRegex = true;
const smartEnabled = false;
runningTable
.column(columnIndex)
.search(input, isRegex, smartEnabled)
.draw();
} else { // if invalid, show the warning
feedbackElement.show();
$(this).addClass('is-invalid');
}
}

$('#hostname-filter').on('keyup', function () {
handleFilterKeyup.call(this, this.value, $('#hostname-feedback'), 0);
});

$('#queue-filter').on('keyup', function () {
handleFilterKeyup.call(this, this.value, $('#queue-feedback'), 3);
});

$('#tableid-filter').on('keyup', function () {
handleFilterKeyup.call(this, this.value, $('#tableid-feedback'), 4);
});

$('#duration-filter').on('keyup', function () {
runningTable.draw();
});

// Custom filter function for duration
$.fn.dataTable.ext.search.push(function (settings, data, dataIndex) {
if (settings.nTable.id !== 'runningTable') {
DomGarguilo marked this conversation as resolved.
Show resolved Hide resolved
return true;
}

const durationStr = data[8]; // duration is in the 9th column (index 8)
DomGarguilo marked this conversation as resolved.
Show resolved Hide resolved
const durationSeconds = parseDuration(durationStr);

const input = $('#duration-filter').val().trim();
if (input === '') {
return true;
}

const match = validateDurationInput(input);
if (!match) {
$('#duration-feedback').show();
return false;
}

$('#duration-feedback').hide();
const operator = match[1];
const value = parseInt(match[2]);
const unit = match[3];
const filterSeconds = convertToSeconds(value, unit);

switch (operator) {
case '>':
return durationSeconds > filterSeconds;
case '>=':
return durationSeconds >= filterSeconds;
case '<':
return durationSeconds < filterSeconds;
case '<=':
return durationSeconds <= filterSeconds;
default:
return true;
}
});

// Helper function to convert duration strings to seconds
function convertToSeconds(value, unit) {
switch (unit.toLowerCase()) {
case 's':
return value;
case 'm':
return value * 60;
case 'h':
return value * 3600;
case 'd':
return value * 86400;
default:
return value;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

212 and 228
Should these throw an error instead? We wouldn't expect the default to occur, correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 263b617. I added console.error logs. I dont think we want things to break if the default case is reached but logging an error seems like a good idea to indicate something has gone wrong.

}
}

// Helper function to validate duration input. Makes sure that the input is in the format of '<operator> <value> <unit>'
function validateDurationInput(input) {
return input.match(/^([<>]=?)\s*(\d+)([smhd])$/i);
}

/**
* @param {number} durationStr duration in milliseconds
* @returns duration in seconds
*/
function parseDuration(durationStr) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried running this again and figured out why I was not seeing any data. I had some config in my browser that was sending /rest/ec/running request to /dev/null.

Once I fixed that I could see data and noticed compaction ages were in years. Looked into this and I think its a unit mismatch. RunningCompactionInfo.java has the following line

   duration = last.getCompactionAgeNanos();

This function is expecting millis and maybe its getting nanos.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea it looks like thats the case. I thought it was a bug with how I was creating the sample data. The bug might be present in 2.1 as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. I didn't notice this. For me, the sample data seemed normal (nothing over a couple hrs)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be fixed via #5007. @keith-turner @kevinrr888

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When experimenting with these changes I was using accumulo-testing cingest to generate compactions. I started 10 compactor processes locally and ran ingest for a bit. The compactions were normally really fast and would not show up reliably. After a bit of data had built up, I forced full table compactions to create longer running compactions that I could look at in the monitor.

// Assuming durationStr is in milliseconds
const milliseconds = parseInt(durationStr, 10);
const seconds = milliseconds / 1000;
return seconds;
}

// Create a table for compaction coordinator
coordinatorTable = $('#coordinatorTable').DataTable({
"ajax": {
Expand Down Expand Up @@ -343,3 +447,13 @@ function refreshRunning() {
// user paging is not reset on reload
ajaxReloadTable(runningTable);
}

// Helper function to validate regex
function isValidRegex(input) {
try {
new RegExp(input);
return true;
} catch (e) {
return false;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,43 @@
<div class="row">
<div class="col-xs-12">
<table id="runningTable" class="table caption-top table-bordered table-striped table-condensed">
<caption><span class="table-caption">Running Compactions</span>&nbsp;&nbsp;
<a href="javascript:refreshRunning();"><span style="font-size: 1.5em; color: black;" class="bi bi-arrow-repeat"></span></a>
<caption>
<div class="d-flex justify-content-between align-items-center mb-3">
<div>
<span class="table-caption">Running Compactions</span>&nbsp;&nbsp;
<a href="javascript:refreshRunning();">
<span style="font-size: 1.5em; color: black;" class="bi bi-arrow-repeat"></span>
</a>
</div>
</div>
<button class="btn btn-secondary" type="button" data-bs-toggle="collapse" data-bs-target="#filterCollapse" aria-expanded="false" aria-controls="filterCollapse">
Toggle Filters
</button>
<!-- Collapsible Filter Section -->
<div class="collapse" id="filterCollapse">
<div class="card card-body">
<div class="mb-3">
<label for="hostname-filter" class="form-label">Hostname Filter</label>
<input type="text" id="hostname-filter" class="form-control" placeholder="Enter hostname regex">
<small id="hostname-feedback" class="form-text text-danger" style="display:none;">Invalid regex pattern</small>
</div>
<div class="mb-3">
<label for="queue-filter" class="form-label">Queue Filter</label>
<input type="text" id="queue-filter" class="form-control" placeholder="Enter queue regex">
<small id="queue-feedback" class="form-text text-danger" style="display:none;">Invalid regex pattern</small>
</div>
<div class="mb-3">
<label for="tableid-filter" class="form-label">Table ID Filter</label>
<input type="text" id="tableid-filter" class="form-control" placeholder="Enter table ID regex">
<small id="tableid-feedback" class="form-text text-danger" style="display:none;">Invalid regex pattern</small>
</div>
<div class="mb-3">
<label for="duration-filter" class="form-label">Duration Filter</label>
<input type="text" id="duration-filter" class="form-control" placeholder="Enter duration (e.g., &gt;10m, &lt;1h, &gt;=5s, &lt;=2d)">
<small id="duration-feedback" class="form-text text-danger" style="display:none;">Invalid duration format</small>
</div>
</div>
</div>
</caption>
<thead>
<tr>
Expand Down