Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Support manual redaction #2433

Merged
merged 101 commits into from
Jan 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
0a13573
Support text selection redaction in the browser
omar-ahmed42 Dec 9, 2024
f1aaa67
Add en_GB translation for manual redaction
omar-ahmed42 Dec 9, 2024
47d55a2
Add manual redaction - tool header
omar-ahmed42 Dec 9, 2024
5e81a09
Fix: Selection box alignment issue at low zoom levels
omar-ahmed42 Dec 9, 2024
776acbf
Use ctrl + S instead of ctrl shift to apply redaction on selected text
omar-ahmed42 Dec 9, 2024
641cbfd
Prevent Redaction wrappers from being selected
omar-ahmed42 Dec 9, 2024
008e26d
Add submit btn and hidden redactions input
omar-ahmed42 Dec 10, 2024
722026e
Assign values for redactionsInput
omar-ahmed42 Dec 10, 2024
70046a3
Update messages_en_GB.properties
omar-ahmed42 Dec 10, 2024
a25d244
Scale x, y, width and height to match PDF page points
omar-ahmed42 Dec 10, 2024
39ddec5
Add manual PDF area-based redaction and supporting classes
omar-ahmed42 Dec 10, 2024
4c40f1d
Fix incorrect y position calculation while adding rect to pdf
omar-ahmed42 Dec 10, 2024
29198c8
Fix formula for redaction points and dimensions
omar-ahmed42 Dec 10, 2024
5ed9a4b
Add support for drawing-based redaction
omar-ahmed42 Dec 10, 2024
04d2e82
Add buttons for text selection and shape drawing
omar-ahmed42 Dec 10, 2024
fa1d2bc
Add redaction mode selection using redactions buttons
omar-ahmed42 Dec 10, 2024
a53492a
Refactor redact.js slightly for more readability
omar-ahmed42 Dec 10, 2024
46fea6a
Remove an unused import
omar-ahmed42 Dec 10, 2024
276fa8e
Refactor code and clear drawing on entering a different canvas/textLayer
omar-ahmed42 Dec 10, 2024
572a54f
Fix a bug that caused redaction areas to be empty after delete operation
omar-ahmed42 Dec 10, 2024
cb350bc
Use correct color for toolbar icons
omar-ahmed42 Dec 10, 2024
c0e260d
Implement mutual exclusivity for Draw-based Redaction and Text Select…
omar-ahmed42 Dec 10, 2024
a64de02
Support pdf to image option
omar-ahmed42 Dec 11, 2024
29d88d2
Add escape shortcut to cancel current drawing
omar-ahmed42 Dec 11, 2024
cfd8c96
Add delete shortcut to delete redacted element
omar-ahmed42 Dec 11, 2024
857e459
Support page-based redactions
omar-ahmed42 Dec 11, 2024
ebc9216
Move redaction tools to the side of the zoom scale
omar-ahmed42 Dec 11, 2024
60da7d4
Rename pageNumbers's input id
omar-ahmed42 Dec 11, 2024
f32b0f1
Support color choosing for page-based redaction
omar-ahmed42 Dec 11, 2024
3dc7bad
Support customization per redaction area
omar-ahmed42 Dec 11, 2024
2f90705
Fix a bug that caused color to always be black
omar-ahmed42 Dec 11, 2024
d8a8d49
Remove unnecessary code
omar-ahmed42 Dec 11, 2024
e81c47c
Add documentation for manual redaction
omar-ahmed42 Dec 11, 2024
ac305ac
Add manual redaction card to home page
omar-ahmed42 Dec 11, 2024
112c6ba
Add a button to apply redaction on selected text
omar-ahmed42 Dec 11, 2024
0fe94ad
Use black as default redaction color for color palettes
omar-ahmed42 Dec 11, 2024
05dd59c
Merge branch 'main' into feature-manual-redaction
omar-ahmed42 Dec 12, 2024
e79a496
Merge branch 'main' into feature-manual-redaction
Frooodle Dec 13, 2024
fdc72d1
Merge branch 'main' of https://github.com/Stirling-Tools/Stirling-PDF…
omar-ahmed42 Dec 14, 2024
a643268
Adjust toolbar and buttons to be bigger
omar-ahmed42 Dec 14, 2024
2314f9a
Add Stirling logo to the toolbar
omar-ahmed42 Dec 14, 2024
605742d
Style and customize the toolbar
omar-ahmed42 Dec 15, 2024
f7275cd
Add a container redactions palette
omar-ahmed42 Dec 15, 2024
33669ac
Style toolbar buttons
omar-ahmed42 Dec 15, 2024
4408ea5
Remove unused classes
omar-ahmed42 Dec 15, 2024
f7906dd
Add manual redaction to the nav bar
omar-ahmed42 Dec 15, 2024
f14a350
Remove eraser icon
omar-ahmed42 Dec 15, 2024
1a392c6
Use new more fitting icons
omar-ahmed42 Dec 15, 2024
7f6c808
Display apply redaction btn in text mode only
omar-ahmed42 Dec 15, 2024
6e33b9d
Add download button and loading spinner
omar-ahmed42 Dec 15, 2024
d6e26cb
Fix hidden redaction overlay due to overlapping with redaction box
omar-ahmed42 Dec 16, 2024
89e0623
Hide nav bar, footer and form on file selection
omar-ahmed42 Dec 16, 2024
5d94c9b
Rename properties popup
omar-ahmed42 Dec 17, 2024
f03040f
Hide input fields in the initial form (in manual redaction) to avoid …
omar-ahmed42 Dec 17, 2024
09e8a84
Bring back nav bar
omar-ahmed42 Dec 18, 2024
7cc0029
Disable scroll Y by hiding overflow Y
omar-ahmed42 Dec 18, 2024
0f645f8
Update viewer height to avoid overflow
omar-ahmed42 Dec 18, 2024
0945469
Add button for pdf to image conversion
omar-ahmed42 Dec 19, 2024
1367e32
Adjust toolbar's z-index to uncover navbar
omar-ahmed42 Dec 19, 2024
5b87a8f
Change page based redaction icon
omar-ahmed42 Dec 21, 2024
2b5394e
Fix old redactions applying to new uploads
omar-ahmed42 Dec 21, 2024
da175d3
Disable page rotation shortcut
omar-ahmed42 Dec 21, 2024
cffef23
Fix incorrect redaction placement bug on rotated pdfs
omar-ahmed42 Dec 23, 2024
5209a9c
Adjust toolbar buttons dimensions
omar-ahmed42 Dec 24, 2024
8966b14
Fix icon overlapping and make design more responsive
omar-ahmed42 Dec 25, 2024
223392a
Fix show more button being unclickable
omar-ahmed42 Dec 25, 2024
e875fcb
Use correct selector
omar-ahmed42 Dec 25, 2024
9e318f6
Display show more button for medium-small screens
omar-ahmed42 Dec 25, 2024
9a8a0cc
Display zoom buttons side by side in small screens
omar-ahmed42 Dec 25, 2024
b27a5bb
Center page based redaction overlay
omar-ahmed42 Dec 25, 2024
3d79461
Adjust page based redaction overlay for small-medium screens
omar-ahmed42 Dec 25, 2024
429637c
Extract text redaction box rotation logic into a function
omar-ahmed42 Dec 27, 2024
d7f3751
Fix box redaction placement in rotated PDFs
omar-ahmed42 Dec 27, 2024
7500614
Fix full PDF not being displayed
omar-ahmed42 Dec 29, 2024
0fe6466
Fix redaction overlay rotating with the page
omar-ahmed42 Dec 29, 2024
51eae27
Place redaction overlay at the bottom of redaction box
omar-ahmed42 Dec 29, 2024
f3e9e6d
Fix redaction overlay being placed out of page
omar-ahmed42 Dec 29, 2024
fd3168a
Fix disappearance of redacted areas
omar-ahmed42 Dec 29, 2024
ae92a41
Fix top offset sign for active overlay coordinates
omar-ahmed42 Dec 30, 2024
5756909
Hide overlay on rotation change
omar-ahmed42 Dec 30, 2024
06a8967
Refactor overlay coordinates logic into a function
omar-ahmed42 Dec 30, 2024
43bae29
Prevent accessing undefined/null activeOverlay in rotationchangin event
omar-ahmed42 Dec 30, 2024
c83163a
Fix box-based redaction for landscape PDFs in Safari
omar-ahmed42 Dec 30, 2024
24baaa7
Disable user/text-selection in box redaction mode
omar-ahmed42 Dec 30, 2024
88d3140
Fix color picker position for color input in the toolbar
omar-ahmed42 Dec 31, 2024
ee5bfd1
Add apply button to save page redaction settings
omar-ahmed42 Dec 31, 2024
d4ad9da
Add page input validation for page number/range/function
omar-ahmed42 Jan 1, 2025
6c4f9ba
Add redacted page preview
omar-ahmed42 Jan 1, 2025
778d39c
Add palette icon next to shortened "redaction color' text
omar-ahmed42 Jan 1, 2025
619051d
Display color picker below button in redaction overlays
omar-ahmed42 Jan 1, 2025
d002ad3
Prepare N Function's format for evaluation
omar-ahmed42 Jan 1, 2025
e5ef67b
Reset feedback messages on change/apply
omar-ahmed42 Jan 1, 2025
b2b4376
Refactor page redaction validation and action
omar-ahmed42 Jan 1, 2025
953c5a4
Handle closing bracket pattern for N Functions
omar-ahmed42 Jan 1, 2025
83d0f15
Add redaction preview to sidebar thumbnails
omar-ahmed42 Jan 1, 2025
d8d100c
Use click and drag mechanism for box redaction
omar-ahmed42 Jan 3, 2025
12bd77d
Draw box when pointer is out of canvas
omar-ahmed42 Jan 3, 2025
901f593
Refactor calculateMouseCoordinateToRotatedBox to reduce cognitive com…
omar-ahmed42 Jan 3, 2025
c3f631c
Disable Stirling icon selection in the tool bar
omar-ahmed42 Jan 3, 2025
83c3121
Adjust code styling format
omar-ahmed42 Jan 3, 2025
54fc17d
Merge branch 'main' into feature-manual-redaction
Frooodle Jan 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ public void init() {
addEndpointToGroup("Security", "remove-cert-sign");
addEndpointToGroup("Security", "sanitize-pdf");
addEndpointToGroup("Security", "auto-redact");
addEndpointToGroup("Security", "redact");

// Adding endpoints to "Other" group
addEndpointToGroup("Other", "ocr-pdf");
Expand Down Expand Up @@ -234,6 +235,7 @@ public void init() {
addEndpointToGroup("Java", "markdown-to-pdf");
addEndpointToGroup("Java", "show-javascript");
addEndpointToGroup("Java", "auto-redact");
addEndpointToGroup("Java", "redact");
addEndpointToGroup("Java", "pdf-to-csv");
addEndpointToGroup("Java", "split-by-size-or-count");
addEndpointToGroup("Java", "overlay-pdf");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,18 @@
import java.awt.*;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.Collections;
import java.util.List;

import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.PDPageTree;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.WebDataBinder;
import org.springframework.web.bind.annotation.InitBinder;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
Expand All @@ -21,12 +26,17 @@
import io.swagger.v3.oas.annotations.tags.Tag;

import lombok.extern.slf4j.Slf4j;

import stirling.software.SPDF.model.PDFText;
import stirling.software.SPDF.model.api.security.ManualRedactPdfRequest;
import stirling.software.SPDF.model.api.security.RedactPdfRequest;
import stirling.software.SPDF.model.api.security.RedactionArea;
import stirling.software.SPDF.pdf.TextFinder;
import stirling.software.SPDF.service.CustomPDDocumentFactory;
import stirling.software.SPDF.utils.GeneralUtils;
import stirling.software.SPDF.utils.PdfUtils;
import stirling.software.SPDF.utils.WebResponseUtils;
import stirling.software.SPDF.utils.propertyeditor.StringToArrayListPropertyEditor;

@RestController
@RequestMapping("/api/v1/security")
Expand All @@ -41,6 +51,107 @@ public RedactController(CustomPDDocumentFactory pdfDocumentFactory) {
this.pdfDocumentFactory = pdfDocumentFactory;
}

@InitBinder
public void initBinder(WebDataBinder binder) {
binder.registerCustomEditor(List.class, "redactions", new StringToArrayListPropertyEditor());
}

@PostMapping(value = "/redact", consumes = "multipart/form-data")
@Operation(summary = "Redacts areas and pages in a PDF document", description = "This operation takes an input PDF file with a list of areas, page number(s)/range(s)/function(s) to redact. Input:PDF, Output:PDF, Type:SISO")
public ResponseEntity<byte[]> redactPDF(@ModelAttribute ManualRedactPdfRequest request) throws IOException {
MultipartFile file = request.getFileInput();
List<RedactionArea> redactionAreas = request.getRedactions();

PDDocument document = pdfDocumentFactory.load(file);

PDPageTree allPages = document.getDocumentCatalog().getPages();

redactPages(request, document, allPages);
redactAreas(redactionAreas, document, allPages);

if (request.isConvertPDFToImage()) {
PDDocument convertedPdf = PdfUtils.convertPdfToPdfImage(document);
document.close();
document = convertedPdf;
}

ByteArrayOutputStream baos = new ByteArrayOutputStream();
document.save(baos);
document.close();

byte[] pdfContent = baos.toByteArray();
return WebResponseUtils.bytesToWebResponse(
pdfContent,
Filenames.toSimpleFileName(file.getOriginalFilename()).replaceFirst("[.][^.]+$", "")
+ "_redacted.pdf");
}

private void redactAreas(List<RedactionArea> redactionAreas, PDDocument document, PDPageTree allPages)
throws IOException {
Color redactColor = null;
for (RedactionArea redactionArea : redactionAreas) {
if (redactionArea.getPage() == null || redactionArea.getPage() <= 0
|| redactionArea.getHeight() == null || redactionArea.getHeight() <= 0.0D
|| redactionArea.getWidth() == null || redactionArea.getWidth() <= 0.0D)
continue;
PDPage page = allPages.get(redactionArea.getPage() - 1);

PDPageContentStream contentStream = new PDPageContentStream(
document, page, PDPageContentStream.AppendMode.APPEND, true, true);
redactColor = decodeOrDefault(redactionArea.getColor(), Color.BLACK);
contentStream.setNonStrokingColor(redactColor);

float x = redactionArea.getX().floatValue();
float y = redactionArea.getY().floatValue();
float width = redactionArea.getWidth().floatValue();
float height = redactionArea.getHeight().floatValue();

PDRectangle box = page.getBBox();

contentStream.addRect(x, box.getHeight() - y - height, width, height);
contentStream.fill();
contentStream.close();
}
}

private void redactPages(ManualRedactPdfRequest request, PDDocument document, PDPageTree allPages)
throws IOException {
Color redactColor = decodeOrDefault(request.getPageRedactionColor(), Color.BLACK);
List<Integer> pageNumbers = getPageNumbers(request, allPages.getCount());
for (Integer pageNumber : pageNumbers) {
PDPage page = allPages.get(pageNumber);

PDPageContentStream contentStream = new PDPageContentStream(
document, page, PDPageContentStream.AppendMode.APPEND, true, true);
contentStream.setNonStrokingColor(redactColor);

PDRectangle box = page.getBBox();

contentStream.addRect(0, 0, box.getWidth(), box.getHeight());
contentStream.fill();
contentStream.close();
}
}

private Color decodeOrDefault(String hex, Color defaultColor) {
Color color = null;
try {
color = Color.decode(hex);
} catch (Exception e) {
color = defaultColor;
}

return color;
}

private List<Integer> getPageNumbers(ManualRedactPdfRequest request, int pagesCount) {
String pageNumbersInput = request.getPageNumbers();
String[] parsedPageNumbers = pageNumbersInput != null ? pageNumbersInput.split(",") : new String[0];
List<Integer> pageNumbers = GeneralUtils.parsePageList(parsedPageNumbers, pagesCount, false);
Collections.sort(pageNumbers);
return pageNumbers;
}

@PostMapping(value = "/auto-redact", consumes = "multipart/form-data")
@Operation(
summary = "Redacts listOfText in a PDF document",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ public String autoRedactForm(Model model) {
return "security/auto-redact";
}

@GetMapping("/redact")
public String redactForm(Model model) {
model.addAttribute("currentPage", "redact");
return "security/redact";
}

@GetMapping("/add-password")
@Hidden
public String addPasswordForm(Model model) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
package stirling.software.SPDF.model.api.security;

import java.util.List;

import io.swagger.v3.oas.annotations.media.Schema;
import lombok.Data;
import lombok.EqualsAndHashCode;
import stirling.software.SPDF.model.api.PDFWithPageNums;

@Data
@EqualsAndHashCode(callSuper = true)
public class ManualRedactPdfRequest extends PDFWithPageNums {
@Schema(description = "A list of areas that should be redacted")
private List<RedactionArea> redactions;

@Schema(description = "Convert the redacted PDF to an image", defaultValue = "false")
private boolean convertPDFToImage;

@Schema(description = "The color used to fully redact certain pages")
private String pageRedactionColor;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package stirling.software.SPDF.model.api.security;

import io.swagger.v3.oas.annotations.media.Schema;
import lombok.Data;

@Data
public class RedactionArea {
@Schema(description = "The left edge point of the area to be redacted.")
private Double x;
@Schema(description = "The top edge point of the area to be redacted.")
private Double y;

@Schema(description = "The height of the area to be redacted.")
private Double height;
@Schema(description = "The width of the area to be redacted.")
private Double width;

@Schema(description = "The page on which the area should be redacted.")
private Integer page;

@Schema(description = "The color used to redact the specified area.")
private String color;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
package stirling.software.SPDF.utils.propertyeditor;

import java.beans.PropertyEditorSupport;
import java.util.ArrayList;
import java.util.List;

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;

import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.api.security.RedactionArea;

@Slf4j
public class StringToArrayListPropertyEditor extends PropertyEditorSupport {

private final ObjectMapper objectMapper = new ObjectMapper();

@Override
public void setAsText(String text) throws IllegalArgumentException {
if (text == null || text.trim().isEmpty()) {
setValue(new ArrayList<>());
return;
}
try {
objectMapper.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);
TypeReference<ArrayList<RedactionArea>> typeRef = new TypeReference<ArrayList<RedactionArea>>() {
};
List<RedactionArea> list = objectMapper.readValue(text, typeRef);
setValue(list);
} catch (Exception e) {
log.error("Exception while converting {}", e);
throw new IllegalArgumentException(
"Failed to convert java.lang.String to java.util.List");
}
}
}
14 changes: 14 additions & 0 deletions src/main/resources/messages_en_GB.properties
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ pages=Pages
loading=Loading...
addToDoc=Add to Document
reset=Reset
apply=Apply

legal.privacy=Privacy Policy
legal.terms=Terms and Conditions
Expand Down Expand Up @@ -474,6 +475,10 @@ home.autoRedact.title=Auto Redact
home.autoRedact.desc=Auto Redacts(Blacks out) text in a PDF based on input text
autoRedact.tags=Redact,Hide,black out,black,marker,hidden

home.redact.title=Manual Redaction
home.redact.desc=Redacts a PDF based on selected text, drawn shapes and/or selected page(s)
redact.tags=Redact,Hide,black out,black,marker,hidden,manual

home.tableExtraxt.title=PDF to CSV
home.tableExtraxt.desc=Extracts Tables from a PDF converting it to CSV
tableExtraxt.tags=CSV,Table Extraction,extract,convert
Expand Down Expand Up @@ -579,6 +584,15 @@ autoRedact.customPaddingLabel=Custom Extra Padding
autoRedact.convertPDFToImageLabel=Convert PDF to PDF-Image (Used to remove text behind the box)
autoRedact.submitButton=Submit

#redact
redact.title=Manual Redaction
redact.header=Manual Redaction
redact.submit=Redact
redact.pageBasedRedaction=Page-based Redaction
redact.convertPDFToImageLabel=Convert PDF to PDF-Image (Used to remove text behind the box)
redact.pageRedactionNumbers.title=Pages
redact.pageRedactionNumbers.placeholder=(e.g. 1,2,8 or 4,7,12-16 or 2n-1)
redact.redactionColor.title=Redaction Color

#showJS
showJS.title=Show Javascript
Expand Down
Loading
Loading