Redaction method and residue checks

Processing steps

1. Text layer: Text characters in the target area are removed, together with any invisible recognized text under that area.
2. Image: Pixels in the matching area of the scanned image are covered; vector graphics that are fully covered are removed.
3. Marking: A black box is drawn, or a box with a “Redacted” label.
4. Rewrite: The file is rewritten from scratch with pikepdf. Old incremental versions are not kept, nor are document information and XMP, attachments, JavaScript, forms, annotations, bookmarks, structure tag text or thumbnails. EXIF, XMP and comment segments are removed from JPEG data streams.

Residue checks

After saving, the output file is reopened and compared item by item against the removed content:

1. Page text and hidden text layers;
2. Scanned image pixels (processed areas are recognized again);
3. Strings and data streams in file objects (several encodings);
4. Document information and XMP metadata;
5. Annotations, forms, attachments, scripts and bookmarks;
6. EXIF / XMP / comments embedded in images;
7. JBIG2 shared symbol dictionaries (warning only when present);
8. Earlier versions (incremental saves).

Each result is pass, warning or fail, and is written to the save banner and the operation record (manifest.json).

Detection scope

The built-in detectors cover 17 kinds of numbers. Where a check digit or date can be validated, it is. See the full list in Secure redaction, and try the rules in the ID number format tester.

Known limits

• Detection depends on OCR. On poor scans a number may be misread, and numbers whose check digit fails are not treated as ID numbers. Review important files manually.
• Black-and-white images compressed with JBIG2 may share glyphs across pages; a warning is shown when this occurs.
• Built-in rules do not detect free text such as names and addresses. Add them as custom keywords.
• Redaction handles file content only, not file names or email bodies.