User's Guide - Redacting Log Messages
In This Topic
Loupe aggregates log messages together based on matching several fields including the Caption. When warnings or errors have unique values inserted into their captions - like timestamps, URLs, database Ids or user names then they won't be merged into a common event like they should be.
Automatic Redaction Rules
Loupe automatically scans each caption and substitutes a label for data that commonly causes poor aggregation. This includes:
| Label |
Example |
| Path |
C:\Program Files (x86)\Gibraltar Software\Loupe |
| URL |
https://app.onloupe.com/ |
| File |
output.html |
| IP Address |
192.168.1.1 |
| Version |
2023.12.1.0 |
| Date |
6/19/2020 |
| Time |
12:12 AM |
| Timestamp |
2021-09-16T14:30:00+03:00 |
| Duration |
0.0004ms |
| GUID |
317A3681-3AB0-4E5D-8B62-A517BA0DDA3C |
| Hex Value |
0x8000FFFF |
| Number |
19.999990 |
Redacting Additional Data
If your log captions include unique values that are not addressed by automatic redaction rules, you can configure a Redaction Rule. This will rewrite the caption during analysis solely for the purpose of how messages are merged.
For example, consider the following event:
In this case the caption includes two unique values that should be redacted so we can merge together occurrences regardless of the name and id (since they represent a common issue).
You can create a redaction rule from this message, specifying the replacement values you want to use for the parts you redact.
Any log mesage that matches a configured redaction rule will skip automatic rule processing.
Creating Redaction Rule from an Event
When you find an Event that has a value in its caption you would like to redact select Redact from the Actions menu in the upper right. This will present you with the Redaction Rule Configuration Dialog, like this:
To redact a value:
- Highlight the text you want to redact.
- Enter the text you want to substitute into the box labeled With.
- Click Redact to add that value redaction to the rule. The Preview area will show what the redacted message would look like.
Once you've redacted all of the text you want to, click Save to commit the rule to the server. It will be applied in the background by the server automatically to existing data. By default, redaction rules only apply to the product the selected event was related to. You can change this by editing the rule in the Administration area.
Managing Redaction Rules
If you are an administrator you can view, edit, and create new redaction rules in the Administration area. To do this:
- Select Administration from your user menu in the upper right hand corner of the screen.
- Select Event Redaction from the list of administration areas on the left.
You can create new redaction rules (provided you have an example caption you want to redact) and edit existing rules. You can also remove a redaction rule that was created incorrectly.
Frequently Asked Questions
Why am I still seeing individual events that should have been changed by a redaction rule I added?
Rules changes are processed asynchronously by the Loupe Server Service after they are modified. Depending on the number of unique events in the system this may take a while to complete. Each event is redacted and merged individually and there are various throttles to prevent this from overwhelming the database. New events will see the rule change immediately.
If I remove a rule will all of the events be restored to how they were?
No, once an event has been redacted the underlying information in the index database has been altered to drop the unique captions. While the original values are still present in the log files the logs would have to be re-analyzed to discover the original values. This can be done on a session-by-session basis in the web UI.
Why is the rule only applying to some sessions?
By default rules are associated with the product that the first event used to make the rule is associated with. This means if you select an event for a session associated with Product A, that rule will only apply to other sessions in applications also associated with Product B. You can go into the Administration area and edit the rule under Event Redaction to change the scope - either narrowing it to just apply to a single application within the product or broadening it to apply to any product.