PII Redaction

Personally identifiable information (PII) is any information that can be used to identify an individual, including data such credit-card numbers, email addresses, and phone numbers. PII can leak into log data that you transmit to a vendor. Laws such as GDPR and HIPAA require masking or removing PII from log data.

The MyDecisive SmartHub) can redact PII values before data leaves your network.

Run the commands on this page from the root directory of the mdai-labs repo you cloned.

Deploy the Demo

Make sure that you’re ready to demonstrate PII Redaction by first installing and running the MyDecisive SmartHub.

Start the demo.

mdai use-case pii --version 0.9.0 --workflow basic

Data should now be flowing to the collector.

Validate the Data Flow

The data we’re interested in is contained in the collector’s logs. We’ll use the K9s app to view them.

After installing K9s, enter k9s in a terminal and hit return
In the pods view, in the mdai namespace, select the gateway-collector.
Press the l key to display the collector logs.

The collector logs contain PII in the form of Social Security numbers, phone numbers, street addresses, credit-card numbers, and more. We don’t want to transmit that data to a vendor.

Add Static Redaction

Update the MyDecisive SmartHub configuration to enable static redaction.

mdai use_case pii --version 0.9.0 --workflow static

Go back and view the collector logs. This time the values for certain fields, or attributes, are no longer visible. The collector is configured with a processor in mdai-labs/0.9.0/use_cases/pii/staic/otel.yaml that deletes a hardcoded list of fields.

...
  processors:
    memory_limiter:
      check_interval: 23s
      limit_percentage: 75
      spike_limit_percentage: 15

    batch:
      send_batch_size: 1000
      send_batch_max_size: 10000
      timeout: 13s

    attributes/redact_pii_fields:
      actions:
        - key: email
          action: delete
        - key: phone
          action: delete
        - key: ssn
          action: delete
        - key: address
          action: delete
  ...
    pipelines:
      logs/customer_pipeline:
        receivers: [fluentforward]
        processors: [
          transform/redact_pii_fields,
          memory_limiter,
          batch
        ]
        exporters: [debug]

In the static config file, this OpenTelemetry attributes processor is named attributes/redact-pii-fields. It always deletes the configured attributes (email, phone, ssn, address) when they appear in the logs. If other attributes with PII data appear in the logs, no action is taken by this processor. Hardcoding attribute names is an ineffective approach in a production environment where the type of PII information logged can change.

Add Dynamic Redaction

Because new attributes containing PII information could appear in the logs at any time, we want to update the configuration as needed. Let’s switch from static to dynamic redaction, which allows us to make updates without restarting the cluster.

Update the MyDecisive SmartHub configuration to switch to dynamic redaction.
```
mdai use_case pii --version 0.9.0 --workflow dynamic
```
In K9s, in the pods view, select the gateway-collector, then press the L key to display the collector logs.
You’ll see logs with full-fidelity PII information.
Port forward the mdai-gateway service to allow programmatic updates to environment variables.
```
kubectl port-forward -n mdai svc/mdai-gateway 8081:8081
```
Note: Don’t confuse the mdai-gateway with the gateway-collector. The mdai-gateway provides an API for managing variables within the MyDecisive SmartHub ecosystem. More usage information can be found in the mdai-gateway repo.

Let’s redact the names of the people whose PII appears in the log lines.

  curl -X POST http://localhost:8081/variables/hub/mdaihub-pii/var/name_template \
    -H "Content-Type: application/json" \
    -d '{"data":"***REDACTED****"}'

The response includes information on the operation that our API call triggered.

  Handling connection for 8081
  {
    "id":"019be158-e6a6-7f06-8925-c5b1de315b5a",
    "name":"var.add",
    "version":1,
    "timestamp":"2026-01-21T16:17:38.470986965Z",
    "payload":"{
      "variableRef":"name_template",
      "dataType":"string",
      "operation":"add",
      "data":"***REDACTED****"
    }",
    "source":"manual_variables_api",
    "source_id":"",
    "hub_name":"mdaihub-pii"
  }

In K9s, in the pods view, select the gateway-collector, then press the L key to display the collector logs. You’ll see logs with the redacted name attribute.

undefined

Understand MyDecisive PII Redaction

Using dynamic PII redaction, we updated a variable to redact PII information without taking down any pods. The collector logs, however, contain more attributes with PII information. To redact those attributes, we need to update their corresponding variables in the same way.

Go back to K9s with the cluster still running to learn about the variables that enable PII redaction.

In K9s, in the configmaps view, select the mdaihub-pii-variables configmap, then press Y. This configmap shows the environment variables in effect, including NAME_TEMPLATE, which is enabling redaction of individuals’ names in the collector logs.
Press Escape to go back to the list of configmaps.
Select the mdaihub-pii-manual-variables configmap, then press Y.This configmap shows the list of updatable variables under data, including name_template, the variable we updated.
Use the same type of API call to update any of the variables, using the variable name in the endpoint.
```
http://localhost:8081/variables/hub/mdaihub-pii/var/<variable_name>
```

Lessons Learned

For security, privacy, and compliance reasons, PII should be removed or masked in log data. In production systems, the PII appearing in logs can change over time, but static redaction schemes force cluster takedowns to put new configuration into effect.

MyDecisive PII Redaction uses dynamic variables that can be updated programmatically while a cluster is still running. If your production system detects new attributes containing PII information, an update is as easy as an API call.

Learn more about the MyDecisive SmartHub architecture

Intelligent Trace Sampling