Data Masking During Migration

📚 Overview & Scope

You list the columns to mask, and the rule for each one, in a JSON file. When a full-copy migration starts, Endrias Bridge checks every rule against the real table definitions before it touches the target. Then it masks each row in memory before writing it.

🔑 The Rules

Under every rule, NULL stays NULL.

1 Choose the columns to mask

In the desktop app (v1.3.131+): on the Migrate tab, click Load Tables from Source, then Masking Rules….

1. Pick a table on the left. Its columns are listed with their type and flags (PK, FK, unique, not null).
2. Pick a column, choose a rule (plus its options), and click Apply to column. The dialog runs the same safety checks as the migration, so a rule that can't work (for example redact on a unique column) is refused right there, with the reason.
3. Click Preview 5 rows to see real source values next to their masked versions. Nothing is written.
4. No key yet? Click Generate key… (see Step 2).
5. Click Save Rules.

Or write the file by hand. Save it as %APPDATA%\EndriasBridge\masking_rules.json, or anywhere you like and point masking_rules_file in migrationbridge.conf at it. Table names can be schema.table or just table. Column and table names are not case-sensitive.

{
  "tables": {
    "dbo.Patient": {
      "SSN":        {"rule": "partial", "keep_last": 4},
      "Email":      {"rule": "email"},
      "LastName":   {"rule": "hash"},
      "BirthDate":  {"rule": "date_shift", "max_days": 30},
      "Notes":      {"rule": "redact"},
      "HomePhone":  {"rule": "null"}
    },
    "dbo.Admission": {
      "AdmitDate":  {"rule": "date_shift", "max_days": 30, "by": "PatientID"},
      "Diagnosis":  {"rule": "redact", "value": "[removed]"}
    }
  }
}

2 Set the secret key

hash, email and date_shift need a secret key. Without it they refuse to run. Generate a long random value once and keep it in your password vault:

python -c "import secrets; print(secrets.token_urlsafe(32))"

In the Masking Rules dialog, Generate key… creates one, saves it as masking_key in your migrationbridge.conf and copies it to the clipboard. Otherwise provide it as the EB_MASKING_KEY environment variable (preferred), or as masking_key in migrationbridge.conf.

3 Run a Full copy and check the log

Choose Full sync mode and start the migration as usual. Each masked table shows one line in the migration log:

[Patient] MASKING SSN: partial, Email: email, LastName: hash, BirthDate: date_shift, Notes: redact, HomePhone: null — streaming path (bulk-load paths and PK-range checkpoints skipped)

From the command line:

set EB_MASKING_KEY=<your key>
migrationbridge migrate --src-type "SQL Server (VM / On-Prem)" --src-host prod-sql --src-db Clinic ^
    --src-user reader --src-pass *** --tgt-type "Azure SQL Database" --tgt-host dev.database.windows.net ^
    --tgt-db Clinic_Dev --tgt-user loader --tgt-pass *** --mode full --verify

If the rules file names a table that isn't being copied, the log shows WARNING (masking): ... matched none of the tables. Check that the names match.

4 Verify

Post-migration verification compares row counts, and checksums of sampled rows, leaving the masked columns out, since those differ from the source by design. Then spot-check the target yourself:

SELECT TOP 5 PatientID, SSN, Email, LastName, BirthDate FROM dbo.Patient;
-- expect XXX-XX-1234, u…@example.invalid, shaped-but-wrong names, dates within ±30 days

The Compare tool's row-level diff doesn't know about masking yet, so it reports masked columns as differences.

⛔ Why a Run Was Refused

⚠ Known Limitations