User Guide v1.2

Endrias Bridge

Enterprise database migration tool — step-by-step guide covering the Setup Wizard, Migration Engine, Replication, HA topologies, and operational diagnostics.

📂 Version: June 2026 💻 Platform: Windows Server / Windows 10+ / Docker 📄 Install Path: C:\MigrationTool
1
Prerequisites
2
Connect
3
Assess
4
Initial Load
5
CDC Stream
6
Validate
7
Cutover
Relational & NoSQL
SQL Server, PostgreSQL, MySQL, Oracle, SQLite, MongoDB, DynamoDB
25+ Engine Types
Any ODBC-compatible source or target engine; full type-mapping matrix
Full / Incremental / CDC
Full copy, watermark incremental, SQL Server CDC, and real-time CDC Stream
BCP Acceleration
Automatic BCP bulk export + BULK INSERT on SQL Server targets; 3-5x throughput
BACPAC Seed
SqlPackage-based export/import for 1 TB+ tables; same mechanism as Azure Portal
Parallel Streams
Per-table PK-range partitioned copy; up to 8 parallel workers per large table
Live Web Dashboard
Built-in FastAPI dashboard at :8765; CI/CD REST endpoint; no RDP required
Docker / Headless
Linux container deployment; Dockerfile + docker-compose.yml included

📖 About This Guide

This guide covers the Endrias Bridge Setup Wizard and Migration Engine end-to-end — from first launch through cutover. Operational depth increases with each section; DBAs should read linearly on first use, then reference individual sections as needed.

Endrias Bridge is an enterprise database migration platform for moving relational and NoSQL data between on-premises, cloud, and hybrid environments. It combines a GUI Setup Wizard (for VM initialization and connection setup), a Migration Engine (full/incremental/CDC modes), and a Replication subsystem (CDC Stream and Scheduled Sync) into a single deployable package.

Endrias Bridge architecture diagram — Network/Security Boundary, Source Database, Target Database, Staging Host modules, connection arrows, security callouts
Security callouts:
Least-privilege DB accounts — SELECT + VIEW DATABASE STATE (source)  ·  CREATE / INSERT / ALTER (target)
TLS 1.2 enforced on all Azure SQL & AWS RDS connections — cert validation configurable per connection profile
Credentials stored in Windows Credential Manager (keyring) — never written to config files in plaintext
Staging host isolation — no inbound internet exposure; restrict source/target SG / NSG to staging host IP only
ℹ️
For Windows-service installation, command-line operation, and deep troubleshooting of the background initialization service, see MigrationBridge_SOP.html. This guide focuses on the GUI wizard and migration engine operation.

🚀 Installing & Launching

Install via the Windows Installer (MigrationBridge_Setup.exe) for production deployments. Docker is recommended for Linux/cloud-native environments.

The recommended installation method is the Windows Installer. Double-click MigrationBridge_Setup.exe and follow the prompts. It installs to %ProgramFiles%\Endrias Bridge\, registers an uninstaller in Add/Remove Programs, and creates a desktop shortcut. No Python or additional runtime dependencies are required — the installer is fully self-contained.

  1. Windows Installer (recommended for production)

    Run MigrationBridge_Setup.exe as Administrator. After installation completes, launch from the Endrias Bridge shortcut in the Start Menu or Desktop.

  2. Desktop shortcut (source install)

    If install_shortcut.py was run after source extraction, double-click the Endrias Bridge shortcut. Shortcut targets launch_gui.pyw with no console window.

  3. Launch script

    Double-click launch_gui.pyw in the install folder (opens silently — no console window attached).

  4. Command line / headless

    Open PowerShell in C:\MigrationTool and run MigrationBridge.exe gui to open the GUI, or use the migrate subcommand to run a full headless migration without opening any window — see the CLI Headless Mode section for full details and flag reference.

  5. Docker (Linux / cloud-native)

    Use the included Dockerfile and docker-compose.yml. ODBC Driver 17 is installed automatically in the container image. Mount migrationbridge.conf as read-only and ./migration_state for checkpoint persistence. Run: docker compose up migration. Dashboard available at http://localhost:8765.

💡
The wizard is a 7-step process. Use Next / Back to navigate — selections are preserved across step transitions. You can open the Endrias Bridge directly from Step 5 at any time without completing the full wizard.

1️⃣ Step 1 — Welcome

Review the operations the wizard will perform on this VM, then confirm the config file path before proceeding.

The Welcome screen summarizes what the wizard will execute on this VM: set hostname, create a local user, configure networking, inject SSH keys, and run userdata scripts — plus any additional plugins enabled in Step 4. It also shows the current Endrias Bridge version and the effective config file path.

FieldDescription
Config filePath to migrationbridge.conf. Defaults to the file in the install folder. Use Browse to point at an alternate config — useful when managing multiple VM templates with different settings.
💡
Click Next to continue to Metadata Source.

2️⃣ Step 2 — Metadata Source

Select where Endrias Bridge reads VM metadata (hostname, keys, network config, userdata) at boot time. Use "No metadata" for bare-metal or isolated VMs.

Choose where Endrias Bridge should read VM metadata from at boot time.

OptionWhen to use it
ConfigDrive v2A ConfigDrive ISO or HDD partition is attached to the VM (common with OpenStack, Proxmox, oVirt, Hyper-V templates). Reads openstack/latest/meta_data.json.
OpenStack HTTPThe VM can reach the link-local metadata service at 169.254.169.254/openstack/latest (OpenStack Nova / DevStack).
Amazon EC2Running on AWS EC2 or an EC2-compatible cloud — reads 169.254.169.254/latest.
No metadataBare-metal or isolated VM with no IMDS. Endrias Bridge uses only the values in migrationbridge.conf (and the User Settings / Plugins you configure in the wizard).

Below the options, set the HTTP base URL (for OpenStack HTTP), and the retry count / interval used when probing a metadata endpoint.

⚠️
If no metadata source is reachable, Endrias Bridge still runs all enabled plugins with empty metadata — they log "skipping" and continue rather than aborting the run.

3️⃣ Step 3 — User Settings

Configure the local Windows account Endrias Bridge creates on first boot. This is a local account only — not a domain join.

Configure the local Windows account that Endrias Bridge creates on first boot.

FieldDescription
UsernameLocal account name to create (default: Admin).
Groups (comma-separated)Local groups to add the account to (default: Administrators).
Generate random password if none in metadataIf the metadata source provides no password, generate a cryptographically secure random 16-character password.
Allow automatic reboot after hostname changeLet SetHostnamePlugin reboot the VM automatically when the hostname is changed.
Enable debug loggingSwitch the log level to DEBUG for this run — captures verbose internal state useful for troubleshooting.
⚠️
Endrias Bridge creates a local Windows account only. It does not join the VM to an Active Directory domain and does not migrate AD users, groups, or Group Policy. Domain-joined environments must provision domain accounts separately.

4️⃣ Step 4 — Plugins

Select which guest-initialization tasks run on this VM. Use Select All / Clear All to batch-toggle all plugins.

Choose which guest-initialization tasks run on this VM. Use Select All / Clear All to quickly toggle everything.

💻 SetHostnamePlugin

Set hostname from metadata (triggers reboot if Allow reboot is enabled)

👤 CreateUserPlugin

Create local Windows user and assign to specified local groups

🔑 SetPasswordPlugin

Inject password from metadata or generate a secure random 16-character password

🌐 NetworkConfigPlugin

Configure static IP address, gateway, and DNS resolver

🗝 InjectSSHKeysPlugin

Write SSH public keys to the user's authorized_keys file

📜 ExecuteUserdataPlugin

Execute PowerShell / CMD / Bash userdata scripts from metadata

ℹ️
Five additional plugins — ExtendVolumesPlugin, SetTimezonePlugin, NTPClientPlugin, WinRMListenerPlugin, and LocalScriptsPlugin — run automatically (configured via migrationbridge.conf) but are not yet shown as toggles on this screen. See the SOP's Plugins Reference for details.

5️⃣ Step 5 — DB Migration 🕙 Typically 2–3 min to configure

Configure source and target database connections. Click "Test Connection" for each before opening the Endrias Bridge. If this VM needs no database migration, check "Skip migration setup."

This step configures the source and target servers/databases for a database migration, and provides access to the full Endrias Bridge. If this VM only needs guest initialization, check Skip migration setup and continue to Review.

BlockFieldsPurpose
Source ServerPlatform, Host/IP, Port, Username, PasswordThe OS-level host running the source database. Platform options: Windows, Linux, VMware vCenter, Hyper-V, AWS EC2, Azure VM, GCP Compute, Other — selecting one auto-fills the default port.
Source DatabaseDB Type, Host/IP, Port, Database, Username, PasswordThe source database connection. DB Type options: SQL Server, PostgreSQL, MySQL, Oracle, SQLite — selecting one auto-fills the default port (1433 / 5432 / 3306 / 1521).
Target ServerSame fields as Source ServerThe OS-level host for the migration target.
Target DatabaseSame fields as Source DatabaseThe target database connection. For Azure SQL and AWS RDS targets, TLS 1.2 is always enforced — certificate validation behavior is configurable per connection profile.
EncryptionEncrypt / TrustServerCertificate togglesAzure SQL and AWS RDS connections always use Encrypt=yes, TLS 1.2. Set TrustServerCertificate=no in production to validate the server certificate against the trusted CA chain. For on-premises SQL Server with a self-signed cert, TrustServerCertificate=yes is acceptable during migration but should be replaced with a valid cert post-cutover.

Each Server / Database block has a Test Connection button — use it to confirm credentials and reachability before opening the Endrias Bridge.

ℹ️
PaaS SQL Authentication requirement: When AWS RDS for SQL Server or Azure SQL Database is selected as the DB Type, a blue banner automatically appears below the credentials fields as a reminder: SQL Authentication is required for AWS RDS and Azure SQL Database — Windows Authentication (Integrated Security) is not supported on PaaS SQL endpoints. Use a SQL login (username + password) rather than Windows / Integrated Security for all RDS and Azure SQL connections.
💡
Click Open Endrias Bridge at any time to launch the dedicated assessment / migration window. See the Assessment Tab and Migration Tab sections below.
ℹ️
Use the Connection String Debug Panel (accessible in Step 5) to inspect the exact ODBC connection string being built — useful when diagnosing authentication failures with Azure AD or certificate-based auth.

6️⃣ Step 6 — Review

Verify every configuration choice before committing. This is the last opportunity to go back and change settings without re-running plugins.

A scrollable summary of every choice made so far, grouped by section:

SectionWhat is shown
MetadataSource type, HTTP URL, retry count/interval
UserUsername, groups, inject password / allow reboot / debug logging flags
PluginsEnabled / Disabled status for each of the six toggleable plugins
Source & TargetEither "Skipped (VM guest init only)", or the full source/target platform, server, and database connection summary from Step 5
Config FileThe path that will be used / written when you click Run
⚠️
Review this screen carefully — clicking Next advances to Step 7. Once the run begins, guest initialization operations (hostname rename, user creation) apply immediately and may trigger a reboot.

7️⃣ Step 7 — Run

Click Run to apply the configuration. A live color-coded log streams in real time. When finished, the Next button becomes Close.

Click Run to apply the configuration. A live, color-coded log streams into the window as Endrias Bridge executes each plugin:

08:00:01 INFO migrationbridge.init_manager: Endrias Bridge initializing... 08:00:01 INFO migrationbridge.init_manager: Running plugin: SetHostnamePlugin 08:00:02 WARNING migrationbridge.plugins.set_timezone: Timezone not specified, skipping 08:00:03 ERROR migrationbridge.plugins.network_config: Adapter "Ethernet2" not found 08:00:04 DEBUG migrationbridge.plugins.execute_userdata: Detected script type: #ps1 08:00:05 INFO migrationbridge.init_manager: All plugins completed.
IndicatorMeaning
Amber — "Starting..."Run in progress.
Green — "Completed successfully!"No ERROR-level messages were logged.
Red — "Completed with errors"At least one ERROR-level message appeared — check the log above, then click Open Error Log Folder.
💡
The Open Error Log Folder button (top-right of this step) opens the error log directory in File Explorer. See Logs & Error Logs for log file paths and rotation details.

Pre-Flight Connectivity Check

Endrias Bridge runs a 3-step pre-flight check automatically when you click Run Migration. If source or target fails, the migration does not start.

When you click Run Migration, Endrias Bridge automatically executes a 3-step pre-flight check before any data movement begins. The check validates connectivity, permissions, and CDC eligibility. Results appear at the top of the migration log.

Pre-Flight Steps

1
Test source connection
Connects to the source, retrieves server version and table count. Reports OK with version string and object count, or FAILED with TCP / auth error detail.
2
Test target connection
Connects to the target, identifies the engine type, and checks whether the target database already exists (will be created if not). Reports OK or FAILED.
3
CDC eligibility check
Only runs if CDC Stream mode is selected. Verifies CDC is enabled on the source database and that SQL Server Agent is running. Issues WARN if CDC is not enabled — migration does not start in CDC Stream mode; switch to Full or Incremental.

Example Pre-Flight Log Output

[preflight] ✓ Source: SQL Server 2019 (15.0.2000) — 154 tables detected [preflight] ✓ Target: Azure SQL Database — database 'prod_target' will be created [preflight] ⚠ CDC: not enabled on source — stream mode unavailable (switch to Full or Incremental)
🚫
If Step 1 (source) or Step 2 (target) reports FAILED, the migration does not start. Resolve the connectivity issue — verify firewall rules, security group inbound rules, ODBC driver installation, and credentials — then click Run Migration again.

Common Pre-Flight Failures and Remediation

FailureLikely causeRemediation
Source TCP timeoutFirewall / security group blocking port 1433 from staging hostAdd an inbound rule allowing the staging host IP to port 1433 on the source server's firewall and OS firewall
Source auth failureSQL login disabled, wrong password, or Windows auth mismatchTest with SQL Server Management Studio from the staging host; verify login is enabled and CHECK_POLICY is not locking it
Target: database engine mismatchWrong DB Type selected for the targetRe-check the DB Type dropdown in Step 5; confirm the target endpoint type (e.g., Azure SQL vs. SQL Server on VM)
CDC: SQL Agent not runningSQL Server Agent service stopped on sourceStart SQL Server Agent: net start SQLSERVERAGENT or via SQL Server Configuration Manager

🔎 Connection String Debug Panel

In Step 5, click "Show connection string" to see the exact ODBC string being built — password masked. Use the Copy button to paste into external diagnostic tools.

On Step 5, a collapsible Show connection string control (collapsed by default) displays the exact ODBC connection string that Endrias Bridge constructs from the current field values. The password field is masked as ****. A Copy button copies the string to the clipboard.

Example Debug Output

DRIVER={ODBC Driver 17 for SQL Server}; SERVER=prod-sql-01.database.windows.net,1433; DATABASE=Acme Corp_Prod; UID=migrator_svc; PWD=****; Encrypt=yes; TrustServerCertificate=no; Connection Timeout=30; ApplicationIntent=ReadWrite;

Common Uses

ℹ️
The debug panel is read-only. To change the connection string, modify the fields in the Step 5 form and re-open the panel. The string rebuilds in real time as you type.

💾 Connection Profiles

Save and reuse named connection profiles for frequently-used source and target databases. Passwords can be stored in Windows Credential Manager.

The Source Database and Target Database blocks in Step 5 each have a Connection profile row, similar to saved connections in Azure Data Studio.

  1. Save As...

    Fill in DB Type, Host, Port, Database, Username (and optionally Password), then click Save As... and provide a profile name. You will be prompted whether to store the password in the Windows Credential Manager.

  2. Load

    Pick a saved profile from the dropdown and click Load to populate DB Type, Host, Port, Database, Username, and Password (if saved).

  3. Delete

    Select a profile from the dropdown and click Delete to remove it permanently (confirmation dialog shown).

ℹ️
Profiles are stored in %APPDATA%\Endrias Bridge\connection_profiles.json. Saving a password requires the optional keyring package — if it is not installed, profiles are saved without a password and the field is left blank when loaded.

🔎 Assessment Tab

Run Assessment to enumerate source objects, flag compatibility issues, and receive a sizing recommendation — before moving any data.

Open the Endrias Bridge from Step 5, then click Run Assessment. This connects to the source database, lists its objects in the Source Objects tree (with row counts), and compares the source/target engine pair against the compatibility matrix in the Compatibility Issues panel.

Source → TargetExample issues flagged
SQL Server → PostgreSQLIDENTITY → SERIAL/GENERATED ALWAYS AS IDENTITY, DATETIME → TIMESTAMP, NVARCHAR(MAX) → TEXT, BIT → BOOLEAN, TOP N → LIMIT N, stored procedures need rewriting in PL/pgSQL
SQL Server → MySQLIDENTITY → AUTO_INCREMENT, NVARCHAR → VARCHAR (UTF8MB4), TOP N → LIMIT N, clustered index behavior differs
PostgreSQL ↔ MySQLSERIAL ↔ AUTO_INCREMENT, BOOLEAN ↔ TINYINT(1), LIMIT/OFFSET syntax differences, JSONB/array support
Any other pairGeneral checklist: review data type mappings, stored procedure/function syntax, index & constraint naming, NULL handling and defaults
💡
If Microsoft Data Migration Assistant (DMA) is installed, click Launch Microsoft DMA in the header for a deeper SQL Server compatibility and feature-parity assessment targeting SQL Server on-prem, Azure SQL Database, or Managed Instance.

Data Type Conversions

When source and target are different engines, the Compatibility Issues panel also lists a Data type conversions section — a column-by-column report of source types that do not exist on the target and what they become:

Source typeConverted to
SQL Server UNIQUEIDENTIFIERCHAR(36) string
SQL Server MONEY / SMALLMONEYNUMERIC(19,4) / NUMERIC(10,4)
SQL Server NVARCHAR / NCHAR / DATETIME / TINYINTVARCHAR / CHAR / TIMESTAMP / SMALLINT
SQL Server XML / SQL_VARIANT / NTEXTTEXT
SQL Server hierarchyid / geography / geometryTEXT (read via .ToString() / .STAsText())
PostgreSQL UUIDCHAR(36) string
PostgreSQL JSON / JSONB / ARRAYTEXT (JSON-encoded)
MySQL ENUM / SETVARCHAR(255)
Oracle ROWID / LONG / BFILEVARCHAR(32) / TEXT / VARCHAR(255)
ℹ️
This is a preview — the same conversions are applied automatically when you run the migration. Any value that is truncated or rounded during conversion is reported as a WARNING line in the Migration Log with a per-column row count.

🔄 Migration Tab

Migrate schema and/or data from source to target. Choose sync mode, load the table list, and click Start Migration. Resume picks up from the last committed checkpoint.

Migrate schema and/or data from the source to the target database.

When to Use Which Sync Mode

ModeBest forRequirementsLatency
Full copyInitial migration, periodic full refresh, small/medium databasesNone — works on all enginesN/A (one-time)
Incremental (watermark)Ongoing delta sync on any engine; tables with a reliable updated_at / row version columnA monotonically increasing watermark column (datetime, rowversion, integer)Configurable interval (minutes to hours)
CDC (SQL Server)Near-zero-downtime cutover; single-pass apply of accumulated changesCDC enabled on source DB; SQL Server Standard/Enterprise or Azure SQL DB tier that supports CDCSingle pass at cutover time
CDC StreamContinuous live replication during cutover window; target stays current while source is still liveSame as CDC above; SQL Agent running; < 64 KB LOB columns (Azure SQL limitation)< 1 second end-to-end
CT SyncDelta sync using SQL Server Change Tracking — lighter-weight than CDC; records which rows changed, not old values. Best for delta refresh after a Full copy seed.CT enabled on source DB and each table (see below); SQL Server 2016+, Azure SQL DB, or AWS RDS for SQL ServerSingle pass at scheduled interval; run continuously via API/CLI
  1. Choose what to migrate

    Select Migrate schema (CREATE TABLE) and/or Migrate data (INSERT rows) — both are enabled by default. Uncheck schema to do a data-only refresh into an existing target schema.

    The third checkbox — Verify after migration (row counts + checksums) — is also enabled by default. When checked, immediately after all data is copied the tool runs a post-migration integrity check for every table: it compares COUNT(*) on source vs target, then samples up to 200 rows (ordered by PK) and computes an MD5 checksum of each row's columns. Mismatches, missing rows, and row count differences are listed in the log and included in the exported report. Uncheck it to skip verification and shorten total run time on very large databases (you can always run Verify Data Integrity manually later).

  2. Choose a sync mode

    Full copy — truncate and recreate, then copy all rows. Automatically uses BCP bulk export + BULK INSERT when bcp.exe is on PATH, falling back to PK-range copy otherwise. Indexes are disabled before load and rebuilt after (2–4x faster INSERT). Incremental — copy only rows newer than the last run, using a watermark column per table. CDC — single-pass apply of changes since last run via SQL Server native CDC. CDC Stream — continuous near-real-time replication, polling the transaction log LSN every 250 ms, applying changes within < 1 second. Runs until you click Stop Stream. Requires CDC enabled on source DB and each tracked table. CT Sync — delta sync using SQL Server Change Tracking (see section below).

  3. Load Tables from Source

    Populates the table list on the left with all user tables. Check the tables to migrate. Use the filter box to search by name in large schemas.

  4. Start Migration

    Creates the target database if it does not exist, then copies schema and data in batches (default 100,000 rows; set db_migration_chunk_size in migrationbridge.conf to adjust). For LOB-heavy tables (nvarchar(max), varbinary(max), xml) the chunk size is automatically reduced to keep each batch under 256 MB — preventing out-of-memory errors on wide-row tables. For same-engine migrations, procedures / views / triggers / functions are applied to the target automatically. If any table's DDL is rejected by the target, schema creation retries table-by-table so a single unmapped type does not roll back the entire schema.

  5. Parallel copy lanes (large vs. small tables)

    Two controls: Table concurrency (default 4 workers) and Large table threshold (default 10,000,000 rows). Tables below the threshold run concurrently in the fast lane so small lookup/reference tables complete without waiting for billion-row fact tables. Tables at or above the threshold run sequentially in the slow lane after the fast lane finishes. Set concurrency to 1 when resuming a large interrupted migration to avoid checkpoint collisions.

  6. PK-range checkpoint seeding with transient-error auto-retry

    Every table with a single INTEGER or BIGINT primary key uses PK-range seeding instead of OFFSET paging. After every committed batch the last PK value is saved to migrationbridge_sync_state.json. If the migration is interrupted (network drop, OS sleep, crash), clicking Resume Migration continues from the exact last committed PK — no rows are deleted, no progress is lost.

    Transient TCP errors (Azure SQL connection resets, error 10054 / 08S01) are automatically retried up to 5 times per batch with exponential back-off (5 s → 15 s → 30 s → 60 s → 120 s). The connection pool is flushed between attempts so a fresh TCP connection is always used.

    The progress bar and ETA on Resume reflect the full database — row counts from skipped (already-complete) tables are included so the bar starts at the correct percentage (e.g. ~80% if 80% of rows are already on the target).

  7. Index disable / rebuild for faster bulk INSERT

    Before data copy begins, all non-clustered non-unique indexes on the target tables are automatically disabled (SQL Server targets) or dropped (PostgreSQL / MySQL targets). This removes per-row B-tree maintenance overhead during INSERT — typically 2–4x faster for large tables. Indexes are rebuilt / recreated automatically in a single pass after all rows are loaded. The clustered index (PK) and unique indexes are never touched. Watch the log for [idx] DISABLED and [idx] REBUILT lines.

  8. Keeping the VM active during migration

    For long-running migrations, prevent the staging VM from sleeping or signing out. Run the following in an Administrator Command Prompt before starting:

    powercfg /change standby-timeout-ac 0
    powercfg /change monitor-timeout-ac 0

    If connecting via RDP, use tscon <sessionid> /dest:console to detach without logging off — the migration process stays alive. Restore sleep settings after migration completes.

⚠️
CDC sync mode requires SQL Server Standard/Enterprise (or Azure SQL DB tiers that support Change Data Capture) — SQL Server Web Edition and some low-end Azure SQL DB tiers (Basic, Standard S0–S2) do not support CDC. Use Full copy or Incremental on those editions. Additionally, Azure SQL Database silently truncates LOB columns > 64 KB via native CDC — the tool automatically downgrades affected tables to watermark sync to prevent this.
💡
BACPAC Seed for very large tables (1 TB+): The BACPAC Seed (Large Tables) button uses Microsoft's SqlPackage CLI to export and import a single table as a .bacpac file — the same mechanism the Azure Portal uses for database exports. Use this for tables that would take days to copy row-by-row.

CT Sync — SQL Server Change Tracking

Change Tracking (CT) is a lighter-weight alternative to CDC. Instead of recording old column values in the transaction log, CT records only which rows changed (PK + operation: Insert / Update / Delete). This means:

Prerequisites

Run the following T-SQL on the source database before selecting CT Sync mode:

-- Enable Change Tracking on the database (5-day retention)
ALTER DATABASE [YourDatabase]
SET CHANGE_TRACKING = ON (CHANGE_RETENTION = 5 DAYS, AUTO_CLEANUP = ON);

-- Enable on each table you want to track
ALTER TABLE [dbo].[YourTable]
ENABLE CHANGE_TRACKING WITH (TRACK_COLUMNS_UPDATED = OFF);

To verify CT is on:

-- Database-level
SELECT name, is_change_tracking_on FROM sys.databases WHERE database_id = DB_ID();

-- Table-level
SELECT s.name + '.' + t.name AS tbl, ct.min_valid_version
FROM sys.change_tracking_tables ct
JOIN sys.tables  t ON ct.object_id = t.object_id
JOIN sys.schemas s ON t.schema_id  = s.schema_id;

Typical Workflow

  1. Perform a Full copy migration first (schema + data) — CT Sync does not copy initial data.
  2. Switch sync mode to CT Sync and click Start Migration to apply deltas since the last known CT version.
  3. Repeat periodically (scheduled task, CI/CD pipeline, or manually) to keep the target current.
  4. At cutover: do one final CT Sync, quiesce writes to the source, do a last CT Sync, then switch the application to the target.
💡
For continuous near-real-time replication using Change Tracking, call ct_tail_loop() from the CLI or API in a background thread. See the CLI section below.

CLI Headless Mode — Pipeline & Automation

Run migrations without opening the GUI — ideal for Azure DevOps pipelines, PowerShell scripts, and nightly jobs. The migrate subcommand accepts the same connection, table, and mode options as the GUI.

Basic syntax

MigrationBridge.exe migrate ^
    --src-host myazure.database.windows.net ^
    --src-db   MyDB ^
    --src-user sa ^
    --src-pass MyP@ssword ^
    --src-type "Azure SQL Database" ^
    --tgt-host myrds.rds.amazonaws.com ^
    --tgt-db   MyDB ^
    --tgt-user admin ^
    --tgt-pass MyP@ssword ^
    --tgt-type "AWS RDS for SQL Server" ^
    --mode full ^
    --verify ^
    --report   C:\Reports\MyDB_migration.html

Available flags

FlagDefaultDescription
--src-hostrequiredSource hostname or IP
--src-port1433Source TCP port
--src-dbrequiredSource database name
--src-userrequiredSource SQL login username
--src-passrequiredSource SQL login password
--src-typeAzure SQL DatabaseSource engine type
--tgt-hostrequiredTarget hostname or IP
--tgt-port1433Target TCP port
--tgt-dbrequiredTarget database name
--tgt-userrequiredTarget SQL login username
--tgt-passrequiredTarget SQL login password
--tgt-typeAWS RDS for SQL ServerTarget engine type
--tablesall tablesComma-separated list of tables to migrate
--modefullfull, incremental, or cdc
--no-schemaschema ONSkip CREATE TABLE (data-only refresh)
--no-datadata ONSkip data copy (schema-only run)
--verifyOFFRun row-count + checksum verification after copy
--concurrency4Parallel table workers
--report FILEnoneWrite HTML (or .txt) migration report to FILE

Exit codes

CodeMeaning
0All tables migrated successfully (and verified, if --verify was set)
1One or more tables errored, or a fatal exception occurred

Azure DevOps pipeline snippet

- task: PowerShell@2
  displayName: 'Migrate MyDB to RDS'
  inputs:
    targetType: inline
    script: |
      & 'C:\MigrationTool\MigrationBridge.exe' migrate `
        --src-host $(SRC_HOST)  --src-db $(SRC_DB) `
        --src-user $(SRC_USER)  --src-pass $(SRC_PASS) `
        --src-type "Azure SQL Database" `
        --tgt-host $(TGT_HOST)  --tgt-db $(TGT_DB) `
        --tgt-user $(TGT_USER)  --tgt-pass $(TGT_PASS) `
        --tgt-type "AWS RDS for SQL Server" `
        --mode full --verify `
        --report $(Build.ArtifactStagingDirectory)\migration_report.html
      if ($LASTEXITCODE -ne 0) { throw "Migration failed" }

- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'migration-report'
⚠️
Store connection credentials in Azure DevOps variable groups or secrets, not in pipeline YAML. Pass them as $(VAR_NAME) references as shown above.

Phase 3 Features

  1. Parallel streams per large table

    The Parallel streams per large table spinbox (default: 1) splits a table's primary key range into N segments copied simultaneously. Set to 4 for tables between 400 M and 1 B rows, or 8 for 1 B+ row tables. Each segment has its own checkpoint — if interrupted, only the affected segment resumes. Leave at 1 to use BCP (fastest for smaller tables). Available for any source/target pair with a single integer PK.

  2. Type mapping overrides (migrationbridge.conf)

    Add a [type_overrides] section to override how specific source column types map to the target:

    [type_overrides]
    geography        = NVARCHAR(4000)
    uniqueidentifier = VARCHAR(36)
    xml              = TEXT

    Overrides apply before all built-in rules. Restart the tool after editing the file to clear the internal type cache.

  3. Azure SQL → AWS RDS: sysname columns

    Azure SQL allows user tables to have columns of type sysname (a system alias for nvarchar(128)). AWS RDS does not permit user-created sysname columns. The migration tool automatically remaps these to NVARCHAR(128) before emitting DDL — no manual action required.

  4. System-versioned temporal tables

    Temporal tables and their history tables always migrate as plain tables — all row data comes across. What happens to system-versioning after migration depends on the target engine:

    • SQL Server / Azure SQL target — system-versioning is re-enabled automatically after data copy. Reported in the log as system-versioning ON. If the target already had versioning ON from a prior run, it is turned OFF before the copy and re-enabled after.
    • Oracle 23ai+ target — the Schema Objects tab generates an ALTER TABLE … ADD PERIOD FOR SYSTEM_TIME … ADD VERSIONING USE HISTORY TABLE script. Run this after migration to wire up the already-migrated history table.
    • Oracle 11g R2–21c target — the Schema Objects tab generates a Flashback Data Archive (FDA) script. Run it after migration to enable automatic history tracking going forward.
    • PostgreSQL / MySQL target — the Schema Objects tab generates a trigger-based emulation pattern. No native system-versioning equivalent exists on these engines.

    Period columns (SysStart, SysEnd, ValidFrom, ValidTo) are migrated as plain TIMESTAMP / DATETIME2 columns on non–SQL Server targets. The migration log flags each one with a pointer to the Schema Objects tab.

  5. Azure SQL → AWS RDS: DateTime.MaxValue / DateTime.MinValue

    SQL Server applications often store 9999-12-31 23:59:59.9999999 (DateTime.MaxValue) as a "never expires" sentinel. ODBC Driver 17's fast_executemany batch path cannot bind fractional-second values at the 9999-12-31 boundary (error 22007). The tool automatically clamps these to 9999-12-31 23:59:59.000000. Similarly, DateTime.MinValue (0001-01-01) is clamped to 1753-01-01 for legacy datetime columns.

  6. Self-referential FK constraints

    Tables where a column references the primary key of the same table cannot be bulk-copied while the FK constraint is active — parent rows may not yet exist when a child row in the same batch is inserted. The tool automatically detects self-referential FKs, disables them on the target before copying, and re-enables them with validation after all rows are present.

  7. Multi-schema databases with identically-named tables

    Databases with the same table name in multiple schemas (e.g. errordata.ClaimHeader and historydata.ClaimHeader) now use schema-qualified keys (schema.name) throughout the FK wave scheduler, table_totals, _est_rows, and copy engine dictionaries. No configuration change required — the fix is automatic.

  8. Spatial column export (geography / geometry)

    SQL Server geography and geometry columns are automatically exported as WKT (Well-Known Text) strings using .STAsText(). Raw WKB bytes returned by some ODBC driver versions are hex-encoded. No additional configuration required.

  9. MongoDB & Azure Cosmos DB (MongoDB API) source / target

    Install pymongo (pip install pymongo) to migrate MongoDB collections or Azure Cosmos DB databases. Select MongoDB or Cosmos DB as the engine type. For Cosmos DB, enter your account hostname (<account>.mongo.cosmos.azure.com), port 10255, account name as username, and primary key as password. SQL Server columns typed as hierarchyid or geography are automatically skipped with a per-table log message. Supports Atlas, on-premises, and Cosmos DB MongoDB API. Do not install azure-cosmos — this tool uses the MongoDB wire protocol exclusively.

  10. Amazon DynamoDB source

    Install boto3 (pip install boto3) to migrate DynamoDB tables to any relational target. Provide AWS region and credentials on the source screen. The tool paginates through the full table scan and infers schema from item samples. Supports IAM roles, environment variable credentials, and DynamoDB Local for testing.

  11. Live web dashboard

    Install fastapi and uvicorn (pip install fastapi "uvicorn[standard]") to enable the built-in web dashboard. When a migration starts, open http://localhost:8765 in any browser to see live progress, per-table status bars, and a color-coded log panel — without RDP access to the staging VM. The dashboard auto-refreshes every 2 seconds. The REST endpoint GET /api/status can be polled by CI/CD pipelines to await migration completion.

  12. Docker / headless deployment

    Run Endrias Bridge in a Docker container for Linux or cloud-native deployments. The included Dockerfile and docker-compose.yml install the Microsoft ODBC driver automatically. Mount migrationbridge.conf as read-only and ./migration_state for checkpoint persistence:

    docker compose up migration
    # Dashboard: http://localhost:8765

    In Docker, bcp.exe is unavailable (Windows-only) — set Parallel streams to 4–8 for best throughput.

📄 Jobs Tab — Concurrent Multi-DB Migration

Run multiple source-to-target migrations in parallel with per-job target overrides. Each job shows live progress and produces its own log. When all jobs finish, export a combined summary report.

The Jobs tab (also called Migration Projects) is designed for bulk migrations where several source databases need to be moved at the same time — for example, migrating a fleet of tenant databases to a new RDS instance.

💡
Opening from the wizard: Clicking Open Migration Projects from Step 5 of the Setup Wizard automatically pre-populates all source and target connection fields (DB Type, Host, Port, Database, Username, Password) from the wizard's Step 5 entries — no need to re-enter credentials.

Step-by-Step

  1. Configure a default Source and Target connection on Step 5 of the wizard (these become the defaults for every job). Opening Migration Projects from Step 5 pre-fills all credential fields automatically.
  2. Open the Jobs tab.
  3. Click + Add Job for each database you want to migrate. Each row shows:
    • Source DB — the database name on the source server
    • Target DB — the database name on the target (can differ from source)
    • Target Override — optionally point this job at a different target server
    • Status — Pending / Running / Done / Error
  4. In the Map Targets step (Step 3), verify the "Auto-create target database if it does not exist" checkbox. This is checked by default and is recommended for RDS and Azure SQL Database targets — the engine issues CREATE DATABASE automatically before migration, so the database does not need to be pre-provisioned.
  5. Set Concurrency (1–10) — how many jobs run at the same time. Start with 2–3 to avoid saturating the network.
  6. Click Run All. Each job bar shows live row counts and percentage. The engine performs real schema + data migration: SQLAlchemy reflection, CREATE TABLE DDL, 10,000-row chunk copy with live progress, and FK constraint application after all tables are loaded.
  7. Use Stop to abort all running jobs cleanly.
  8. When finished, click Export Combined Report to save a CSV summarising all jobs: rows copied, errors, duration.

Auto-Create Target Database

Migration Projects automatically creates the target database before migration begins — no manual pre-provisioning is required. The engine issues CREATE DATABASE [name] on the target server. If the database already exists the command is silently skipped. This is the default behavior for all AWS RDS and Azure SQL Database targets.

ℹ️
The auto-create log entries appear as: [preflight] Created target database: <name> or [preflight] Target database already exists — skipping CREATE. To disable auto-create for a specific job, uncheck the "Auto-create target database if it does not exist" checkbox in the Map Targets step.
ℹ️
Each job runs the full production migration engine — SQLAlchemy schema reflection, type-mapped DDL, FK parent gap-fill, PK-range seeding, transient-error retry, PK checkpoint resume, and index disable/rebuild. All sync modes (Full, Incremental, CDC) and phase features apply per job. Chunk size is controlled by db_migration_chunk_size in migrationbridge.conf (default 100,000 rows/batch).
ℹ️
True parallel execution: Each job runs in its own independent thread with its own database engines, connection strings, and log file. Jobs do not share GUI state — starting job B never overwrites job A's connection settings. Each job produces a timestamped log file in the error log folder (<db>_YYYYMMDD_HHMMSS.log). FATAL/ERROR lines are also pushed to the Error Log tab.
High concurrency against a shared source server can cause blocking. Monitor source CPU and wait stats during the run. Reduce concurrency if you see contention.

📊 Health Monitor Tab — Live Migration Dashboard

Real-time stat cards, donut chart, per-table progress bars, throughput sparkline, system resource monitors (CPU/Memory/Network/Disk), and live disk-space warnings — all updating every second.

The Health tab (labelled 📊 Health in the tab bar) is the first tab and is always visible. It updates live during migrations and retains final totals after a run completes. The entire tab body is scrollable — use the mousewheel to scroll all panels.

Stat Cards (Row 1)

CardWhat it shows
TABLES TOTALTotal tables selected for migration
COMPLETEDTables finished successfully (green)
IN PROGRESSTables currently being copied (amber)
ERRORS / WARNINGSTables with at least one error or warning (red)

Stat Cards (Row 2)

CardWhat it shows
TOTAL ROWS COPIEDCumulative rows written to target so far
ROWS / SEC (avg5)Throughput — rolling 5-sample average
ELAPSED TIMEWall-clock time since migration started (HH:MM:SS)
EST. TIME LEFTProjected time remaining based on current throughput

Live Table Dashboard

The table tree below the KPI cards shows one row per table with the following columns:

ColumnWhat it shows
TableSchema-qualified table name
StatusQueued / Running / Done / Skip / Warn / Error (colour-coded)
Rows Copiedcopied / total (pct%) e.g. 1,500,000 / 14,830,028 (10%). Row totals are pre-seeded before copy starts so the denominator is available from the first batch.
Rows/minPer-table copy throughput in rows per minute based on wall-clock time since the table started
ElapsedWall-clock time this table has been copying (e.g. 2m34s). Frozen at the final value once the table finishes.
ETA LeftEstimated time remaining for this table based on current throughput (e.g. 18m22s). Shows once complete.

Middle Panel

PanelWhat it shows
Donut ChartTable status breakdown: Queued / Running / Done / Error slices with legend
Per-Table Progress BarsOne bar per table with percentage complete. Scrollable with mousewheel.

Throughput Sparkline

Rows/sec over the last 60 seconds plotted as a line chart. Shows peak, average, and sample count. Updates every second.

Chart Hover Tooltips

All five chart canvases — Throughput, CPU %, Memory %, Net MB/s, and Disk MB/s — show a floating tooltip when the cursor hovers over the chart area. The tooltip displays the exact HH:MM:SS timestamp and the metric value at the hovered sample point (e.g. 14:32:07 — 847 rows/s or 14:32:07 — CPU 72%). A dashed vertical crosshair line tracks the cursor position. The tooltip disappears automatically when the cursor leaves the chart canvas.

System Resources Panel

Four live sparkline charts for the staging machine (requires psutil — installed automatically):

MetricDescription
CPU %Total processor utilisation across all cores (orange sparkline)
Memory %RAM utilisation as a percentage of total installed RAM (purple sparkline)
Net MB/sCombined network send + receive throughput in MB/s (cyan sparkline)
Disk MB/sCombined disk read + write throughput in MB/s (green sparkline)
ℹ️
If psutil is not installed, the resource panel shows an install hint. Run pip install psutil and restart the app.

Disk Space Bar (Top of Tab)

Always-visible row beneath the header showing free / total GB for both source and target connections. Refreshes every 30 seconds automatically; the ↻ button triggers an immediate refresh.

ScenarioDisplay
On-prem / RDS / Azure SQL MI2.4 GB free / 500 GB (99%) — C:\ [disk]
Azure SQL Database (PaaS)0.8 GB free / 4 GB (80%) — Azure SQL DB [DB space]
MySQL / PostgreSQLDisk monitoring N/A (DMV not available on non-SQL-Server engines)

Disk Warning Banner

A banner appears automatically above the stat cards when target disk space is low:

ConditionBanner colourMessage
Target free < 5 GBRed⚠ CRITICAL: Target disk critically low — X.X GB free on [drive]
Target free < 20 GB or < 15%Amber⚠ WARNING: Target disk low — X.X GB free on [drive] (Y% free)
Target healthyHiddenNo banner shown

Status Badge

The badge in the header bar shows: ● Idle (no migration running), ● Running (green, migration active), or ● Finished (blue, last run complete).

ℹ️
Switch to the Health tab at any point during a migration — it updates live regardless of which tab was active when the migration started.

🔁 Replication Tab

The Replication tab is separate from the Migration tab. It provides continuous or scheduled data synchronization — CDC Stream for near-real-time, Scheduled Sync for any engine.

The Replication tab is a dedicated panel for ongoing data synchronization, separate from the one-time (or resumed) migration workflow on the Migration tab. It supports two modes:

CDC Stream Near-Real-Time Replication

Polls the SQL Server transaction log at the current LSN every 250 ms. Applies INSERT / UPDATE / DELETE changes to the target within < 1 second end-to-end. SQL Server source only — requires CDC enabled on the source database and on each tracked table. Suitable for cut-over windows where the target must stay synchronized while production traffic continues on the source.

Scheduled Sync Watermark-Based Periodic Sync

Queries the source for rows newer than the last sync watermark (datetime or row version column) at a configurable interval. Works with any supported engine (SQL Server, PostgreSQL, MySQL, Oracle, SQLite). Suitable for data warehouse refresh patterns, reporting replicas, and environments where CDC is not available or practical.

Per-Table Dashboard

The Replication tab displays a live per-table dashboard showing:

ColumnDescription
Table nameSchema-qualified table name (e.g. dbo.Orders)
ModeCDC Stream or Scheduled Sync, as configured for that table
Rows syncedCumulative row change count since stream started
Last syncTimestamp of the most recent successful batch apply
Change rateChanges/second (rolling 60-second window) — warns if rate exceeds target apply capacity
LSN (CDC only)Current log sequence number being consumed on the source
StatusRunning / Lagging / Suspended

Schema Drift Alerts

If a column is added to or removed from a source table while the CDC stream is running, the affected table is automatically suspended and a WARN alert is logged. All other tables in the stream continue uninterrupted. To re-enable: stop the stream, apply the matching DDL change to the target, then restart the stream.

Sync History Log

The Replication tab maintains a scrollable sync history log showing each batch apply event with timestamp, table name, row count, LSN range (for CDC), and duration. The log persists to migration_logs\replication_<db>_<date>.log for post-incident analysis.

⚠️
CDC Stream lag alert: If the source is producing changes faster than the target can apply them, the tool logs a warning with the rolling change rate: rate=127/s — target apply lagging. If sustained, increase the staging host's CPU/memory, reduce table concurrency on competing workloads, or switch to a larger target tier.

📄 Replicas Tab — Read Replica Manager

Create, monitor, and promote read replicas on AWS RDS for SQL Server and Azure SQL Database — directly from the GUI without switching to the AWS Console or Azure Portal.

The Replicas tab is split into two sub-panels: AWS RDS and Azure SQL. Both require optional SDK packages (installed once):

pip install boto3                          # AWS RDS panel
pip install azure-mgmt-sql azure-identity  # Azure SQL panel
# or install both:
pip install migrationbridge[all]

AWS RDS Panel

ActionWhat it doesRequired permissions
Refresh InstancesLists all SQL Server RDS instances in the selected region with role (source / replica / standalone), status, AZ, instance class, and replica lag (seconds, via CloudWatch).rds:DescribeDBInstances, cloudwatch:GetMetricStatistics
Create New ReplicaProvisions an RDS read replica of the selected source. Supports same-region and cross-region. Cross-region replication uses the source ARN automatically.rds:CreateDBInstanceReadReplica
Promote to PrimaryPromotes the selected replica to a standalone DB instance (active when a replica row is selected). Use for cutover — after promotion the instance no longer receives changes from the former source.rds:PromoteReadReplica

Leave Access Key / Secret Key blank to use the standard boto3 credential chain (environment variables, ~/.aws/credentials, or EC2 instance role).

Azure SQL Panel

ActionWhat it doesRequired role
Refresh ServersLists Azure SQL servers and databases in the subscription. Shows secondary type and replication state for databases enrolled in active geo-replication.Reader on subscription
Create Geo-ReplicaCreates an active geo-replication secondary in a different Azure region. Uses create_mode=Secondary with the source database resource ID. Supports any GP / BC SKU.Contributor on target resource group
Failover (Promote)Triggers a planned failover on the selected database's first replication link — no data loss. The secondary becomes the new primary; the old primary becomes a secondary. Use for scheduled cutover or DR drills.SQL DB Contributor on secondary server

Leave Tenant / Client ID / Secret blank to use DefaultAzureCredential (Azure CLI, environment variables, managed identity). For automation, provide a Service Principal with the minimum roles above.

ℹ️
Forced failover (allow data loss) is not exposed in the GUI — it requires deliberate CLI or Portal action. The GUI always performs a planned failover which blocks until the secondary is fully caught up, guaranteeing zero data loss.

Export & Cloud Tab

Export source data to Amazon S3, Amazon Redshift, Azure Event Hubs, Azure Blob Storage, Snowflake, Google BigQuery, or Aurora PostgreSQL — and run AI-assisted schema compatibility analysis using Claude AI.

The Export & Cloud tab appears between Replicas and Error Log in the tab bar. Each destination is a self-contained card with its own fields and status line. All exports run in a background thread — the UI stays responsive while the export runs.

ℹ️
Prerequisite: Connect to a source database in the Assessment tab before using any export destination. The export engine reads directly from the connected source.

1 — Amazon S3 Export (Parquet / CSV / JSON)

Exports every source table to an S3 bucket in Parquet, CSV, or JSON format. Uses pyarrow for Parquet encoding. Supports partition-by-column (e.g. partition by year to create s3://bucket/table/year=2024/data.parquet sub-paths).

Install: pip install boto3 pandas pyarrow

FieldDescriptionExample
S3 BucketFull S3 URI including prefix paths3://my-bucket/exports/
AWS RegionRegion of the target bucketus-east-1
Formatparquet, csv, or jsonparquet
Partition ColOptional column name to partition by. Leave blank for no partitioning.created_year
AWS ProfileNamed profile from ~/.aws/credentials. Use default or leave blank for env/role auth.default
ℹ️
Leave Access Key / Secret Key fields blank to use the standard boto3 credential chain: environment variables (AWS_ACCESS_KEY_ID), ~/.aws/credentials, or EC2 / ECS instance role.

2 — Amazon Redshift Replication

Copies source tables into Amazon Redshift. Schema is auto-converted: NVARCHARVARCHAR, DATETIMETIMESTAMP, UNIQUEIDENTIFIERCHAR(36). Tables are created or replaced on each run.

Install: pip install sqlalchemy-redshift psycopg2 pandas

FieldDescription
Redshift HostCluster endpoint, e.g. mycluster.abc123.us-east-1.redshift.amazonaws.com
PortDefault 5439
DatabaseRedshift database name, e.g. dev
User / PasswordRedshift master user or IAM database user credentials
S3 Stage BucketS3 bucket used for COPY staging (large tables)
IAM Role ARNARN of the IAM role attached to the Redshift cluster with S3 read permissions

3 — Azure Event Hubs Streaming Sink

Streams every row from every source table as a JSON change event to an Azure Event Hub. Each event has the schema: { "op": "I", "table": "OrderHeader", "row": { "id": 1, ... } }. Designed to work alongside the Replication tab CDC stream — use for downstream consumers (Databricks, Stream Analytics, Azure Functions).

Install: pip install azure-eventhub

FieldDescription
Connection StringFrom Azure Portal → Event Hubs namespace → Shared access policies. Starts with Endpoint=sb://...
Event Hub NameThe specific hub (topic) within the namespace, e.g. migration-cdc-events
Partition Key ColOptional. Routes rows with the same value to the same partition — useful for ordering guarantees per entity.

4 — Azure Blob Storage Export

Exports source tables to Azure Blob Storage as Parquet, CSV, or JSON. Streams row batches directly to blob blocks — no local temp files. Supports SAS token or DefaultAzureCredential (Azure CLI login, managed identity, environment variables).

Install: pip install azure-storage-blob pandas pyarrow

FieldDescriptionExample
Storage Account URLBase URL of the storage accounthttps://myaccount.blob.core.windows.net
ContainerBlob container name (created if it doesn't exist)exports
Path PrefixPrefix prepended to each blob namemigration/2026/
Formatparquet, csv, or jsonparquet
SAS TokenPaste a SAS token for key-based auth, or leave blank to use DefaultAzureCredentialLeave blank for Azure CLI auth
ℹ️
To use DefaultAzureCredential, run az login in a terminal before launching Endrias Bridge, or assign a Managed Identity with Storage Blob Data Contributor role to the host VM.

5 — Zero-ETL Direct Load (Snowflake · BigQuery · Aurora PostgreSQL)

Loads source tables directly into a cloud analytics platform with no staging files. Schema is auto-detected and tables are created or replaced on each run. All source tables are loaded in sequence; status updates inline per table.

PlatformRequired packageAuth method
Snowflakepip install snowflake-connector-python pandasUsername + password, or key-pair
Google BigQuerypip install google-cloud-bigquery pandasService account JSON key file path, or ADC
Aurora PostgreSQLpip install psycopg2Username + password
FieldSnowflakeBigQueryAurora PG
Target PlatformSnowflakeBigQueryAurora PostgreSQL
Account / ProjectSnowflake account ID (e.g. xy12345.us-east-1)GCP project IDAurora hostname
Database / DatasetSnowflake database nameBigQuery datasetPostgreSQL database
Schema / DatasetSchema name (default PUBLIC)Same as datasetSchema (default public)
User / SA Key PathSnowflake usernamePath to service account .json keyPostgreSQL username
Password / TokenSnowflake passwordLeave blank (key file used)PostgreSQL password
WarehouseSnowflake virtual warehouse, e.g. COMPUTE_WHN/AN/A

6 — AI-Assisted Schema Compatibility Analysis

Scans up to 20 source tables and sends the schema metadata to Claude Haiku (Anthropic AI) for compatibility analysis. The AI returns a scored report for each table covering incompatible types, unsupported features, and recommended migration actions. Results open in a popup window.

Install: pip install anthropic

API Key: Set ANTHROPIC_API_KEY as an environment variable, or paste it directly into the field. Get a key at console.anthropic.com.

FieldDescriptionExample
Source TypeThe source database engineSQL Server
Target EngineThe target database engine to analyse againstPostgreSQL
TablesComma-separated list of table names to analyse. Leave blank to scan all tables (capped at 20).Orders, OrderItems, Products
API KeyAnthropic API key. Leave blank if ANTHROPIC_API_KEY env var is already set.sk-ant-...

The analysis report includes:

Schema metadata (column names and data types only — no row data) is sent to the Anthropic API. Do not use this feature if your column names contain sensitive identifiers that cannot leave your environment.

Azure SDK — Install Notice

The Azure SQL DB and Azure SQL MI sub-tabs in the Replicas tab, and the Azure Blob / Azure Event Hubs destinations in the Export tab, all require the Azure SDK. If not installed, a highlighted amber panel is shown:

Azure SDK not installed
Install the required packages, then restart Endrias Bridge:
    pip install azure-mgmt-sql azure-identity

Run the command in a terminal, then restart Endrias Bridge. The panel is replaced with a green Azure SDK ready badge once the packages are detected.

🧩 Schema Objects Tab

Lists stored procedures, views, triggers, functions, table types, temporal tables, and SQL Agent jobs. For same-engine migrations these are applied automatically — use this tab for manual export, cross-engine review, or error recovery.

Lists stored procedures, views, triggers, functions, table types, temporal tables, and SQL Agent jobs on the source database.

ℹ️
For same-engine migrations (SQL Server → SQL Server / Azure SQL, MySQL → MySQL, PostgreSQL → PostgreSQL), procedures, views, triggers, and functions are applied to the target automatically by the Migration tab — you only need this tab to apply them manually, export to .sql, or handle cross-engine migrations where DDL must be rewritten.
ButtonWhat it does
Scan Source ObjectsPopulates the Objects tree. Select an object to view its definition (or migration guidance) in the right-hand pane.
Export All to .sqlWrites every scanned object's definition to a single .sql file for review or manual application.
Apply to TargetExecutes the object definitions against the target database connection configured in Step 5.
Error Log tabEvery migration run streams errors and warnings to the Error Log tab in real time, and saves the full log to MigrationTool/migration_logs/migration_<db>_<timestamp>.log. The tab label updates to show the issue count when the run finishes.
Temporal TablesPopulates the Schema Objects tree with all temporal tables detected in the source — for inspection only. The actual 5-step re-enable sequence (drop defaults, drop period, fix SysEnd, add period, set SYSTEM_VERSIONING=ON) runs automatically at the end of Resume Migration.
Temporal tables — errors 13575 and 13537 (Change 53): Both errors are handled automatically by the current build. Error 13575 — clamped SysEnd values are restored to .9999999 before re-enabling versioning. Error 13537 — a DROP PERIOD FOR SYSTEM_TIME is issued before the SysEnd UPDATE to clear the GENERATED ALWAYS attribute. If you see either error, update to a build with Change 53 or later and click Resume Migration.

👤 Users & Logins Tab

Scan source database users, generate re-create DDL, compare with target, and apply selected users — all without touching the source database.
⚠️
Source is read-only. This tab only reads from the source. All writes go to the target only.

Summary Banner

After scanning, a summary banner appears at the top showing:

Users Found: 35  ·  ✅ Ready: 31  ·  ⚠ Manual: 3  ·  ❌ Unsupported: 1

User Types & Migration Status

TypeLabelMigration StatusNotes
SQL_USER (with login)SQL User✅ ReadyRequires matching login on target
SQL_USER (contained)Contained User✅ ReadyReplace <StrongPassword> in generated DDL
EXTERNAL_USERAzure AD User⚠ Manual Action RequiredSame Azure AD tenant only; otherwise use Portal
EXTERNAL_GROUPAzure AD Group⚠ Manual Action RequiredSame Azure AD tenant only
WINDOWS_USER / WINDOWS_GROUPWindows User / Group⚠ Manual Action RequiredDomain login must exist on target SQL instance
CERT / ASYMMETRIC KEYCert/Key User❌ UnsupportedMust be recreated manually via certificate transfer

Toolbar Buttons

ButtonAction
Scan Source UsersQueries sys.database_principals + sys.database_role_members on the source. Populates the user list with type, auth method, roles, and generated DDL.
Compare with TargetQueries the target DB and shows Missing / Exists / Role diff per user in the Compare tab. Updates the Target column in the main list.
Preview DDLShows full re-create DDL for all checked users at once in the DDL panel (dark code editor). Includes ALTER ROLE statements for role membership.
Apply Selected to TargetRuns DDL + ALTER ROLE for each checked user on the target. Updates per-row status to ✅ Applied or ❌ error.
Export DDL to .sqlSaves DDL for checked (or all) users to a .sql file. Includes role membership statements and header comments.
Select All / ☐Checks or unchecks all visible rows. Click individual ☑ cells to select specific users.

Filter Bar

Filter the user list by Type (SQL User, Azure AD User, Windows User, Contained User, External Group) and/or Status (Ready, Manual Action Required, Unsupported). Filters apply live as you change the dropdowns.

Compare with Target Tab

After clicking Compare with Target, the right-hand Compare tab populates with a side-by-side view:

ColumnDescription
User NameLogin name
Source TypeUser type on source
Source RolesComma-separated role memberships on source
In Target?✅ Exists or ❌ Missing
Target RolesRole memberships on target (if user exists)
DifferenceEmpty (match), Role diff, or Not in target

Azure AD Warning

⚠️
Azure AD users are tenant-specific. If the target SQL server is in a different Azure AD tenant, CREATE USER … FROM EXTERNAL PROVIDER will fail. Those users must be added through the Azure Portal instead. SQL-contained users and role memberships are always re-creatable across tenants.

🔎 Comparison Tab — Source ↔ Target Diff

Apple-to-apple comparison of tables, row counts, columns, constraints, indexes, views, procedures, functions, triggers, types, and statistics between source and target.

Use the Comparison tab after a migration to verify that the target is structurally and data-wise identical to the source, or to detect schema drift during ongoing replication.

Step-by-Step

  1. Connect both source and target on Step 5 of the wizard.
  2. Open the Comparison tab.
  3. Click Run Comparison. The engine queries both databases concurrently and builds a diff report.
  4. Use the Show filter bar to narrow results:
    • All — every object and attribute
    • Match — identical on both sides (green rows)
    • Mismatch — present on both sides but different (red rows)
    • Source only — exists on source but not target (amber rows)
    • Target only — exists on target but not source (blue rows)
  5. Click any row to inspect the Source value vs Target value.
  6. Click Export to .csv to save the full diff for change management records.

Comparison Categories

CategoryWhat is compared
TablesTable existence, row count, column count
ColumnsName, data type, nullability, default value, identity, computed
ConstraintsPrimary keys, foreign keys, unique constraints, check constraints
IndexesName, type (clustered/non-clustered), columns, uniqueness, fill factor
ViewsExistence and definition text
ProceduresExistence and definition text
FunctionsExistence and definition text
TriggersExistence, enabled/disabled state
TypesUser-defined table types and scalar types
StatisticsAuto-created statistics presence per table
ℹ️
Row counts are approximate for very large tables (SQL Server uses sys.partitions for speed). Run DBCC UPDATEUSAGE on both databases for exact counts before comparing.
💡
Run Comparison after every migration or replication window as part of your validation runbook. Export the CSV and attach it to your migration sign-off ticket.

🔨 Tools Tab

The Tools tab shows available migration utilities and provides access to the Configuration Health Check.

Shows which migration tools are available on this machine and provides the Configuration Health Check utility:

ToolPurpose
Microsoft Data Migration Assistant (DMA)Free Microsoft tool for assessing SQL Server databases before migrating to SQL Server on-prem, Azure SQL Database, or Managed Instance. Shows "Installed" with a launch button if found at C:\Program Files\Microsoft Data Migration Assistant\Dma.exe, otherwise shows a download link.
Built-in SQLAlchemy Migration EngineThe engine used by the Migration tab. Supports SQL Server, PostgreSQL, MySQL, Oracle, and SQLite. Shows whether SQLAlchemy is installed (pip install sqlalchemy if not).
Configuration Health CheckClick Run Configuration Health Check to validate the full deployment configuration before starting a migration. See the Configuration Health Check section for details.

📋 Configuration Health Check

Available from the Tools tab. Runs 8 automated verifications of your deployment configuration. Run this before every migration to confirm the environment is ready.

Available from the Tools tab. Click Run Configuration Health Check to validate the full deployment configuration before starting a migration. The check runs 8 automated verifications and reports each result with a specific remediation step if it fails.

Health Check Verifications

1
Source connection reachable (TCP + auth)
Opens a TCP socket to the source host/port, then authenticates. OK = connected and authenticated. Remediation: verify firewall rules and credentials using the Connection String Debug Panel.
2
Target connection reachable (TCP + auth)
Same check against the target. OK = connected. Remediation: confirm the target endpoint, security group inbound rules, and login credentials in Step 5.
3
Source database user permissions (SELECT, VIEW DATABASE STATE)
Runs a permissions probe on the source. OK = both permissions confirmed. Remediation: GRANT SELECT, VIEW DATABASE STATE TO [migrator_svc]; on the source database.
4
Target database user permissions (CREATE TABLE, INSERT, UPDATE, DELETE, ALTER)
Runs a permissions probe on the target. OK = all permissions confirmed. Remediation: GRANT CREATE TABLE, INSERT, UPDATE, DELETE, ALTER TO [migrator_svc]; on the target database.
5
CDC eligibility (SQL Server source only)
Checks sys.databases.is_cdc_enabled, sp_cdc_enable_db output, and whether SQL Server Agent is running. Only evaluated when CDC or CDC Stream mode is selected. WARN if CDC is not enabled — does not block migration if Full or Incremental mode is selected.
6
Disk space on staging host
Reads free disk space on the drive containing the checkpoint and log directories. WARN if < 20 GB free. Remediation: clear old checkpoint files (migrationbridge_sync_state_*.json) and old migration logs from migration_logs\.
7
ODBC Driver 17 installed
Checks the registry for ODBC Driver 17 for SQL Server. FAILED if not found. Remediation: download and install from Microsoft's ODBC Driver download page, or use the installer included in prereqs\.
8
bcp.exe on PATH (required for BCP acceleration)
Runs where bcp.exe. WARN (not error) if not found — migration continues using PK-range copy, which is slower for large tables. Remediation: install SQL Server Command Line Utilities (mssql-tools) and ensure the bin directory is on the system PATH.

Sample Health Check Output

[health] 1. Source connection ................. OK (SQL Server 2022, 15.0.4355) [health] 2. Target connection ................. OK (Azure SQL Database, GP S4) [health] 3. Source permissions ................ OK (SELECT, VIEW DATABASE STATE confirmed) [health] 4. Target permissions ................ OK (CREATE TABLE, INSERT, UPDATE, DELETE, ALTER) [health] 5. CDC eligibility ................... WARN (CDC not enabled on source — Full/Incr only) [health] 6. Disk space (staging) .............. OK (142 GB free on C:\) [health] 7. ODBC Driver 17 ................... OK (17.10.4.1 found) [health] 8. bcp.exe on PATH .................. WARN (not found — BCP acceleration unavailable)   Summary: 6 OK, 2 WARN, 0 FAILED — migration can proceed with Full or Incremental mode.

🏠 Multi-AZ / High Availability Topologies

Endrias Bridge supports HA source and target topologies — Always On AGs, RDS Multi-AZ, Azure SQL Failover Groups. Always connect via the listener or endpoint, not the instance name, to follow failover transparently.

Endrias Bridge supports high-availability migration architectures for both source and target databases. The key principle: always connect via the listener DNS name or managed endpoint rather than the individual instance name — this ensures Endrias Bridge follows a failover transparently without manual intervention.

HA Topology Support Matrix

TopologySource supportTarget supportNotes
SQL Server Always On AG Connect via AG listener Migrate to primary; secondary syncs via AG Listener DNS must resolve from the staging host. Use MultiSubnetFailover=Yes in the connection string for multi-subnet AGs.
RDS Multi-AZ Use RDS endpoint Failover transparent via RDS endpoint Use the cluster endpoint (e.g. db.xxxx.rds.amazonaws.com), not the instance endpoint. Multi-AZ standby syncs automatically via RDS-managed replication.
Azure SQL Failover Group Use read-write listener endpoint Geo-replica syncs automatically Use the .database.windows.net read-write listener endpoint. The geo-replica secondary receives all changes automatically via Azure SQL's built-in geo-replication.
Azure SQL Geo-Replication Connect to primary Secondary syncs after cutover No extra configuration required. The geo-replica secondary receives rows as they are inserted into the primary via Azure SQL's built-in replication channel.

Source HA: Reading from Always On AG or RDS Multi-AZ

Endrias Bridge reads from the primary replica only — it does not split reads across replicas or use the readable secondary. For Always On AGs, use the AG listener DNS name as the Host/IP in Step 5. For RDS Multi-AZ, use the RDS cluster endpoint.

ℹ️
During CDC Stream mode, the LSN stream is read from the primary. If a failover occurs while streaming, the stream will briefly disconnect (connection error will surface in the Replication tab), then auto-reconnect through the listener. The CDC stream resumes from the last committed LSN — no changes are missed provided the new primary was in sync before failover.

Target HA: Migrating to an AG Primary or RDS Multi-AZ

Migrate to the primary replica of the target AG or RDS Multi-AZ deployment. The secondary automatically receives the same rows via SQL Server native replication (AG) or RDS-managed replication (Multi-AZ) — there is no need to separately seed the secondary.

Read Replica Seeding

After the initial migration, point CDC Stream at the target primary. The AG secondary or RDS Multi-AZ standby receives the same changes via SQL Server replication — no separate seeding step is required for the secondary replica.

Architecture Recommendation: Network Placement

Recommended Staging Host Placement
SAME REGION · SAME VNet / VPC SQL Server AG Primary (or RDS endpoint) SOURCE Endrias Bridge Migration Host same VNet / VPC STAGING HOST Azure SQL / AWS RDS (FG / Multi-AZ EP) TARGET <1 ms <1 ms
Co-locate staging host in the same VNet/VPC as both source and target
Avoids cross-region egress costs (significant for TB-scale migrations)
Reduces latency from >50 ms (cross-region) to <1 ms (same VNet)
Never place the staging host in a different Azure region or AWS region than source/target
⚠️
For multi-subnet Always On AGs: add MultiSubnetFailover=Yes to the connection string via the Connection String Debug Panel. This enables parallel connection attempts to all subnet replicas, reducing failover detection time from 15–20 s to under 2 s.

📋 Logs & Error Logs

Every run writes to a main log and an error-only log. Migration runs write a per-run timestamped log under migration_logs\. Check the error log first when something goes wrong.

Every run writes to two locations in addition to the live log shown in Step 7:

LogDefault locationContains
Main logC:\Windows\Temp\migrationbridge.logEverything at the configured level (INFO, or DEBUG if enabled) — rotates at 5 MB, keeps 3 backups.
Error logC:\Endrias Bridge\ErrorLogs\errors.logERROR-level messages only — fastest way to see what went wrong without scrolling through informational output. Also rotates at 5 MB, keeps 3 backups.
Migration run logsC:\MigrationTool\migration_logs\migration_<db>_<timestamp>.logPer-run migration log with full detail — progress, row counts, type conversion warnings, index rebuild timings, and any errors. Retained indefinitely (no rotation).
Replication logsC:\MigrationTool\migration_logs\replication_<db>_<date>.logPer-day replication activity log for CDC Stream and Scheduled Sync. Includes LSN ranges, batch apply times, and drift alerts.
  1. Open Error Log Folder

    On the Run step (Step 7), click Open Error Log Folder to open C:\Endrias Bridge\ErrorLogs in File Explorer and inspect errors.log.

  2. Change the log locations

    Edit log_file and error_log_dir in migrationbridge.conf to write logs to a different path — useful for centralized log collection on a shared drive or forwarding to a SIEM.

ℹ️
For service-level logs (Windows Event Log, service status, first-boot marker), see the Verification and Troubleshooting sections of MigrationBridge_SOP.html.

💻 Hardware & Sizing Recommendations

Size the staging host based on source database size. Run Assessment to get an automatic sizing recommendation. Co-locate staging host with source and target in the same region.

Recommended specifications for the machine running Endrias Bridge (the "staging" host), based on source database size:

Source DB sizevCPURAMStaging diskNetwork
Small / Medium (< 500 GB)832 GB100–250 GB SSD1 Gbps+
Medium / Large (500 GB – 3 TB)1248 GB250–500 GB Premium SSD / NVMe1–10 Gbps
Large (3 TB+)1664 GB500 GB+ Premium SSD / NVMe10 Gbps

Network Topology Guidance

Staging host network placement has a significant impact on migration throughput and cost:

💡
Run Assessment on the Assessment tab performs sizing automatically: it measures the actual source database size and shows the matching tier and recommendation at the bottom of the Compatibility Issues panel, alongside data type conversions and a CDC sync mode warning if the source edition does not support Change Data Capture.
⚠️
CDC limitations: SQL Server Web Edition does not support Change Data Capture, and some low-end Azure SQL Database tiers (Basic, Standard S0–S2) either do not support CDC or have throughput limits that make it impractical. Confirm your source and target tiers before selecting CDC Stream mode.

🛠 Phase 4 — Hardening Features

Phase 4 features are automatic or configurable via migrationbridge.conf — no GUI changes required.

These changes ship in Phase 4 and require no GUI interaction — they are enabled via migrationbridge.conf or activate automatically.

Index Disable/Rebuild for PostgreSQL & MySQL (Change 23)

When copying data to a PostgreSQL or MySQL target, the tool now automatically drops non-unique secondary indexes before bulk INSERT and recreates them after. This provides the same 2–4x INSERT speedup that SQL Server targets already had. No configuration required — it is automatic for PostgreSQL and MySQL targets.

Per-Table Chunk Size Override (Change 24)

Add a [chunk_overrides] section to migrationbridge.conf to set an exact row batch size for specific tables. Useful for very wide tables that cause out-of-memory errors at the global chunk size, or narrow lookup tables that run faster with larger batches:

[chunk_overrides]
BigLobTable          = 500
AggregateLineFact    = 200000

Oracle Instant Client Guidance (Change 25)

When Oracle is selected as a source or target engine, the tool now checks for the required driver package before attempting a connection and shows clear step-by-step installation instructions if it is missing, rather than a cryptic DPI-1047 error. The instructions reference the correct Instant Client version for the target Oracle server version.

CDC Lag Rate Alert (Change 26)

In CDC Stream mode, the tool computes a rolling change rate (changes/second over the last 60 seconds) and logs a warning if the source is producing changes faster than the target can apply them. The rate is displayed on every cycle line in the Replication tab log: rate=127/s. If sustained above target capacity, scale up the staging host or reduce concurrency of competing workloads.

CDC Schema Drift Detection (Change 27)

If a column is added to or removed from a source table while the CDC stream is running, the affected table is automatically suspended and a warning is logged. All other tables in the stream continue uninterrupted. To re-enable a suspended table: stop the stream, apply the matching DDL change to the target, then restart the stream. The schema drift event is recorded in the replication log with the table name, column name, and change type (ADD / DROP).