Endrias Bridge Documentation
Everything you need to install, configure, and run Endrias Bridge — the Windows guest-initialization tool and built-in cross-database Migration Assistant.
Overview
Endrias Bridge is two tools in one install. Both are reachable from the same
Setup Wizard at C:\MigrationTool.
Automated first-boot provisioning: detects metadata sources
(ConfigDrive, HTTP IMDS, EC2, Azure), creates the local user, sets the
hostname/network/timezone, injects SSH keys, runs user-data scripts, and
more — driven by migrationbridge.conf and a plugin pipeline.
Assess, migrate, and keep in sync across SQL Server, PostgreSQL, MySQL, Oracle, and SQLite: schema + data copy, automatic data type conversion, incremental/CDC delta sync with throughput & ETA, schema object export, and hardware sizing recommendations.
Documentation Index
Deep-dive guides for each part of the tool:
Engine-Pair SOPs
Standalone runbooks for each supported migration path — validated row counts, live migration setup, type conversion tables, and known issues per engine pair.
Use Cases
Lift-and-shift an on-premises SQL Server, PostgreSQL, or MySQL database to a cloud-hosted equivalent (Azure SQL Database, AWS RDS, Google Cloud SQL) with a Full copy, then keep both sides in sync with Incremental or CDC until cutover.
Move off a licensed engine such as Oracle or SQL Server onto PostgreSQL or MySQL. The Assessment tab flags every syntax difference up front and the Schema Objects tab exports procedures/views/triggers for rewrite.
Full-copy production data into a scratch SQLite or local SQL Server/PostgreSQL/MySQL instance for development, QA, or training — without exposing production credentials to developers.
Keep a read-only reporting/BI database in sync with production using Incremental sync, so heavy reporting queries never touch the OLTP database.
Run periodic Incremental or SQL Server CDC sync to a standby database on a different host, region, or cloud as a low-cost warm copy for DR drills.
Use the Schema Objects tab to inventory every stored procedure, view, trigger, function, table type, temporal table, and SQL Agent job before planning a migration or decommission.
Quick Start
-
Install
Run
MigrationBridge_Setup.exe, or use the per-user shortcut installer (install_shortcut.py). Both place a Endrias Bridge shortcut on the Desktop and Start Menu. -
Launch the Setup Wizard
Double-click the shortcut, or run
launch_gui.pyw. The wizard opens on Step 1 (Welcome). -
Open the Endrias Bridge
From Step 5 (DB Migration) enter your source and target connection details, then click Open Endrias Bridge.
-
Run an Assessment
On the Assessment tab click Run Assessment to discover tables/views, compatibility issues, automatic data type conversions, and the recommended staging hardware tier.
-
Migrate
On the Migration tab, load tables, pick a sync mode (Full / Incremental / CDC), and click Start Migration. Progress, throughput, and an ETA are shown live.
Prerequisites
- Windows 10/11 or Windows Server 2016+ to run Endrias Bridge itself.
- Network connectivity (TCP) from the machine running Endrias Bridge to both the source and target database.
- A database account on the source with read access to the tables/schema being migrated.
- A database account on the target with permission to create schema objects and insert data.
- For SQL Server CDC sync: SQL Server Agent running, and an
account that can execute
sys.sp_cdc_enable_db/sys.sp_cdc_enable_table.
Database drivers
Endrias Bridge connects via SQLAlchemy. Each engine needs its driver available on the machine running Endrias Bridge:
| Engine | Driver | Notes |
|---|---|---|
| SQL Server | ODBC Driver 17 for SQL Server (pyodbc) | Install the Microsoft ODBC driver separately |
| PostgreSQL | psycopg2 | Bundled with the installer |
| MySQL | PyMySQL | Bundled with the installer |
| Oracle | oracledb — pip install oracledb | No Instant Client needed (thin mode). cx_Oracle is legacy and does not build on Python 3.13+ — do not use it. See Oracle setup guide. |
| SQLite | Built into Python | No extra driver needed |
Migration Scenarios
Common source → target pairs and the recommended starting point. All pairs benefit from running Assessment first.
| Source | Target | Recommended Sync Mode | Notes |
|---|---|---|---|
| SQL Server | PostgreSQL | Full, then Incremental or CDC | CDC requires Standard Edition+ (not Express/Web) |
| SQL Server | MySQL | Full, then Incremental or CDC | Schema namespaces may need adjustment |
| SQL Server | Oracle | Full, then Incremental | PL/SQL rewrite required for procedures/functions |
| PostgreSQL | MySQL | Full, then Incremental | JSONB/Arrays not natively supported on MySQL 5.x |
| MySQL | PostgreSQL | Full, then Incremental | AUTO_INCREMENT → SERIAL |
| Oracle | PostgreSQL | Full, then Incremental | Most common Oracle exit path; CONNECT BY → recursive CTE |
| Oracle | MySQL | Full, then Incremental | SEQUENCE + trigger → AUTO_INCREMENT |
| Any | SQLite | Full copy | Local/dev target; incremental and CDC are not supported |
Sync Mode Comparison
| Mode | Captures | Requirements | Progress shown |
|---|---|---|---|
| Full copy | All rows, every run | None | Rows copied / total rows, % done & remaining, throughput (rows/sec), ETA clock time |
| Incremental (watermark column) | Inserts & updates since last run | A watermark column (e.g. updated_at, RowVersion) and a primary key |
Tables done / total, % done & remaining, per-table delta throughput (rows/sec) and overall summary |
| CT Sync — Change Tracking New v75 | Inserts, updates & deletes since last CT version checkpoint. PK + operation only — no full before/after row image. | SQL Server source with Change Tracking enabled at DB + table level. No CDC license or SQL Server Agent required. | Changes applied per table (ins/upd/del breakdown), current CT version checkpoint, table skip warnings if CT version expired |
| SQL Server CDC | Inserts, updates & deletes since last run (full row before/after) | SQL Server source with sys.sp_cdc_enable_db / sys.sp_cdc_enable_table run; Standard Edition or higher |
Tables done / total, % done & remaining, per-table change throughput (changes/sec) and overall summary |
CLI Headless Mode New v75
Run any migration from the command line — no GUI window required. Useful for Azure DevOps pipelines, PowerShell scripts, Windows Task Scheduler, and Docker containers.
MigrationBridge migrate \ --src-type azuresql --src-host SERVER.database.windows.net --src-db SOURCE \ --src-user user --src-pass pass \ --tgt-type rds --tgt-host rds.xxxx.rds.amazonaws.com --tgt-db TARGET \ --tgt-user user --tgt-pass pass \ --mode full --schema --data --verify --report report.html
Exit code 0 = success (all tables migrated and verified). Exit code 1 = one or more errors. Use $LASTEXITCODE in PowerShell to gate pipeline stages.
See SOP → CLI Headless Mode for the full flag reference and Azure DevOps YAML example.
Endrias Bridge Tabs
Supported Versions
| Engine | Supported Versions | Notes |
|---|---|---|
| SQL Server | 2014 and later (incl. Azure SQL Database / Managed Instance) | CDC requires Standard Edition or higher |
| PostgreSQL | 11 and later | Includes Azure Database for PostgreSQL, AWS RDS, Google Cloud SQL |
| MySQL | 5.7, 8.0, and MariaDB 10.3+ | JSON/ENUM/SET types are converted on copy — see Data Type Conversions |
| Oracle | 11g and later (12c+ recommended) | 12c+ enables IDENTITY columns and FETCH FIRST syntax |
| SQLite | 3.x (bundled with Python) | Target only — Full copy mode only |
Limitations
- CDC sync mode is available for SQL Server sources only; other engines use Incremental (watermark column) sync instead.
- Incremental sync requires every migrated table to have a watermark column
(e.g.
updated_at,RowVersion) and a primary key. - SQLite as a target supports Full copy only — Incremental and CDC sync are not available.
- Sync is one-directional (source → target) — bi-directional / multi-master replication is not supported.
- Stored procedures, views, triggers, and functions are migrated automatically when the source and target are the same engine (e.g. SQL Server → SQL Server / Azure SQL, MySQL → MySQL, PostgreSQL → PostgreSQL). Across different engines they are not auto-translated (T-SQL ≠ PL/pgSQL ≠ PL/SQL); the Schema Objects tab exports them for manual review and rewrite.
- Oracle
BFILEcolumns migrate the file path only — the referenced file's contents are not copied.
Release Notes & Upgrading
June 2026
- Fix: Temporal table system-versioning crash —
_copy_schema_objectswas missing afrom sqlalchemy import textimport, causing aNameErroron theALTER TABLE … SYSTEM_VERSIONING = ONstep for every temporal table. All temporal tables now have system-versioning re-enabled automatically after migration. - Fix: CDC/DMS system tables processed during Resume —
The resume row-count loop now skips
cdc.*,*_CT,attrep_*,_attrep_*, andsystranschemasentries. Previously these appeared in old unpatched table selections and causedInvalid object nameerrors during resume. - Fix: "Converting decimal loses precision" not retried —
The batch-error retry path (which previously only handled string truncation)
now also catches
'loses precision'and'converting decimal'errors thrown by pyodbcfast_executemany. Affected batches are retried row-by-row on a non-fast_executemanyconnection so all decimal rows land correctly. - Fix: Sequences not created on Resume / data-only runs —
ensure_sequences_existwas called only insideif do_schema:, so resume and data-only passes skipped it. Sequence creation now runs before the data copy on all paths when the target is SQL Server family. - Azure SQL → AWS RDS DDL compatibility conversions —
jsoncolumns,NullTypecolumns (includinghierarchyid/ spatial types),decimalprecision above 38, and Azure-only server-side defaults (getutcdate(),newsequentialid(), etc.) are automatically rewritten beforeCREATE TABLE— see Azure SQL → AWS RDS Compatibility. - Fix: "Not Responding" on large databases —
The UI queue was flooded with a progress message on every single batch, and the
_logwidget called.index()on every single log line (an expensive Tk operation). Progress messages are now emitted every 10 batches and the line-count trim runs every 50 log calls instead of every call. Eliminates the Tkinter freeze on large databases such as acme_erp (200+ tables). - Feature: Email alerts on migration failure or completion —
Configure SMTP credentials in
migrationbridge.conf [alerts]to receive an HTML email when a table errors, the migration crashes (FATAL), or the migration completes. Disabled by default (leavesmtp_hostblank). Supports Office 365 / STARTTLS (port 587) and SSL (port 465). - Migration tab table list — table name column is wider with a
horizontal scrollbar so long fully-qualified names (e.g.
salesdata.ProviderRateMatrix) are fully readable. - CDC/DMS table exclusion on Load Tables — Load Tables
now filters
cdc.*,*_CT,attrep_*, andsystranschemasout of the table list automatically.
a previous build
- Initial release: Setup Wizard (guest initialization) and DB Migration Assistant.
- Assessment, Migration, Schema Objects, Tools, Connection Profiles, and Logs tabs.
- Full, Incremental, and SQL Server CDC sync modes with live progress, ETA, and throughput.
- Automatic data type conversion and compatibility matrix across SQL Server, PostgreSQL, MySQL, Oracle, and SQLite.
- Hardware/VM sizing recommendations and a Windows installer
(
MigrationBridge_Setup.exe).
Upgrading
Download and run the latest MigrationBridge_Setup.exe over an
existing installation — it updates the application files in place.
Connection profiles
(%APPDATA%\Endrias Bridge\connection_profiles.json) and error
logs (C:\Endrias Bridge\ErrorLogs) live outside the install
directory and are preserved across upgrades.
db_migration.py are only picked up by the rebuilt executable.
Databases that were migrated with the a previous build build and have temporal tables
should use Re-apply Schema Objects from the Schema Objects tab to
re-enable system-versioning without re-migrating data.
Azure SQL → AWS RDS DDL Compatibility
Azure SQL Database and AWS RDS for SQL Server are both SQL Server family engines,
so Endrias Bridge applies their DDL directly — no cross-engine type rewrite. However,
Azure SQL exposes certain types and schema artifacts that are not valid on AWS RDS.
These are automatically rewritten by retype_azure_for_rds during schema
creation, with a note line in the migration log for each affected column.
| Azure SQL artifact | Converted to | Reason |
|---|---|---|
json column type |
NVARCHAR(MAX) |
The json type is Azure SQL-only (added in SQL Server 2022 / Azure SQL).
AWS RDS SQL Server 2019 and earlier do not support it as a column type;
JSON data is stored as NVARCHAR(MAX) instead. |
NullType column (SQLAlchemy reflection of hierarchyid,
geography, geometry, or other driver-opaque types) |
NVARCHAR(MAX) |
These types are reflected as NullType when the ODBC driver does not
expose them. They are stored as their string representation
(hierarchyid → /1/2/ path, spatial → WKT text). |
decimal / numeric with precision > 38 |
decimal(38, s) — precision clamped to 38 |
SQL Server's maximum precision for decimal / numeric
is 38. A reflected precision above 38 (can appear when the source schema was
created via a non-ODBC path) would cause a CREATE TABLE failure. |
| Server-side column defaults using Azure-only functions | Default expression stripped (column retains no default) |
Azure SQL supports functions such as getutcdate(),
newsequentialid(), and sysdatetime() as column
defaults. These resolve correctly on RDS too, but Azure-specific variants
(e.g. SYSUTCDATETIME() used in certain Azure system views, or
custom Azure-only built-ins) are stripped to prevent DDL failures.
Row values are copied explicitly so defaults are not needed during migration.
|
NOTE: [table].[column]: <description>
lines in the migration log under the
--- Azure SQL → RDS DDL compatibility conversions --- header.
No data is lost — only the column's DDL type is adjusted.
Change Data Capture (CDC) Replication
SQL Server CDC is Endrias Bridge's most advanced sync mode: it continuously replicates inserts, updates, and deletes from a SQL Server source to any supported target — a continuous copy / replication relationship built directly into the Migration tab.
How it works
- The Migration tab calls
sys.sp_cdc_enable_dbandsys.sp_cdc_enable_tableon the source (if not already enabled) so SQL Server begins recording changes in CDC capture tables. - Each sync pass reads only the rows that changed since the last run (tracked via the CDC log sequence number), and applies inserts, updates, and deletes to the target in the same order.
- Per-table and overall change throughput (changes/sec), % complete, and ETA are shown live — see Sync Mode Comparison.
Requirements & fallback
- Source must be SQL Server Standard Edition or higher (Express and Web editions do not support CDC) with SQL Server Agent running.
- If CDC isn't available, Assessment recommends Incremental sync instead (requires a watermark column and primary key on each table).
Encryption & Secure Connections
Endrias Bridge connects via SQLAlchemy connection strings. Append the engine-specific SSL/TLS parameters below whenever the source or target is reachable over a public network — including any cloud-hosted database.
| Engine | How to enable SSL/TLS |
|---|---|
| SQL Server | Add Encrypt=yes;TrustServerCertificate=yes (or no with a trusted CA) to the ODBC driver options |
| PostgreSQL | Append ?sslmode=require (or verify-full with sslrootcert=) |
| MySQL | Append ?ssl_ca=<path-to-ca.pem> and ssl_verify_cert=true |
| Oracle | Use a tcps:// protocol with an Oracle Wallet (TNS_ADMIN pointing at the wallet directory) |
Working with Cloud-Hosted Databases
Azure SQL Database / Managed Instance, AWS RDS / Aurora, Google Cloud SQL, and SQL Server / MySQL / PostgreSQL running on a cloud VM all work as a standard source or target — just supply the cloud endpoint hostname, port, and credentials on the connection screen.
- Allow-list the public IP address of the machine running Endrias Bridge in the cloud database's firewall / security group rules.
- Cloud-hosted databases almost always require SSL/TLS — see Encryption & Secure Connections.
- For large or long-running migrations, run Endrias Bridge on a VM in the same region/VNet/VPC as the target to minimize latency — see Hardware & Sizing.
- SQL Server CDC requires Standard Edition or higher — check the tier of a managed instance (e.g. Azure SQL Database Business Critical/General Purpose, or AWS RDS SQL Server Standard/Enterprise) before relying on CDC sync.
Compatibility Matrix
Shown on the Assessment tab as Compatibility Issues, based on the selected source/target engines.
SQL Server → PostgreSQL
- IDENTITY → SERIAL / GENERATED ALWAYS AS IDENTITY
- DATETIME → TIMESTAMP
- NVARCHAR(MAX) → TEXT
- BIT → BOOLEAN
- TOP N → LIMIT N
- Stored procedures need rewriting in PL/pgSQL
- CHECK constraints with functions may differ
SQL Server → MySQL
- IDENTITY → AUTO_INCREMENT
- NVARCHAR → VARCHAR (UTF8MB4)
- TOP N → LIMIT N
- Schema namespaces may need adjustment
- Clustered index behaviour differs
PostgreSQL → MySQL
- SERIAL → AUTO_INCREMENT
- BOOLEAN → TINYINT(1)
- LIMIT y OFFSET x → LIMIT x,y
- Arrays / JSONB not natively supported in MySQL 5.x
MySQL → PostgreSQL
- AUTO_INCREMENT → SERIAL
- TINYINT(1) → BOOLEAN
- ENGINE=InnoDB → remove (not applicable)
- LIMIT x,y → LIMIT y OFFSET x
Oracle → PostgreSQL
- NUMBER → NUMERIC
- VARCHAR2 / NVARCHAR2 → VARCHAR / TEXT
- SEQUENCE + trigger → SERIAL / GENERATED ALWAYS AS IDENTITY
- ROWNUM → LIMIT / OFFSET
- DUAL table not needed (SELECT expr without FROM works directly)
- DECODE() → CASE WHEN ... END
- NVL() → COALESCE()
- (+) outer join syntax → ANSI LEFT/RIGHT JOIN
- CONNECT BY → WITH RECURSIVE (recursive CTE)
- PL/SQL packages / procedures need rewriting in PL/pgSQL
PostgreSQL → Oracle
- SERIAL / IDENTITY → SEQUENCE + trigger (or GENERATED ALWAYS AS IDENTITY on 12c+)
- TEXT → CLOB / VARCHAR2(4000)
- LIMIT / OFFSET → ROWNUM or FETCH FIRST n ROWS ONLY (12c+)
- BOOLEAN → NUMBER(1) (no native boolean before Oracle 23c)
- WITH RECURSIVE → CONNECT BY or recursive WITH (11g+)
- PL/pgSQL functions need rewriting in PL/SQL
Oracle → MySQL
- NUMBER → DECIMAL / INT depending on precision
- VARCHAR2 / NVARCHAR2 → VARCHAR (utf8mb4)
- SEQUENCE + trigger → AUTO_INCREMENT
- ROWNUM → LIMIT
- DECODE() → CASE WHEN ... END
- NVL() → IFNULL() / COALESCE()
- CONNECT BY → recursive CTE (MySQL 8.0+) or application-side recursion
- PL/SQL packages / procedures need rewriting as MySQL stored procedures
MySQL → Oracle
- AUTO_INCREMENT → SEQUENCE + trigger (or IDENTITY column on 12c+)
- TINYINT(1) → NUMBER(1)
- LIMIT x,y → OFFSET x ROWS FETCH NEXT y ROWS ONLY (12c+) or ROWNUM
- ENGINE=InnoDB → remove (not applicable)
- ON DUPLICATE KEY UPDATE → MERGE statement
Oracle → SQL Server
- NUMBER → DECIMAL / INT / BIGINT depending on precision
- VARCHAR2 / NVARCHAR2 → VARCHAR / NVARCHAR
- SEQUENCE + trigger → IDENTITY
- ROWNUM → TOP N or OFFSET-FETCH
- DECODE() → CASE WHEN ... END
- NVL() → ISNULL() / COALESCE()
- CONNECT BY → recursive CTE (WITH ... AS)
- PL/SQL packages / procedures need rewriting in T-SQL
SQL Server → Oracle
- IDENTITY → SEQUENCE + trigger (or IDENTITY column on 12c+)
- NVARCHAR(MAX) → CLOB / NVARCHAR2(4000)
- TOP N → ROWNUM or FETCH FIRST N ROWS ONLY
- ISNULL() → NVL()
- GETDATE() → SYSDATE / SYSTIMESTAMP
- T-SQL stored procedures / functions need rewriting in PL/SQL
Any other pair
- Review data type mappings for the target platform
- Check stored procedure / function syntax compatibility
- Verify index and constraint naming conventions
- Test NULL handling and default values
Data Type Conversions
Applied automatically when the source and target engines differ — both at
schema creation time (column type) and at row-insert time (value
conversion, with truncation/rounding reported as WARNING lines
in the migration log).
At schema creation time, source-engine artifacts that the target won't
accept are also stripped from the generated CREATE TABLE:
column collations (e.g. SQL Server
SQL_Latin1_General_CP1_CI_AS) and server-side defaults
(e.g. SQL Server getdate() / newid()). Row values
are copied explicitly, so column defaults aren't needed for the copy.
| Source Engine | Source Type | Converted To | Note |
|---|---|---|---|
| SQL Server | UNIQUEIDENTIFIER | UUID (PostgreSQL) / CHAR(36) string (others) | Uses the native UUID type on PostgreSQL; falls back to a 36-char string where the target has no native UUID type |
| SQL Server | SMALLMONEY | NUMERIC(10,4) | — |
| SQL Server | MONEY | NUMERIC(19,4) | — |
| SQL Server | DATETIMEOFFSET | TIMESTAMP WITH TIME ZONE | — |
| SQL Server | SMALLDATETIME | TIMESTAMP | — |
| SQL Server | DATETIME2 | TIMESTAMP | — |
| SQL Server | DATETIME | TIMESTAMP | No DATETIME type on PostgreSQL |
| SQL Server | NTEXT | TEXT | Deprecated SQL Server type |
| SQL Server | IMAGE | BLOB / BYTEA | Deprecated SQL Server type |
| SQL Server | VARBINARY | BLOB / BYTEA | No VARBINARY type on PostgreSQL; length preserved where the target supports it |
| SQL Server | BINARY | BLOB / BYTEA | No BINARY type on PostgreSQL |
| SQL Server | XML | TEXT | No portable XML type; stored as text (lossless) |
| SQL Server | SQL_VARIANT | TEXT | Values converted to their string representation |
| SQL Server | ROWVERSION | BINARY(8) | Auto-increment-on-write behaviour is lost |
| SQL Server | TIMESTAMP (rowversion) | BINARY(8) | Auto-increment-on-write behaviour is lost |
| SQL Server | BIT | BOOLEAN | — |
| SQL Server | TINYINT | SMALLINT | No 1-byte integer type on most other engines |
| SQL Server | NVARCHAR | VARCHAR | No national-character type on target; length preserved |
| SQL Server | NCHAR | CHAR | No national-character type on target; length preserved |
| SQL Server | hierarchyid / geography / geometry | TEXT | No SQLAlchemy/driver mapping; read via .ToString() (hierarchyid → /1/2/ path, spatial → WKT). Original type is lost |
| PostgreSQL | UUID | CHAR(36) string | Target has no native UUID type |
| PostgreSQL | JSONB | TEXT | Stored as JSON text |
| PostgreSQL | JSON | TEXT | Stored as JSON text |
| PostgreSQL | ARRAY | TEXT | Stored as JSON text; array semantics are lost |
| PostgreSQL | INTERVAL | VARCHAR(40) string | — |
| PostgreSQL | CIDR | VARCHAR(45) string | — |
| PostgreSQL | INET | VARCHAR(45) string | — |
| PostgreSQL | MACADDR | VARCHAR(17) string | — |
| MySQL | SET | VARCHAR(255) string | — |
| MySQL | ENUM | VARCHAR(255) string | Allowed values become free text |
| MySQL | YEAR | SMALLINT | — |
| Oracle | ROWID | VARCHAR(32) string | — |
| Oracle | LONG | TEXT | — |
| Oracle | BFILE | VARCHAR(255) | File path only; referenced file contents are not migrated |
| Oracle | INTERVAL | VARCHAR(40) string | — |
Row-value conversions
Independent of the table above, individual values are coerced so inserts don't fail against the target type:
- UUID objects → strings (when the target column is not a UUID type)
dict/listvalues → JSON text (for String/Text targets)timedelta→ stringmemoryview/bytearray→bytesbool→int(for Numeric targets)Decimalvalues rounded to the target column's scale (reported asWARNING: ... rounded ...)- Strings longer than the target
VARCHAR(n)truncated (reported asWARNING: ... truncated ...)
Performance & Tuning
Data is copied in batches (fetch + multi-row INSERT). Throughput is driven by a few levers, most of which apply automatically:
| Lever | Default / behaviour | Notes |
|---|---|---|
| Batch size | 5,000 rows default; current install set to 100,000 via
db_migration_chunk_size in migrationbridge.conf |
Larger = fewer round-trips = faster, at the cost of more memory per batch.
100,000 is the confirmed production setting
(Azure SQL → AWS RDS; actual row widths all < 2,000 bytes).
50,000 is the proven safe floor. Lower to 20,000–25,000 for very wide /
BLOB / nvarchar(MAX) tables.
The migration log prints Copying in batches of N rows.
See MigrationBridge_Troubleshooting.html → Batch size: what to do and NOT to do
for the full sizing guide.
|
| SQL Server target | fast_executemany=True (automatic) |
pyodbc otherwise does one round-trip per row; batching is often 10–100× faster on inserts. |
| PostgreSQL target | executemany_mode='values_plus_batch' (automatic) |
psycopg2 sends batched multi-row VALUES, page-sized to the
batch. |
| Source read | Server-side streaming cursor (stream_results, automatic) |
Large tables stream in batches instead of buffering the whole result in memory. |
| Parallel tables | Concurrent migration jobs (batch/jobs runner) | Independent tables/databases can copy in parallel on background threads. |
db_migration_chunk_size = 100000 for narrow tables (confirmed
baseline — actual row widths < 2,000 bytes);
(3) increase RDS IOPS / instance class for the migration window and scale back after;
(4) never increase resources on the source — all write pressure is on
the target. See the full guide at
Troubleshooting → Migration Is Too Slow.
acme_audit observed performance
The acme_audit database (52 GB, two schemas: auditdata and
analyticsdata) was observed to take 17+ hours on a laptop over
VPN with the default 5,000-row batch size. Root causes: default batch size (5,000 rows)
and two WAN hops per round-trip (Azure SQL → laptop → AWS RDS). Expected duration with
recommended settings (100k batch size, VM co-located with source): 35–45
minutes. See
Troubleshooting → Migration Is Too Slow
for the step-by-step runbook.
Hardware & Sizing
The Assessment tab measures the source database size and recommends a staging-host tier for the migration:
| Source DB Size | vCPU | RAM | Staging Disk | Network |
|---|---|---|---|---|
| Small / Medium (< 500 GB) | 8 | 32 GB | 100-250 GB SSD | 1 Gbps+ |
| Medium / Large (500 GB - 3 TB) | 12 | 48 GB | 250-500 GB Premium SSD / NVMe | 1-10 Gbps |
| Large (3 TB+) | 16 | 64 GB | 500 GB+ Premium SSD / NVMe | 10 Gbps |
- Use Windows Server 2022 for the migration/staging VM where possible.
- Keep the source, staging host, and target in the same region / VNet / VPC to minimize latency and avoid cross-region egress costs.
- For large migrations, use a dedicated VM with no other workloads running on it during the migration window.
Email Alerts
MigrationBridge can send an HTML email when a migration event occurs. Configure the
[alerts] section in C:\MigrationTool\migrationbridge.conf.
All alerts are disabled by default — leave smtp_host
blank to keep them off.
Supported events
| Event key | When it fires |
|---|---|
error | A single table fails during data copy (non-fatal — migration continues with remaining tables) |
fatal | The entire migration crashes — connection lost, schema creation failed, unhandled exception |
complete | Migration finishes successfully (all tables done) |
Configuration — Office 365 / STARTTLS (port 587)
[alerts] smtp_host = smtp.office365.com smtp_port = 587 smtp_use_tls = true smtp_use_ssl = false smtp_user = migrations@endriasbridge.com smtp_password = YourAppPassword alert_to = dba@endriasbridge.com alert_from = MigrationBridge <migrations@endriasbridge.com> alert_on = error,fatal,complete
Configuration — SSL (port 465)
[alerts] smtp_host = smtp.gmail.com smtp_port = 465 smtp_use_tls = false smtp_use_ssl = true smtp_user = you@gmail.com smtp_password = YourAppPassword alert_to = you@gmail.com alert_on = error,fatal
alert_to = dba@endriasbridge.com, oncall@endriasbridge.com
What the email contains
- error — database name, table name, error text, rows copied before the error
- fatal — database name, full exception text, source and target connection summary (password masked)
- complete — database name, table count, sync mode, total duration
migrationbridge.conf, rebuild MigrationBridge.exe with
PyInstaller so the new [alerts] section is bundled into the executable.
The conf file is read at runtime from the same directory as the EXE, so no rebuild
is needed if you only change the SMTP credentials — only the initial bundle of
db_alerts.py requires a rebuild.
Configuration & Extensibility Reference
Endrias Bridge settings
| Setting | Default | Purpose |
|---|---|---|
error_log_dir (in migrationbridge.conf) | C:\Endrias Bridge\ErrorLogs | Persistent per-run error logs from the Migration tab |
| Connection profiles | %APPDATA%\Endrias Bridge\connection_profiles.json | Saved source/target connection details |
Guest-init plugin & metadata API
The guest-initialization side of Endrias Bridge (the Setup Wizard's
first-boot pipeline) is driven by the same migrationbridge.conf
and is extensible with custom metadata sources and plugins. For the full
configuration keys, plugin pipeline, and metadata source reference, see:
Service, Logs & Troubleshooting
For the guest-initialization side of Endrias Bridge (the Windows service / first-boot plugin pipeline), see the Standard Operating Procedure:
- Architecture
- Check Service Status
- Read the Logs
- Service Management
- Configuration Reference
- Plugin Reference
- Troubleshooting
- Event Log Commands
For Endrias Bridge logs and persistent error logs, see User Guide → Logs & Error Logs.
Comparison Report Interpretation
After a migration completes, the db_comparison CSV (produced by running
the comparison tool against source and target) is the primary sign-off artifact. Each row
has a Status of MATCH, SOURCE ONLY, or
TARGET ONLY. Here is how to read each status:
NoSQL engines: Comparison is supported for MongoDB, Cosmos DB, DynamoDB, Redis, Cassandra, and Elasticsearch. For NoSQL pairs, the comparison shows collection/index/key-group names and document/item counts (MATCH / MISMATCH / SOURCE ONLY / TARGET ONLY). Schema-level comparison (columns, indexes, constraints) is not applicable for NoSQL engines.
| Status | Meaning | Action |
|---|---|---|
| MATCH | Object or row count is identical on source and target | No action — this is the expected outcome |
| SOURCE ONLY | Object exists on source but is missing on target | Investigate. For tables: re-run migration for the missing table. For schema objects (procs/views/types): re-apply via the Schema Objects tab |
| TARGET ONLY | Object exists on target but not in source | Usually expected — see below for the common categories that produce TARGET ONLY entries |
Why TARGET ONLY entries are usually expected
A healthy comparison report for an enterprise database may contain a large block of
TARGET ONLY stored procedures, functions, and table types. These come from
application schemas that are deployed via SSDT / application installers
— not via MigrationBridge — and exist on the shared RDS instance but not in the
source Azure SQL database being compared. Common schemas:
| Schema | Description | Deployed by |
|---|---|---|
coreapi | Configuration API stored procs and TVPs (~73 procs, ~56 types) | SSDT / app installer |
priceapi | Fee schedule API stored procs, functions, TVPs (~50 procs, 1 func, ~17 types) | SSDT / app installer |
customapi | Custom edit API procs and TVPs | SSDT / app installer |
referenceapi | Age diagnosis code API procs and TVPs | SSDT / app installer |
catalogapi | Custom CPT/HCPCS API procs and TVPs | SSDT / app installer |
mappingapi | Message map API procs and TVPs | SSDT / app installer |
limitsapi | Frequency limit API procs and TVPs | SSDT / app installer |
rulesapi | MUE API procs and TVPs | SSDT / app installer |
dbo | TVP type aliases shared across the product | SSDT / app installer |
util | Utility procs, functions, and types not present in all source DBs | SSDT / app installer |
auditdata, analyticsdata,
salesdata, configdata).
Pass criteria for a completed migration
- Zero SOURCE ONLY rows — every source object is present on target
- All table row counts MATCH — data volume is identical
- All column types MATCH — no schema drift
- All constraints and indexes MATCH — PKs, FKs, unique constraints all present
- TARGET ONLY entries limited to the application-deployed schemas listed above
A report that meets all five criteria can be signed off as PASS.
Known validated databases
| Database | Comparison file | Tables | Columns | SOURCE ONLY | Status | Date |
|---|---|---|---|---|---|---|
| AdventureWorks2019 → Oracle SQL Server 2019 → Oracle 21c (XE) |
migration log verified | 71/71 PASS | All columns migrated | 0 | PASS | July 2026 |
| acme_audit | db_comparison-toaudit.csv |
27/27 MATCH | 570/570 MATCH | 0 | PASS | June 2026 |
| acme_erp | db_comparison-acme_erp.csv |
0/99 MATCH — all SOURCE ONLY | All SOURCE ONLY | 99 tables + all columns/constraints/indexes | INCOMPLETE | June 2026 |
acme_audit — detail
The db_comparison-toaudit.csv comparison run against acme_audit
(schemas auditdata and analyticsdata) returned 27/27 tables MATCH,
570/570 columns MATCH, 28/28 constraints MATCH, 29/29 indexes MATCH, and 56/56 statistics
MATCH. Zero SOURCE ONLY entries. TARGET ONLY entries were exclusively application-schema
objects (coreapi, priceapi, customapi, etc.) deployed independently of the migration.
Status: PASS — signed off June 2026.
acme_erp — detail (INCOMPLETE — pending re-migration)
The db_comparison-acme_erp.csv comparison run against acme_erp
shows that all 99 data tables (schemas coredata, salesdata,
customdata, referencedata, catalogdata,
limitsdata, mappingdata, rulesdata) are SOURCE ONLY
— target column shows — for every table row count, meaning the tables were never
created on the RDS target. The migration did not land the data schema.
The SSDT-deployed API objects (148 stored procs, 7 functions, 92 TVP types across
coreapi, priceapi, customapi, etc.) all show
MATCH — those were deployed independently and are healthy.
Remediation — required steps:
- Rebuild
MigrationBridge.exewith PyInstaller to pick up all the current build fixes. - Run a Full migration (not Resume) from an Azure VM or EC2 instance.
- The tool automatically handles any partial state on the target — no manual DROP needed. See Troubleshooting → Full vs Resume.
- After migration, re-run the comparison tool and verify all 99 data tables show MATCH.
- System-versioning on the 38 temporal tables (
salesdata,referencedata,catalogdata,limitsdata,mappingdata,rulesdata) will be re-enabled automatically by the current build at the end of the migration.
Warnings & Messages Reference
Messages that can appear in the Migration tab's log:
| Message | Meaning |
|---|---|
| WARNING ... rounded to fit NUMERIC(p,s) | A decimal value had more digits than the target column's scale and was rounded |
| WARNING ... truncated to fit VARCHAR(n) | A string value was longer than the target column and was truncated |
| WARNING: batch truncation / precision error, retrying N rows individually | A batch INSERT failed with a truncation or decimal-precision error (fast_executemany pyodbc quirk). The batch is automatically retried row-by-row on a plain connection — no data is lost |
| progress / ETA below is approximate | One or more tables' row counts couldn't be pre-scanned, so the overall percentage/ETA is an estimate (the copy itself is unaffected) |
| --- Full copy: ... --- | Final summary line: total rows copied, duration, and throughput for a Full copy |
| --- Delta sync: ... --- | Final summary line: total rows synced, duration, and throughput for an Incremental sync |
| --- CDC sync: ... --- | Final summary line: total changes applied, duration, and throughput for a CDC sync |
| Schema: ready (per schema name) | Target schema was created or already existed — DDL succeeded for that schema |
| Sequence [schema].[name]: created on target | A SEQUENCE object required by a DEFAULT constraint was created on the target before the data copy |
| [table]: system-versioning ON (history table: …) | Temporal table system-versioning was successfully re-enabled on the target after the data copy |
| [table]: system-versioning FAILED | The ALTER TABLE … SYSTEM_VERSIONING = ON step failed for this temporal table. Use Re-apply Schema Objects from the Schema Objects tab to retry without re-migrating data. Fixed in the current build. |
| WARNING [TableName]: verification error - | Non-fatal: SQLAlchemy reflection on schema-qualified tables failed during post-creation verification. Tables were still created successfully — confirmed by the Result: all N table(s) OK summary line |
NOTE: [table].[col]: json → NVARCHAR(MAX) (and similar) | Azure SQL → AWS RDS DDL compatibility conversion applied to this column — see Azure SQL → AWS RDS Compatibility |
Copying in batches of N rows | Informational: shows the active db_migration_chunk_size value for this run (default 5,000; 100,000 is confirmed stable for narrow tables) |
FAQ
Which databases are supported?
SQL Server, PostgreSQL, MySQL, Oracle, and SQLite as both source and target, via SQLAlchemy. See the Compatibility Matrix for per-pair notes.
Do I need to write any SQL to migrate schema and data?
No. The Migration tab reflects the source schema, applies any required data type conversions, creates the tables on the target, and copies the data in chunks.
Does the target database have to exist first?
No. If the target database named on the connection screen doesn't exist,
Endrias Bridge creates it automatically (connecting to the server's
maintenance database) before creating schemas and tables - including names
with dots like dba.stackexchange.com. This needs an account with
permission to create databases (PostgreSQL CREATEDB, MySQL
CREATE, SQL Server dbcreator); if the account lacks
it, the log shows a clear message telling you to create the database manually.
SQLite creates its file automatically.
Can the target database have a different name from the source?
Yes. Source and target are independent connections, each with its own Database
field - the names don't have to match (e.g.
AdventureWorks2019 → analytics_warehouse).
Only the schema namespaces inside the database (e.g.
HumanResources, Person, Sales) are
recreated by the same name within your differently-named target database; the
database/catalog name itself is whatever you set on the target connection.
Are stored procedures and views migrated?
For same-engine migrations (e.g. SQL Server → SQL Server / Azure SQL, MySQL → MySQL, PostgreSQL → PostgreSQL), procedures, views, triggers, and functions are applied to the target automatically after the tables and data (best-effort: applied views → functions → procedures → triggers, with one retry pass for inter-object dependencies; failures are logged, never fatal). Across different engines they can't be auto-translated - use the Schema Objects tab to export and rewrite them.
Are system-versioned (temporal) tables supported?
The temporal table and its history table migrate as ordinary tables, so all
data comes across. The system-versioning itself
(PERIOD FOR SYSTEM_TIME, GENERATED ALWAYS AS ROW START/END,
SYSTEM_VERSIONING = ON) is detected and scripted by the Schema
Objects tab for you to apply. Re-enabling versioning after migration depends on the target engine:
- SQL Server / Azure SQL target — fully automatic. The Schema Objects tab generates a T-SQL re-enable script and can apply it in one click.
- Oracle 23ai+ target — the Schema Objects tab generates an Oracle
ALTER TABLE … ADD PERIOD FOR SYSTEM_TIME+ADD VERSIONING USE HISTORY TABLEscript. The already-migrated SQL Server history table is wired in directly so pre-migration history is immediately queryable. - Oracle 11g R2–21c target — the Schema Objects tab generates a
Flashback Data Archive (FDA) script:
CREATE FLASHBACK ARCHIVE+ALTER TABLE … FLASHBACK ARCHIVE. FDA tracks all post-migration DML automatically; the migrated history table covers pre-migration history. - PostgreSQL / MySQL target — no native equivalent; the Schema Objects tab generates a trigger-based emulation pattern (history table + BEFORE UPDATE/DELETE triggers).
Period columns (SysStart, SysEnd, ValidFrom,
ValidTo) are migrated as plain TIMESTAMP columns on non–SQL Server
targets. The migration log flags each one with a note pointing to the Schema Objects tab.
How do I keep the target in sync after the first migration?
Use Incremental sync mode if your tables have a watermark
column (e.g. updated_at, RowVersion) and a primary
key, or SQL Server CDC sync mode if the source is SQL
Server Standard Edition or higher with CDC enabled.
What if my source is too large to estimate an ETA up front?
If a table's row count can't be determined, the migration log notes that progress/ETA is approximate for that table, but the copy still proceeds normally — only the percentage/ETA display is affected.
Where do I find error details after a failed migration?
Check the Migration tab's log first, then the persistent error logs via
User Guide → Logs & Error
Logs (folder configurable via error_log_dir in
migrationbridge.conf).
Can I export the discovered schema objects (procedures, views, etc.)?
Yes — the Schema Objects tab can export everything found to a single
.sql file, or apply same-platform objects directly to the
target.
Why did my decimal columns fail with "Converting decimal loses precision"?
This is a pyodbc fast_executemany quirk: the driver
infers the decimal parameter type from the first row of a batch, and if a later
row needs more integer digits (wider precision), the driver rejects the batch.
As of the current build, these batches are automatically caught and retried row-by-row on
a plain (non-fast_executemany) connection — no data loss, no manual
action required. See
Troubleshooting → Converting decimal loses precision.
Why were some SEQUENCE objects missing after a resume run?
In a previous build, SEQUENCE objects were only created during a full
schema+data migration. Resume or data-only runs skipped the step, causing
INSERT failures on tables whose DEFAULT constraint
references a sequence. As of the current build, sequence creation runs before the data
copy on all paths (full, resume, and data-only).
Why did my temporal tables not have system-versioning after migration?
A missing import in a previous build caused a NameError on the
ALTER TABLE … SYSTEM_VERSIONING = ON step. Tables and data were
copied successfully, but versioning was not re-enabled. Fixed in the current build. If
your database was migrated with a previous build, use Re-apply Schema Objects
from the Schema Objects tab to re-enable system-versioning without touching
the data.
My migration is running much slower than expected — what should I check?
The three most common causes, in order of impact:
- Batch size still at default (5,000). Set
db_migration_chunk_size = 100000inmigrationbridge.confand restart the GUI. At 100k batches vs 5k batches, round-trips drop 20×. Verify the change loaded by checking the migration log forCopying in batches of 100000 rows. - Running from a laptop over VPN. Each batch makes two WAN hops (source read + target write). Move MigrationBridge to an Azure VM co-located with the Azure SQL source, or an EC2 instance co-located with the RDS target. Either eliminates one hop entirely and cuts latency from 20–80 ms to <2 ms.
- RDS IOPS exhausted. Check CloudWatch
WriteLatencyduring a live run — if >20 ms, switch to gp3 storage and provision 6,000–12,000 IOPS for the migration window. Scale back afterwards.
For the full diagnosis runbook with CloudWatch checks, expected throughput numbers, and the acme_audit case study (17 hrs → ~40 min), see Troubleshooting → Migration Is Too Slow.
What does a "TARGET ONLY" entry in the comparison report mean?
It means an object exists on the target RDS instance but not in the source database being
compared. For enterprise deployments this is almost always expected — application-deployed
schemas are installed on the shared RDS instance via SSDT or application installers,
not via MigrationBridge. Only investigate
TARGET ONLY entries under schemas that were part of the migration itself (e.g.
auditdata, analyticsdata). See
Comparison Report Interpretation for the full pass criteria.
Should I use Full migration or Resume if my previous migration did not complete?
It depends on what is already on the target:
- Data schema is missing (tables not created) — use Full migration.
Resume skips schema creation (
CREATE TABLE) entirely, so it will try to INSERT into tables that don't exist and fail for every table. Full migration runsdrop_all(checkfirst=True)+create_all(checkfirst=True)first, which is a no-op for tables that don't exist, then copies all data. No manual DROP is needed — the tool self-manages everything. - Tables exist but data copy was interrupted mid-run — use Resume. Resume compares source vs target row counts per table: tables that already match are skipped, partially-loaded tables are cleared (DELETE, not DROP) and re-inserted in FK-safe order, and missing tables are queued for a fresh copy.
For acme_erp (all 99 data tables absent from target), Full migration is required. See Troubleshooting → Full vs Resume.
Does migrating from Azure SQL to AWS RDS require special settings?
No manual changes are needed. Endrias Bridge detects the
Azure SQL Database → AWS RDS for SQL Server pair and automatically
applies DDL compatibility conversions: json → NVARCHAR(MAX),
NullType (hierarchyid/spatial) → NVARCHAR(MAX),
decimal precision clamped to 38, and Azure-only server defaults stripped.
Each conversion is logged. See
Azure SQL → AWS RDS Compatibility.