SQL Server → Redis Migration
Key-Value Mapping Strategy
MigrationBridge represents each SQL Server row as a Redis Hash. Every column becomes a Hash field; the key is derived from the table's primary key(s). When no primary key exists, a sequential row_number is used as a fallback identifier.
Key Pattern Format
| Scenario | Key Pattern | Example | Notes |
|---|---|---|---|
| Single-column PK (INT / BIGINT) | {schema}:{table}:{pk_value} | HumanResources:Employee:1 | Most common; compact and collision-free |
| Single-column PK (UNIQUEIDENTIFIER) | {schema}:{table}:{guid} | Sales:Customer:6A2C7B3E-… | GUIDs are stored as uppercase strings |
| Composite PK (2+ columns) | {schema}:{table}:{v1}:{v2}:… | Sales:OrderDetail:43659:1 | Columns joined with : in PK ordinal order |
| No PK (heap / view) | {schema}:{table}:row:{n} | dbo:ErrorLog:row:42 | Sequential — stable only within a single migration run |
| Custom namespace prefix | {prefix}:{schema}:{table}:{pk} | prod:HumanResources:Employee:1 | Optional; configured in the Namespace field |
Hash Field Example — HumanResources:Employee:1
1) "BusinessEntityID"
2) "1"
3) "NationalIDNumber"
4) "295847284"
5) "LoginID"
6) "adventure-works\ken0"
7) "JobTitle"
8) "Chief Executive Officer"
9) "BirthDate"
10) "1969-01-29"
11) "HireDate"
12) "2009-01-14"
13) "VacationHours"
14) "99"
15) "SickLeaveHours"
16) "69"
17) "rowguid"
18) "F01251E5-96A3-448D-981E-0F99D789110D"
19) "ModifiedDate"
20) "2014-06-30 00:00:00"
TTL Behaviour
| TTL Setting | Behaviour | Use Case |
|---|---|---|
| 0 (default) | No expiry — keys persist until evicted by memory policy or manual DEL | Reference / lookup data, master tables |
| > 0 (seconds) | EXPIRE is called on every key after HSET | Session data, cache warm-up, temp staging |
| Per-table override | Available in Advanced mode; overrides the global TTL for that table's keys | Mixed retention requirements |
TTL is not persisted. If you migrate with TTL > 0 and the Redis instance restarts without AOF/RDB persistence, all keys expire. Ensure persistence is configured before setting TTL on production data.
SQL Server Readiness
Confirm the following before launching MigrationBridge. All checks must pass; failed checks produce warnings in the assessment panel (Step 2).
Minimum Permission Grant
-- Run as db_owner or sysadmin on the source database
USE [AdventureWorks2019];
GO
-- Create login if it does not exist
IF NOT EXISTS (
SELECT 1 FROM sys.server_principals
WHERE name = N'MigrationBridgeSvc'
)
BEGIN
CREATE LOGIN [MigrationBridgeSvc]
WITH PASSWORD = N'<strong_password>';
END
GO
-- Create user and assign db_datareader
IF NOT EXISTS (
SELECT 1 FROM sys.database_principals
WHERE name = N'MigrationBridgeSvc'
)
BEGIN
CREATE USER [MigrationBridgeSvc]
FOR LOGIN [MigrationBridgeSvc];
END
GO
ALTER ROLE db_datareader ADD MEMBER [MigrationBridgeSvc];
GO
ODBC Driver Check
Get-OdbcDriver | Where-Object { $_.Name -match 'SQL Server' } |
Select-Object Name, Platform
# Expected output includes:
# ODBC Driver 17 for SQL Server 64-bit
Pre-Flight T-SQL Checks
Run these four queries on the source database before starting the migration. Review output in the assessment step.
SELECT
SCHEMA_NAME(t.schema_id) AS SchemaName,
t.name AS TableName,
SUM(p.rows) AS EstimatedRows
FROM sys.tables t
JOIN sys.partitions p
ON t.object_id = p.object_id
AND p.index_id IN (0, 1)
GROUP BY t.schema_id, t.name
ORDER BY EstimatedRows DESC;
SELECT
SCHEMA_NAME(t.schema_id) AS SchemaName,
t.name AS TableName
FROM sys.tables t
WHERE NOT EXISTS (
SELECT 1
FROM sys.indexes i
WHERE i.object_id = t.object_id
AND i.is_primary_key = 1
)
ORDER BY SchemaName, TableName;
SELECT
SCHEMA_NAME(t.schema_id) AS SchemaName,
t.name AS TableName,
c.name AS ColumnName,
TYPE_NAME(c.user_type_id) AS DataType,
c.max_length
FROM sys.columns c
JOIN sys.tables t ON c.object_id = t.object_id
WHERE c.max_length = -1 -- -1 = MAX in SQL Server
OR TYPE_NAME(c.user_type_id) IN ('xml', 'image', 'text', 'ntext')
ORDER BY SchemaName, TableName, c.column_id;
SELECT
SCHEMA_NAME(t.schema_id) AS SchemaName,
t.name AS TableName,
COUNT(ic.column_id) AS PKColumnCount,
STRING_AGG(c.name, ', ')
WITHIN GROUP (ORDER BY ic.key_ordinal) AS PKColumns
FROM sys.indexes i
JOIN sys.tables t ON i.object_id = t.object_id
JOIN sys.index_columns ic ON i.object_id = ic.object_id
AND i.index_id = ic.index_id
JOIN sys.columns c ON ic.object_id = c.object_id
AND ic.column_id = c.column_id
WHERE i.is_primary_key = 1
GROUP BY t.schema_id, t.name
HAVING COUNT(ic.column_id) > 1
ORDER BY PKColumnCount DESC;
Redis Readiness
Redis 5.0 or higher is required. MigrationBridge uses the HSET variadic form (Redis 4+) and checks server version at connect time.
redis-py Version Check
pip show redis
# Minimum required: redis>=4.0.0 (redis-py; not valkey)
# Install / upgrade
pip install "redis>=4.0.0"
Connection String Reference
| Deployment Target | Connection String / Host | Notes |
|---|---|---|
| Local (no auth) | localhost:6379 | Default dev setup, no password |
| Local (with auth) | localhost:6379 + password field | requirepass set in redis.conf |
| Azure Cache for Redis (Basic/Standard) | <name>.redis.cache.windows.net:6380 | TLS on port 6380; use SSL toggle |
| Azure Cache for Redis (Premium) | <name>.redis.cache.windows.net:6380 | VNet injection; confirm NSG allows 6380 outbound |
| AWS ElastiCache (in-transit disabled) | <cluster>.abc123.cfg.use1.cache.amazonaws.com:6379 | Must run from within same VPC |
| AWS ElastiCache (TLS enabled) | <cluster>.abc123.cfg.use1.cache.amazonaws.com:6380 | Enable SSL; AUTH token required if auth-token set |
Connectivity Test — redis-cli
$ redis-cli -h localhost -p 6379 PING
PONG
# Auth test (Azure / ElastiCache)
$ redis-cli -h myinstance.redis.cache.windows.net -p 6380 --tls -a <access_key> PING
PONG
# Check server version (must be >= 5.0.0)
127.0.0.1:6379> INFO server | grep redis_version
redis_version:7.2.4
Version requirement: Redis 5.0+ required. Redis 4.x supports variadic HSET but is EOL. Versions below 4.0 use the deprecated HMSET command which is unsupported by MigrationBridge.
Connection Setup
Enter credentials for both SQL Server (source) and Redis (target) in the MigrationBridge connection panel. Use the Test Connection buttons to verify before proceeding.
■ Source — SQL Server
■ Target — Redis
DB Number (0–15) is a logical database partition available in standalone Redis. Azure Cache for Redis Basic/Standard supports DBs 0–15. Azure Cache Premium and AWS ElastiCache in Cluster mode only support DB 0 — entering any other value will cause a connection error.
Schema Assessment
MigrationBridge introspects the source schema and displays a compatibility report. Review all warnings before proceeding to table selection.
PK-less Table Fallback
For tables without a primary key, MigrationBridge assigns a synthetic key using ROW_NUMBER():
SELECT
ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS _mb_row_id,
*
FROM [dbo].[ErrorLog];
-- Resulting Redis key: dbo:ErrorLog:row:1, dbo:ErrorLog:row:2, …
-- WARNING: row numbers are NOT stable across re-runs unless the table
-- has a deterministic physical order (clustered index or ORDER BY).
BLOB / Binary Hex-Encoding Warning
VARBINARY, IMAGE, and VARBINARY(MAX) columns are hex-encoded as strings in Redis Hash fields (e.g., 0x89504E47…). Values exceeding 512 MB (Redis single value limit) will raise an error and that row will be skipped. A count of skipped rows appears in the migration log.
Type Coercion Summary
| SQL Server Type | Redis Storage | Example SQL Value | Stored as |
|---|---|---|---|
| INT / BIGINT | String (integer) | 42 | "42" |
| DECIMAL / NUMERIC | String (decimal) | 19.99 | "19.99" |
| BIT | String (0 or 1) | True | "1" |
| DATETIME / DATETIME2 | ISO 8601 string | 2024-01-15 08:30:00.000 | "2024-01-15 08:30:00" |
| UNIQUEIDENTIFIER | Uppercase GUID string | 6a2c7b3e-… | "6A2C7B3E-…" |
| NULL | Empty string | NULL | "" |
| VARBINARY | Hex string | 0x0A0B0C | "0x0A0B0C" |
| XML | String (raw XML) | <root/> | "<root/>" |
Assessment Summary — AdventureWorks2019
Table Selection
Select the tables to include in the migration. Deselect tables with BLOB-heavy columns or those not required in Redis to reduce key space and migration time.
Composite PK Key Format
For composite primary keys, column values are joined with : in primary key ordinal order:
127.0.0.1:6379> HGETALL Sales:SalesOrderDetail:43659:1
1) "SalesOrderID"
2) "43659"
3) "SalesOrderDetailID"
4) "1"
5) "OrderQty"
6) "1"
7) "UnitPrice"
8) "2024.9940"
Collision warning: If a PK value itself contains : (colon), the key segments will be ambiguous. MigrationBridge URL-encodes values containing colons (replaces : with %3A) to prevent collisions.
Best practice: Deselect ProductPhoto and any audit/log tables before migrating to Redis. BLOB data in Redis significantly increases memory consumption and these tables are rarely accessed as key-value lookups.
Run Migration
Configure migration options and start the transfer. MigrationBridge pipelines HSET commands using redis-py pipeline batches (default batch size: 500 commands).
■ Migration Options
Validate Data
Run the following checks in sequence. All validations should pass before decommissioning the SQL Server source or promoting the Redis instance to production traffic.
1 — DBSIZE (Total Key Count)
(integer) 740311
Compare to the row count from Check 1 (pre-flight). A difference of 2 here matches the 2 skipped rows reported in the migration log.
2 — HGETALL Spot-Check
1) "BusinessEntityID" 2) "1"
3) "NationalIDNumber" 4) "295847284"
5) "LoginID" 6) "adventure-works\ken0"
7) "JobTitle" 8) "Chief Executive Officer"
3 — Per-Table SCAN Count
127.0.0.1:6379> EVAL "local c=0; local cur=0; repeat local r=redis.call('SCAN',cur,'MATCH','Sales:SalesOrderHeader:*','COUNT',500); cur=tonumber(r[1]); c=c+#r[2]; until cur==0; return c" 0
(integer) 31465
4 — Interactive SCAN Cursor (redis-cli)
127.0.0.1:6379> SCAN 0 MATCH Production:Product:* COUNT 100
1) "342" ← next cursor (not 0, so more results remain)
2) 1) "Production:Product:316"
2) "Production:Product:707"
3) "Production:Product:708"
… (up to ~100 keys returned per call)
127.0.0.1:6379> SCAN 342 MATCH Production:Product:* COUNT 100
1) "0" ← cursor 0 = iteration complete
2) … remaining keys …
5 — TTL Check
(integer) -1 ← -1 = no expiry set (TTL=0 in migration options)
6 — HLEN Field Count
(integer) 16
# Compare to column count from SQL Server
# SELECT COUNT(*) FROM sys.columns WHERE object_id = OBJECT_ID('HumanResources.Employee') → 16
7 — Cross-Validation T-SQL (Source vs Redis Count)
-- Compare output against SCAN counts or DBSIZE on Redis side
SELECT
SCHEMA_NAME(t.schema_id) AS SchemaName,
t.name AS TableName,
SUM(p.rows) AS SourceRows,
SCHEMA_NAME(t.schema_id) + ':' + t.name + ':*' AS RedisScanPattern
FROM sys.tables t
JOIN sys.partitions p
ON t.object_id = p.object_id
AND p.index_id IN (0, 1)
GROUP BY t.schema_id, t.name
ORDER BY SourceRows DESC;
Post-Migration Configuration
After validating data, apply production-readiness settings to the Redis instance before routing application traffic.
Memory Snapshot — INFO memory
used_memory:412680192
used_memory_human:393.65M
used_memory_rss:451215360
used_memory_rss_human:430.36M
used_memory_peak:414220288
used_memory_peak_human:395.12M
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
Encoding Check — OBJECT ENCODING
"ziplist" ← efficient for small hashes (<128 fields, values <64 bytes)
127.0.0.1:6379> OBJECT ENCODING Sales:SalesOrderDetail:43659:1
"hashtable" ← promoted to hashtable (larger values or fields >128)
Set maxmemory-policy (Production)
127.0.0.1:6379> CONFIG SET maxmemory 2gb
OK
127.0.0.1:6379> CONFIG SET maxmemory-policy allkeys-lru
OK
Persistence Configuration Reference
| Mode | Config | Durability | Performance Impact | Use Case |
|---|---|---|---|---|
| RDB only | save 900 1 / save 300 10 | Point-in-time snapshot; may lose up to 5 min of data | Low — background fork | Cache warm-up, reference data |
| AOF only | appendonly yes appendfsync everysec |
Up to 1 second of data loss max | Medium — fsync every second | Session or transaction data |
| RDB + AOF | Both enabled + aof-use-rdb-preamble yes | Strongest; AOF used for recovery, RDB for fast restart | Medium-high | Production workloads requiring RPO < 1s |
| None | save "" + appendonly no | No persistence — all data lost on restart | Lowest | Pure ephemeral cache (re-populate on restart) |
Post-Migration Checklist
- DBSIZE matches expected key count (within skipped row tolerance)
- Spot-checked HGETALL on ≥5 representative rows across different tables
- HLEN matches column count from sys.columns for each checked table
- TTL confirmed (−1 for no-expiry or expected seconds for TTL-enabled keys)
- maxmemory and maxmemory-policy set per workload requirements
- Persistence mode configured (RDB, AOF, or both) and verified via CONFIG GET save
- ACL / password authentication enabled if not already (Redis 6+: ACL SETUSER)
- BLOB/MAX columns reviewed — large binary fields confirmed acceptable in Redis memory budget
- Application connection strings updated to point to Redis endpoint
- Monitoring/alerting configured (CloudWatch, Azure Monitor, or redis-cli INFO stats baseline)
Type Conversion Reference
Complete mapping of all SQL Server data types to Redis Hash field string representations used by MigrationBridge.
| SQL Server Type | Example SQL Value | Redis Field Value (string) | Notes |
|---|---|---|---|
| INT | 42 | "42" | Lossless; Redis INCR/INCRBY compatible |
| BIGINT | 9876543210 | "9876543210" | Lossless |
| FLOAT | 3.14159 | "3.14159" | Python str(float) — may have rounding artifacts |
| DECIMAL / NUMERIC | 19.9900 | "19.9900" | Trailing zeros preserved from pyodbc Decimal |
| MONEY / SMALLMONEY | 1234.5600 | "1234.5600" | 4 decimal places preserved |
| BIT | True / False | "1" / "0" | Boolean mapped to 1/0 |
| VARCHAR / NVARCHAR / CHAR | Hello World | "Hello World" | UTF-8; null bytes stripped |
| TEXT / NTEXT | Long text… | "Long text…" | Deprecated types; treated as VARCHAR(MAX) |
| DATETIME | 2024-01-15 08:30:00.000 | "2024-01-15 08:30:00" | Milliseconds stripped; ISO 8601 format |
| DATETIME2 | 2024-01-15 08:30:00.1234567 | "2024-01-15 08:30:00.123456" | Microsecond precision; 7th digit truncated |
| DATE | 2024-01-15 | "2024-01-15" | ISO 8601 date string |
| TIME | 08:30:00.0000000 | "08:30:00" | Fractional seconds stripped if zero |
| DATETIMEOFFSET | 2024-01-15 08:30:00 +05:30 | "2024-01-15 08:30:00+05:30" | Offset preserved; no space before +/− |
| UNIQUEIDENTIFIER | 6a2c7b3e-4f1d-4b2a-9c8e-d3f2a1b0c5e7 | "6A2C7B3E-4F1D-4B2A-9C8E-D3F2A1B0C5E7" | Uppercased; standard 8-4-4-4-12 format |
| VARBINARY / BINARY | 0x0A0B0C | "0x0A0B0C" | Hex-encoded with 0x prefix |
| VARBINARY(MAX) / IMAGE | (binary blob) | "0x89504E47…" | Full hex; rows >512 MB skipped with log entry |
| XML | <root></root> | "<root></root>" | Raw XML string; no validation or formatting |
| NULL (any type) | NULL | "" | Empty string; use HLEN to distinguish missing field vs empty |
Known Issues & Limitations
Understand the structural differences between a relational model and a key-value store before migrating. These limitations are inherent to the paradigm shift, not bugs in MigrationBridge.
| # | Issue / Limitation | Impact | Mitigation |
|---|---|---|---|
| 1 | No SQL query capability | Applications that use SELECT … WHERE column = value cannot be replicated in vanilla Redis — Redis is a key-value store | Use Redis Search (RediSearch module) or maintain a secondary index (e.g., a Redis Set per lookup value) |
| 2 | No JOINs | Foreign-key relationships are not preserved; related data must be fetched with multiple round trips | Denormalize data before migrating, or use application-layer join logic |
| 3 | TTL causes data loss on restart | If TTL > 0 and persistence is not configured, all keys expire after restart | Configure RDB or AOF persistence before setting TTL on production data |
| 4 | Namespace collisions | If the same Redis DB already contains keys matching Schema:Table:PK patterns, they will be silently overwritten (HSET semantics) | Use a unique namespace prefix, flush the DB before migrating, or use a dedicated DB number |
| 5 | Azure Cache SSL certificate | If the client system is missing the Baltimore CyberTrust / DigiCert root CA, TLS handshake fails with SSL: CERTIFICATE_VERIFY_FAILED | Update the OS/Python certificate store; or pass ssl_cert_reqs=None in dev only (not production) |
| 6 | ElastiCache VPC isolation | ElastiCache clusters are not publicly accessible; connection from outside the VPC will time out | Run MigrationBridge from an EC2 instance in the same VPC, or use an SSH tunnel / VPN |
| 7 | Redis Cluster hash slots | In Cluster mode, keys are distributed across 16,384 hash slots by key prefix. Cross-slot multi-key operations (MGET, pipeline) may fail with CROSSSLOT error | MigrationBridge uses individual HSET commands in Cluster mode (pipeline disabled automatically when Cluster is detected) |
| 8 | DBSIZE inaccurate in Cluster mode | DBSIZE returns the count for a single shard, not the total cluster key count | Use redis-cli --cluster call <host>:<port> DBSIZE to sum across all shards |
UI Element States During & After Migration
MigrationBridge disables or modifies certain UI controls while a migration is running to prevent accidental interruption or double-execution.
| UI Element | During Migration | After Completion | After Failure / Cancel |
|---|---|---|---|
| Start Migration button | Disabled — replaced by progress bar | Enabled — re-run allowed | Enabled — retry allowed |
| Cancel button | Active — graceful stop (finishes current batch) | Hidden | Hidden |
| Connection fields (SQL Server) | Locked — read-only | Editable | Editable |
| Connection fields (Redis) | Locked — read-only | Editable | Editable |
| Table selector checkboxes | Locked | Editable | Editable |
| Migration options fields | Locked | Editable | Editable |
| Log panel | Live scroll — auto-scrolls to latest entry | Scrollable — auto-scroll stops | Scrollable — last entry shows cancel/error reason |
| Export Log button | Disabled | Enabled — saves .txt log file | Enabled — partial log exportable |
Cancel graceful-stop behaviour: Clicking Cancel signals MigrationBridge to stop after the current pipeline batch completes. Keys already written to Redis are not rolled back — partial migration state is preserved. The log will record how many keys were written before cancel and which table was in progress.