Collection Compare

The Collection Compare activity enables you to analyze differences between MongoDB collections across databases, environments, or connections — and between SQL tables and MongoDB collections via Debezium change data capture (CDC). With visual drag-and-drop link setup, categorized field-level diffs, and bidirectional sync plans, you can reconcile discrepancies and keep heterogeneous systems consistent.

Compare Modes

VisuaLeaf supports two comparison modes selected automatically based on the source connection type:

• MongoDB ↔ MongoDB — Compare two Mongo collections on the same cluster, across two clusters, or between environments (staging vs. production). Uses direct cursor scans against both sides.
• SQL → MongoDB — Compare a relational table (PostgreSQL, MySQL, MariaDB, SQL Server, Oracle) against a Mongo collection. Uses Debezium CDC to consume the write-ahead log / binlog, so the SQL side reflects committed rows at the exact snapshot moment — no locking, no impact on OLTP workload.

Comparison Setup

The comparison setup uses a three-panel layout with drag-and-drop functionality to create collection links.

Source and Target Panels

Both panels display your connected MongoDB instances in a hierarchical tree:

• Connection nodes: Show connection name with online/offline status (click broken link icon to connect)
• Database nodes: Expandable to show collections
• Collection nodes: Draggable items for creating comparison links
• Search filter: Filter nodes by name in each panel

The Source panel (teal) represents your reference data. The Target panel (orange) represents the destination you're comparing against.

Creating Comparison Links

To link collections for comparison:

1. Expand the connection and database in the Source panel
2. Drag a collection from the Source panel
3. Drop it onto a collection in the Target panel
4. A colored leader line appears connecting the two collections
5. The link appears in the Comparison Links panel below

Comparison Links Panel

Each link in the panel displays:

• Color indicator: Click to change the link line color
• Source/Target info: Collection names, databases, and document statistics
• Time estimate: Estimated comparison duration based on collection size
• Match Keys: Fields used to identify matching documents (displayed as blue badges)
• Source/Target Filters: Optional query filters to limit compared documents
• Configure button: Open detailed link settings
• Remove button: Delete the comparison link

Link Configuration

Click the configure (gear) button on a link to open the Link Configuration modal:

Option	Description	Default
Match Keys	Fields used to identify matching documents between collections (e.g., _id, email, orderId)	_id
Source Filter	MongoDB query to filter source documents (e.g., {"status": "active"})	None
Target Filter	MongoDB query to filter target documents	None

Session Management

Comparison configurations can be saved and reused:

• Session Name: Enter a name for your comparison session in the header
• Save Session: Click to save the current link configuration (shown for new sessions)
• Tracked Session: Green badge indicates the session is saved and will auto-update on changes

Running Comparisons

Click the "Run Comparison" button to execute. The comparison runs as a background job with progress tracking:

• Progress indicator shows current operation and elapsed time
• Cancel button available to abort long-running comparisons
• Results are stored in the configured storage directory

Comparison Results

Results open in a new activity tab with multiple views:

Result Tabs

Tab	Description
Summary	Overview with timing, status, document counts, differences breakdown, and comparison configuration
All	Combined view of all comparison results
Modified	Documents that exist in both collections but have different field values
Missing in Source	Documents present in target but not in source
Missing in Target	Documents present in source but not in target
Log	History of sync operations performed on this comparison

Results Table

The results table displays documents with the following columns:

• Checkbox: Select documents for sync operations
• Expand arrow: For modified documents, click the row or arrow to expand and see field-level differences
• Document ID: The match key value identifying the document
• Status: MODIFIED, MISSING_IN_RIGHT, MISSING_IN_LEFT, or IDENTICAL
• Source Value: Document preview from source (click to open in Collection View)
• Target Value: Document preview from target (click to open in Collection View)

Viewing Field Differences

For modified documents, click anywhere on the row to expand and see which fields differ. Each field row shows:

• Field Path: The path to the changed field (e.g., "address.city")
• Status badge: MODIFIED, ADDED, or REMOVED
• Source Value: The value from the source collection
• Target Value: The value from the target collection

Field-Level Diff Detection

Each document is classified into one of four buckets so you can filter results and design targeted sync plans:

Bucket	Meaning
Identical	Match keys align and every compared field is byte-equal on both sides
Modified	Match keys align but one or more fields differ — expandable diff shows path, source value, target value
Missing in Target	Document exists in the source (or SQL row exists) but not in the target Mongo collection
Missing in Source	Document exists in the target but not in the source

Nested paths are compared with dot notation. Arrays are compared element-by-element with configurable order-sensitivity. BSON types (ObjectId, Date, Decimal128) are normalized before comparison so a Mongo Date equals a Postgres timestamptz at the same instant.

Walkthrough: Compare a Postgres users Table to a Mongo users Collection

This example uses Debezium CDC to compare a PostgreSQL public.users table against a Mongo app.users collection — a common check after migrating an auth service off Postgres.

Prerequisites

• PostgreSQL with wal_level = logical (edit postgresql.conf and restart)
• A replication user: CREATE ROLE debezium REPLICATION LOGIN PASSWORD '...';
• The pgoutput plugin (bundled with Postgres 10+) or wal2json
• A MongoDB connection with read access to the target collection

Step 1 — Add the Postgres connection

Open Connection Manager and add a PostgreSQL connection using a URI like:

jdbc:postgresql://db.internal:5432/appdb?user=debezium&password=...&sslmode=require

Step 2 — Create the compare link

1. Open the Collection Compare activity
2. In the Source panel, expand the Postgres connection → database → public schema, then drag the users table
3. Drop it onto app.users in the Target Mongo panel — a leader line connects them
4. Click the gear icon on the new link to open Link Configuration

Step 3 — Configure match keys and filters

Set the following in the Link Configuration modal:

• Match Keys: id on the SQL side, _id on the Mongo side (VisuaLeaf coerces types for you)
• Source Filter (SQL): WHERE deleted_at IS NULL to exclude soft-deleted rows
• Target Filter (Mongo): {"deletedAt": {"$exists": false}}
• Field Mapping: map created_at → createdAt, email_address → email, etc.

Step 4 — Run the comparison

Click Run Comparison. Behind the scenes VisuaLeaf spins up a Debezium 2.5.4 embedded engine, takes an initial snapshot of the filtered rows, and streams them into the diff engine while the Mongo cursor reads in parallel. Live progress shows rows read from Postgres, documents read from Mongo, and diffs computed so far.

Step 5 — Review and sync

Once complete, open the Modified tab to inspect fields that drifted — for example a user whose email changed in Postgres but was never re-synced to Mongo. Select rows, choose direction Source (Postgres) → Target (Mongo), and click Generate Sync Plan. The plan will show upserts scoped to just the drifted documents. Because Debezium keeps its offset, you can re-run the compare later and only the newly-drifted rows will show up.

Sync Operations

After reviewing comparison results, you can synchronize selected documents between collections.

Sync Direction (Bidirectional Plans)

Use the direction toggle in the action bar to set sync direction. Plans are fully bidirectional — the same comparison result can produce either a forward sync or a reverse sync without re-running the compare.

• Source → Target: Push changes from source to target collection
• Target → Source: Pull changes from target back to source collection

Click the toggle button to swap direction. The labels animate to show the current direction clearly. Note: for SQL → MongoDB compares, only the Source → Target direction is enabled by default (writes back to Postgres/MySQL are gated behind an explicit setting to prevent accidental production writes).

Live Progress Monitoring

Both compares and syncs stream progress in real time using Server-Sent Events (SSE):

• Rows read from source per second
• Documents read from target per second
• Running diff bucket counts (identical / modified / missing)
• ETA calculated from throughput of the last 30 seconds
• Cancel button aborts the underlying cursors and closes the Debezium engine cleanly

Generating a Sync Plan

1. Select documents using checkboxes (or use Select All/Deselect All buttons)
2. Set the sync direction using the toggle
3. Click "Generate Sync Plan"
4. Review the sync plan preview showing all operations

Sync Plan Operations

Operation Type	Description
Insert	Document will be inserted into the target collection (for missing documents)
Update/Replace	Existing document will be replaced with source values (for modified documents)
Delete	Document will be removed from the target collection

Executing Sync

1. Review the sync plan operations carefully
2. Check the estimated sync time at the bottom of the modal
3. Click "Execute Sync" to apply changes
4. Monitor progress as operations complete
5. Check the Log tab for operation history and any errors
6. Use "Re-compare" button in the header to verify sync results

Sync Log

The Log tab records all sync operations with:

• Operation type: Insert, Update, Replace, Delete, or Recompare
• Status: Success or Failed
• Collection and database: Where the operation was performed
• Match key value: The document identifier
• Timestamp: When the operation occurred
• Error message: Details if the operation failed

Additional Features

Opening Documents

Click on source or target values in the results table to open documents:

• Click: Open document in Collection View
• Cmd/Ctrl + Click: Open in a new panel
• Bulk Open: Click the external link icon in column headers to open multiple documents

Storage Configuration

Comparison results are stored locally. Click the storage path displayed in the results header to open settings and configure the results directory.

Re-comparing

Click "Re-compare" in the results header to run the comparison again with the same configuration. Useful after executing sync operations to verify changes were applied correctly.

Common Use Cases

Environment Synchronization

Compare development and production collections to identify drift. Use sync operations to push or pull changes between environments.

Data Migration Validation

After migrating data, compare source and destination to verify completeness. Check for missing or modified documents.

Backup Verification

Compare a collection against its backup to ensure data integrity and identify any discrepancies.

Multi-Region Consistency

Compare collections across different MongoDB instances or regions to ensure data consistency.

Pro Tips

1. Use filters for large collections: Add source and target filters to limit comparison scope and improve performance.
2. Choose appropriate match keys: Use unique identifiers like _id or business keys (email, orderId) for accurate document matching.
3. Save sessions for recurring comparisons: Named sessions preserve your link configuration for quick future comparisons.
4. Review before syncing: Always review the sync plan operations before executing to prevent unintended changes.
5. Check the Log tab: After sync operations, review the log for any failed operations or errors.
6. Re-compare after sync: Run a new comparison to verify that sync operations completed successfully.

Related Features

• Collection Activity - Open and edit individual documents from comparison results
• Export Data - Create backups before performing sync operations
• Collection Statistics - Review collection sizes before running large comparisons