How It Works

The Bigquery synapse enables data synchronization between Bigquery’s schema and Syncari entities. Using the Syncari’s Sync Studio, users can map Bigquery tables to Syncari entities. Data synchronization can be controlled at the entity level, allowing the user to turn off synchronization for a single entity.

Once the entities are enabled in the Sync Studio for synchronization, the initial sync starts. All existing records for the entity are pulled efficiently from Bigquery and synced to Syncari. This is called the inbound pipeline for the entity. Once all existing data is extracted, the sync switches automatically to incremental sync. During incremental sync, the delta changes from Bigquery are synced to Syncari in near real time.

To perform the incremental data load from Bigquery to Syncari, the watermark field needs to be defined for every Bigquery table in Syncari.

Supported Objects and Workflow

The Bigquery Synapse allows synchronization on the below listed objects:

• All Tables In Bigquery dataset

Syncari synchronizes Bigquery’s schema and its mappings with Syncari’s schema during every sync cycle. The newly added columns on the Bigquery tables and newly added tables are detected automatically and available for usage in the pipelines. They are not activated by default. The deleted fields and tables in Bigquery are detected automatically. These objects and fields are marked as deleted in Syncari. They no longer participate in synchronization.