Recently, FiveTran announced a partnership with Census to “simplify end-to-end integration.” Despite their bold claim to simplify, we believe the new partnership actually complicates data unification initiatives while exploiting affected businesses by ensuring they pay for two tools to achieve an outcome that should have always been possible with one. Let’s dive in.
In this context, “end-to-end integration” means that with subscriptions to both products, a customer could create ELT (extract, load, and transform) workflows with FiveTran to bring operational data into a warehouse (e.g. Snowflake), analyze it, then send actionable insights back to the operational systems and people that need them using reverse ETL workflows created in Census. See the diagram below for more detail.
Before we discuss why this approach will create some significant operational headaches, let’s first discuss a few key reasons why someone might be willing to go down this path at all.
Growing need for data unification
Integrating data from two or three top systems just won’t cut it today. To compete, organizations are under tremendous pressure to bring together all their customer data, which is often spread out across tens, if not hundreds, of disparate applications. APIs, modern architectures, and ETL approaches are enabling this by providing real-time, streaming access to the data we need, but not all integrations are created equal.
Cry for help on data models
Every application, it seems, describes your data in a slightly different way, making 100% unification impractical, if not impossible. A generation from now, I expect they’ll look back at this time and wonder why we made life so hard on ourselves. Until then, customers are looking to vendors and technologies to help them align and transform disparate, ever-changing schemas and data into something more manageable: a unified data model.
The limitations of a data warehouse as your single source of truth
Data warehouse vendors like Snowflake are leading the market conversation right now because they’ve made it stupid simple to bring massive amounts of data into a central location for analysis. There are two obvious problems with this approach. (1) it creates a non-operational data silo and (2) insights need to be made available on a timely basis to the operational teams that need them to make day-to-day decisions — hence the emergence of Census and “Reverse ETL.”
3 solutions + operational friction vs. 1 solution + better data
And now, let’s discuss the limitations of delivering on the above with Census, FiveTran and Snowflake. If a user were to mimic the solution architecture of diagram 1, they’ll run into a few rather predictable headaches:
Operational complexity
When end-system schemas change (e.g. custom field gets added/deleted, API gets updated), how do those changes propagate across all three systems? Do FiveTran and Census update their connectors at the same time? Will a data model change in FiveTran or SnowFlake break any scripts running in Census? And what happens to data in transit while the connections and data models are out of sync? Managing this change is likely feasible if your deployments are simple – but I doubt any of these vendors only want you to solve simple problems with their solutions.
There’s also a cost consideration here, which has multiplied since FiveTran updated its pricing to a monthly active row model. Read more about the true cost of the modern data stack for some real dollar amounts.
Visibility gaps
Data is never perfect, and it seems the one constant with cross-functional data transactions is that something almost always goes wrong. The main problem with a 3 system configuration is that when something goes wrong, pinpointing WHERE it went wrong becomes extremely difficult. If you’ve ever experienced “swivel-chair management” you know how hard it is to correlate error messages in one management pane with those of another.
Cross-system data chaos
Any unification or normalization exercise will almost certainly result in data conflicts and database duplicates. Data warehouses control for this from a reporting and analytics lens, not an operations lens. Attempts to bring data that’s been standardized for reporting BACK into operational systems with some significant due diligence around HOW that data was used will almost certainly result in data chaos.
Modernize your data stack with Syncari
Syncari uniquely combines the ELT capabilities of FiveTran and the “reverse ETL” capabilities of Census in one complete platform to realistically simplify end-to-end integration. Having these two technology domains on one complete data automation platform, along with a fully managed data warehouse of your choosing (e.g. SnowFlake or RedShift), gives you unprecedented visibility, governance and control over your data without the operational headache of a multi-vendor approach.
As you can see in the diagram below, Syncari’s complete data automation platform delivers the basics from Census + FiveTran and so much more:
Let’s explore the capabilities in a bit more detail:
Automated data operations
- Self-healing connectivity to extract and load data
- Resilient, multi-directional sync to all your systems inline
- Automatic schema change management
Continuous data unification
- A unified data model that automatically normalizes cross-system data
- Codeless data transformation and management (or via dbt if that’s your thing)
- Centralized enrichment, merge and dedupe
Customer 360 + actionable insights everywhere
- Interactive, cross-system data views
- Cross-system Data Fitness Index™ + inline improvement recommendations
- Extensible to leading Data Lake and Warehouse platforms (e.g. Snowflake, Redshift)
- Extensible to the BI tool of your choice (e.g. Tableau, Looker)
- Lead management process flow
Why put up another barrier to operational efficiency and increased revenue when Syncari is designed to knock these barriers down? It’s time to truly simplify end-to-end integration. Learn more by visiting our On-Demand Demo Hub.
