Composable CDP built on Google BigQuery

What is a composable CDP built on BigQuery?

A composable customer data platform (CDP) is a marketing technology infrastructure built from multiple, interchangeable components that consolidates a company’s customer data for activation across channels. Composable CDPs are built on top of a company's cloud data warehouse, and provide a single source of truth for all customer data. They differ from other types of CDP by being modular, flexible, and customizable.

BigQuery is a serverless data warehouse solution built on Google Cloud, which offers tools to store, analyze, and share data for organizations. A BigQuery composable CDP is a composable CDP that uses BigQuery as the data warehouse for its foundation. 

Why would I want to build a composable CDP on BigQuery?

Composable CDPs provide greater flexibility for data and marketing teams — they integrate with a variety of data from multiple sources and a variety of channels for data activation. This means you can use the best-in-breed solutions or the best solutions for your company — including a data warehouse solution like BigQuery.

Using a composable CDP and storing your customer data in BigQuery also gives your organization more control over the data than if you stored it in a traditional CDP solution. A traditional or packaged CDP requires copying data to a third party source. But using BigQuery means your organization has more direct access to your data and control over who can access that data through privacy controls and security settings.

How do I know a composable CDP on BigQuery is right for my company?

A composable CDP is a good solution for organizations that aren’t otherwise supported by traditional CDP. Composable CDPs benefit organizations that have strict regulatory compliance needs, have too much data, don’t want to be locked into a specific vendor, or need more value from their data warehouse. Compared to a traditional CDP, composable CDPs also offer greater flexibility for growing organizations because they don’t have limitations on data storage or sources and can add or swap-out marketing channels as needed.

Organizations that can benefit from a composable CDP on BigQuery include:

• Operate on a hub-and-spoke model, such as an airline or sports team

• Use business-to-business (B2B) data models that don’t work with traditional CDPs

• Have too much data for your current CDP solution

If your organization intends to build a composable CDP, BigQuery is a good option for organizations that are already using Google Cloud Platform (GCP) tools for other parts of your business, as BigQuery is already integrated with those solutions. Google is a leader in data analytics tools and has a strong track record. Using BigQuery gives your existing tools access to your data stack, providing fast time-to-value. It also means you can pay one subscription for the Google Services you are using rather than having a separate data storage solution and integrating it with your Google tools.

What features should I look for in a composable CDP that works with BigQuery?

Other than compatibility with BigQuery, the features you should look for when building a composable CDP on BigQuery include:

• Data portability: If you are not already hosting your data warehouse in BigQuery, look into how to transport your data from your current solution into BigQuery. A variety of tools may be able to help, including some through Google.

• Identity resolution: When collecting customer information, you will get duplicate data about the same customer. Identity resolution solutions allow your data warehouse to consolidate all that data about one person into a single entry, streamlining your data and giving a more complete picture of each customer.

• Data activation: Data collected in your warehouse is not active, it is a collection of gathered and behavioral facts about a customer. Data activation is the process of making that data into information that marketing applications can use to build customer profiles and customer journeys to use in campaigns.

• Machine learning (ML) and artificial intelligence (AI): Many marketing tools now use algorithms and other technologies that draw on machine learning to automate and streamline processes using your existing data. These tools can take your data and automatically generate insights, profiles, and more. 

• Security and compliance: Given the number of data privacy regulations and laws worldwide, verify that any composable CDP solutions you are considering will meet the standards for any data laws that apply to your business. When in doubt, consult your legal team or a lawyer.

How do I set up BigQuery to work with a composable CDP?

Setting up a composable CDP on BigQuery requires some preparation before the technical efforts begin. This includes deciding on the right tools and solutions for your teams, including choosing BigQuery and a suite of compatible tools. Preparation also requires cleaning your data, which means performing an audit of the data you currently have and clearing out any duplicate or test records, addressing any inaccuracies, and deciding what to do with any legacy records you may have.

After that, you will need to get buy-in from the relevant teams and assemble a team of subject matter experts (SMEs) (see the below section on buy-in). These experts will help you put the composable CDP together and create a data governance plan for managing changes in data structure, new sources of data, and the feedback loop that defines who’s involved and what their responsibilities are. 

Keep in mind that you may need to check the vendors’ websites and documentation for specific compatibility with BigQuery. Most Google tools should be compatible with BigQuery.

What team members do I need to implement a composable CDP on BigQuery?

After securing buy-in from key stakeholders, you will need to work with your marketing and data teams. These teams will be the main users and managers for the composable CDP and its data. 

• Ask the data teams for details about how data is collected, including the tools and channels they use to import the data into the data warehouse solutions you currently have. 

• Work with them to refine or define data governance policies that dictate how you organize the data that you collect. These policies can be as granular as date formats and names of fields, or as broad as who is responsible for fixing data errors.

• Ask the marketing teams for information about how they use the data and any goals they may have for the data, such as developing certain audience profiles or using generative AI tools. 

• Work with them to determine data gaps they currently have, and how the composable CDP can provide access to that data. Also, ensure you understand which tools they need for activating your data across marketing channels. This can include any tools or dashboards they need to monitor data for their campaigns.

Many solutions that work with composable CDPs are built with data analysts and marketers in mind, so the tools support low-code or no-code use with minimal set-up. For cloud-hosted tools, much of the technical maintenance happens on the vendor’s end. However, consult with your engineering team about any on-premises hosted components that interact with your composable CDP. 

What steps are involved in connecting a composable CDP to BigQuery?

With the team assembled, and support from the top, creating a CDP is largely about organizing your data and connecting the modular components your teams want to use. Consider also periodically testing the data you’re collecting to ensure it meets your business requirements and security policies.

At a high level, you will need to take the following steps

Create data governance policies

Your company is already collecting data through the customer data infrastructure you have in place. This is a framework of tools and functionalities that work with each other, to gather and store the customer data that your company uses. However, the variety of sources for this data means that the data may be coming in with different fields and formats. So, before you begin connecting your CDP tools, start by cleaning up the data and defining its governing rules. 

Data governance is a set of rules and processes about the data and its formatting. Work with your data and marketing teams to agree on the fields your data sources can collect. For example: date formats, “First Name” and “Last Name” being a single field or two separate fields, or using email addresses instead of usernames for customer accounts.

Your data governance policies should also set standards for security, legal compliance, and availability of the stored data. Depending on the kinds of personally identifiable information (PII) your company collects, you may be subject to certain regulations or laws. Your teams also need access to the data and have controls and policies to prevent hackers from getting in. 

Connect composable CDP modules

Consult the team members you have assembled to compile a list of all the tools and solutions you currently use and which ones your teams want to use. 

Determine which tools you need to configure first and which have lower priorities. Begin with the tools that collect data, so your data warehouse can begin filling with the events and customer data your other tools will work with. This also allows you to verify that you’ve set up your data governance properly as you move on to other stages.

You may need to consult vendor documentation or contact the vendor’s customer support teams for more details about connecting your specific components. As an example of what to look for, try to find something similar to GrowthLoop’s documentation on connecting to BigQuery.

Set up one tool at a time, and decide on a reasonable amount of time to let it run before moving on to the next one, such as a week or two. This makes it easier to troubleshoot if something breaks or impacts the data. 

Test the CDP and your models

At various points throughout the process, create reports and test the tools that you’re using. As your data governance policies develop, you will want reports that help you determine whether your tools follow those policies. This could include random samples of customer data, ensuring that all the fields look similar enough, or cybersecurity reviews to ensure your data is safe. 

As you connect other tools, you can work with your marketing and data teams to create models and reports based on their use cases. Run small batches of tests using those models to make sure that they’re getting the data they need. 

What best practices should I consider when using a composable CDP with BigQuery?

The following tips will help you set up and maintain your BigQuery-hosted composable CDP:

• Create policies for who to contact and what to do if any of your data or tools do not look or act as expected. This includes which teams are responsible for fixing the errors, as well as detailed procedures of how to fix issues if those teams are unavailable.

• Develop data hygiene policies. You can adapt these from the audit you conducted when you cleaned up your data before setting up the composable CDP and BigQuery.

• Set a cadence to check in with your composable CDP and tools, checking periodically to make sure that data tools are still collecting and operating as expected. While this could be time-consuming, it will be worth it to ensure you don’t lose data.

• Maintain a list of tools that interact with your composable CDP and BigQuery, to ensure that you properly maintain all of the components in the future and stay up to date with necessary security features. 

• Remember that composable CDPs are modular, which means that as your team’s needs change, you can swap out tools that no longer work properly or fit your needs. Develop a change management plan for these cases.