Handling Changes to Schema

A schema for an integration is determined in one of two ways.

  • A user selects the tables and columns to replicate for a particular source

  • A predetermined schema is enabled for the integration which cannot be modified by a user.

A detailed list of the schema selection flexibility offered by Daton is documented in the table below.

Source

Table Selection

Field Selection

Adwords

Yes

Yes

AWS Aurora

Yes

Yes

AWS Redshift

Yes

Yes

BigCommerce

Yes

Yes

BingAds

Yes

Yes

Capillary

Yes

Yes

Chargebee

Yes

Yes

Criteo

Yes

Yes

Dropbox

Yes

Yes

Facebook Ads

Yes

Yes

Freshdesk

Yes

Yes

Fresh Sales

Yes

Yes

GCP MySQL

Yes

Yes

GCP PostgreSQL

Yes

Yes

Google Analytics

Yes

Yes

Intercom

Yes

Yes

Jira

Yes

Yes

Klaviyo

Yes

Yes

Linkedin

Yes

Yes

Magento

Yes

Yes

Mailchimp

Yes

Yes

MySQL

Yes

Yes

Optimove

Yes

Yes

Outbrain

Yes

Yes

PostgreSQL

Yes

Yes

PushEngage

Yes

Yes

Razorpay

Yes

Yes

RDS MySQL

Yes

Yes

RDS PostgreSQL

Yes

Yes

ReCharge Payments

Yes

Yes

Salesforce

Yes

Yes

Sendgrid

Yes

Yes

Shopify

Yes

Yes

Survey Monkey

Yes

Yes

Uniware

Yes

Yes

Zendesk

Yes

Yes

Zendesk Chat

Yes

Yes

Schema Change in Source

Handling new data columns in source

When source schemas as modified by the provider of that source, those fields are made available for selection to the user. In order to incorporate those fields into the data warehouse tables, users will have to use the Edit Integration functionality. Once the integration is edited, the user can select the newly available fields in the source.

When new columns are added to an existing integration, users are given a couple of choices

  • Add columns without reloading data

    • When users select this option, newly selected columns are added to the table in the data warehouse and any replication job after this change will extract data for the newly added column(s) in addition to the existing columns.

  • Add columns and reload data

    • When users select this option, newly selected columns are added to the table in the data warehouse and a full reload of the entire table is performed. When user selects this option, all of the rows in the table will be re-processed and therefore will be counted towards the billing quota.

Handling deprecated data columns in source

When a source provider deprecates a column from their APIs, the data for those columns can no longer be populated in the data warehouse. The column, however, will continue to exist in the data warehouse. Deprecated columns are automatically handled by Daton and no intervention is needed from the user.

Schema Changed by User

Use can use the Edit Integration functionality to modify the schema in the destination. There are a few scenarios that are possible.

Adding New Columns

Users can modify an integration and add additional columns to table. This may be required for various reasons and Daton effortless handles these type of scenarios. When new columns are added to an existing integration, users are given a couple of choices

  • Add columns without reloading data

    • When users select this option, newly selected columns are added to the table in the data warehouse and any replication job after this change will extract data for the newly added column(s) in addition to the existing columns.

  • Add columns and reload data

    • When users select this option, newly selected columns are added to the table in the data warehouse and a full reload of the entire table is performed. When user selects this option, all of the rows in the table will be re-processed and therefore will be counted towards the billing quota.

Removing Selected Column(s)

Users can modify an integration and de-select previously selected column(s). This may be required for various reasons and Daton effortlessly handles these type of scenarios. When column(s) are de-selected, the column will continue to exist in the data warehouse but no new data will be populated in the table. This action has no impact on billing.