Amazon Redshift ETL connector for data replication
Instructions for granting Amazon Redshift access and setting up Amazon Redshift as a destination in Daton.
Amazon Redshift Setup Information
Setting up access to your Amazon Redshift data warehouse is not very complicated and just takes a couple of simple steps. If you don't have an existing account on the AWS, you can sign up a free tier here.
Step 1 - Grant Cluster access
Instructions to grant Redshift cluster access and (optionally) creating a user and schema for setting up Redshift as a destination in Daton.
Click Amazon Redshift & enter any integration name
Enter the JDBC URL, database username, and password from above steps
Click Next, if successfully connected, select the schema from the options
Click Finish to complete setup
Daton has been built to handle multiple data warehouses. Since all data warehouses are not built the same, the choice of a data warehouse may impact how Daton loads data. This section tables about some key capabilities and limitations of RedShift that impact how data looks in your data warehouse.
Redshift doesnt allow updating while insert mechanism. Instead, Daton will append data to the end of the table with every replication run.
Full table replication
Adding columns to existing integration
When new columns are added to an existing table in Amazon Redshift, user is given two options. Depending on their choice, the new column(s) is added and entire table is re-loaded or there is no re-load. Read the edit integration (proceed link) functionality to understand the impact of making changes.
Removing columns to existing integration
Users can use the edit integration functionality
Adding tables to existing integration
When tables are deselected from an integration, the loads into those tables stop. Daton doesn't delete the tables from the data warehouse. This is great for cases where usage of a source has been discontinued, but there is a desire to capture that data.
Querying Daton created tables on RedShift
RedShift data warehouse does not support nested tables. Hence, Daton creates separate 'child' tables for each nested data present in source schema(or parent table).
At present Daton creates separate child tables up to three levels.
For example, if data is present in Listrecommendations.Fulfillmentrecommendations.Member.Itemdimensions.Dimensiondescriptionthen the following tables would be created -