Amazon Redshift is a fast, fully-managed data warehouse that makes it simple and cost-effective to analyze all your data using standard SQL and your existing Business Intelligence (BI) tools. Read more about Amazon Redshift. Etlworks Integrator fully supports Amazon Redshift.
What can you do with Amazon Redshift in Etlworks Integrator
|
ETL data into Amazon Redshift Using Redshift-optimized Flows, you can extract data from any of the supported sources, transform and load it into Redshift. |
Load multiple tables by a wildcard name You can ETL data from multiple database objects (tables and views) into Redshift by a wildcard name without creating individual source-to-destination transformations. |
|
Setup incremental replication using change data capture (CDC) Change Data Capture (CDC) pipeline allows extracting changes in real-time from the databases which support CDC and loading them into Amazon Redshift. |
Setup incremental replication using high watermark (HWM) Using HWM replication you can load only new and updated records into Amazon Redshift. |
Related resources
|
Directly load files in Redshift This Flow loads CSV files directly into Redshift. The files can be anywhere, so long as Etlworks Integrator can read them. |
Work with Amazon Redshift as a relational database You can use any |
ELT with Amazon Redshift The Etlworks Integrator supports executing complex ELT scripts directly in Redshift, which greatly improves the performance and reliability of the data ingestion.
|
|
Data type Mapping for Redshift It is important to understand how we map various JDBC data types for the Redshift data types. |
Configure Redshift Configure permissions and firewall. |
Connect to Redshift Create a connection for the Redshift cluster. |
Related case study
|
Professional social network Load data into the Amazon Redshift from multiple sources
|
"A typical CDC Flow can extract data from multiple tables in multiple databases, but having a single Flow pulling data from 55000+ tables would be a major bottleneck as it would be limited to a single blocking queue with a limited capacity. It would also create a single point of failure." |
Configure Redshift
Configure the Firewall
Typically, the TCP port 5439 is used to access Amazon Redshift. If Amazon Redshift and Eltworks Integrator are running on different networks, it is important to enable inbound traffic on port 5439.
Learn how to open an Amazon Redshift port for inbound traffic.
Configure permissions for Redshift
- The user used to access the Redshift must have
INSERTprivilege for the Amazon Redshift table. - If you are loading data into Redshift from an Amazon S3 bucket, you will need to grant access to S3 for the Redshift user.
- For access to Amazon S3 with the ability to use
COPYandUNLOAD, choose either theAmazonS3ReadOnlyAccessrole or theAmazonS3FullAccessrole.
Connect to Redshift
To work with Redshift, you will need to create a Connection to Amazon Redshift.
Comments
0 comments
Please sign in to leave a comment.