Marketo connector

• Startup
• Business
• Enterprise
• On-Premise
• Add-on

Overview

Etlworks Marketo connector allows fast, real-time access to Marketo data. The connector supports most of the objects available through the Marketo API, and it works just like any other database connector.

Etlworks partnered with CData to provide access to the Marketo API using industry-standard JDBC protocol. Read about CData Marketo JDBC connector. 

When to use Marketo connector

Use this connector to create Flows that work with data in Marketo. 

Prerequisites

1. Enable Marketo connector for your Etlworks Integrator account. Contact support@etlworks.com  to enable connector.
2. Obtain the OAuthClientId and OAuthClientSecret values. To do this, navigate to the LaunchPoint option on the Admin area. Click the View Details link for the desired service, and a window containing the authentication credentials will be displayed.
3. Obtain the REST Endpoint URL, which can be found on your Marketo Admin area, specifically on the Integration > Web Services option in the REST API section. The Identity Endpoint will not be needed.

Create a Connection

Follow the steps below in creating a Marketo Connection:

Step 1. In the Connections window, click +, and type in marketo.

Step 2. Enter the following Connection parameters:

• URL: the JDBC Connection string. Typically, you only need to replace <instance> with the Marketo instance name, for example, 177-FXY-123.
• Client ID: the OAuth2 client ID.
• Client Secret: the OAuth2 secret. Read about Marketo API client ID and client secret.
• Other Parameters: optional parameters as key=value pairs. 

Use Other parameters to specify the Connection string options. Read about available Connection string options.

Marketo data model

Read about the Marketo REST API data model, specifically, the filters required to retrieve data from some objects. 

Stored procedures

Stored Procedures are available to complement the data available from the REST Data Model. Sometimes, it is necessary to update the data available from a view using a stored procedure because it does not provide direct, table-like, two-way updates. In these situations, the data retrieval is done using the appropriate view or table, while the update is done by calling a Stored Procedure. Stored Procedures take a list of parameters and return a dataset that contains the collection of tuples that constitute the response.

Read about available Stored Procedures. 

To call stored procure from the SQL Flow or the Before/After SQL, use EXEC sp_name params=value syntax.

For example:

EXEC SelectEntries ObjectName = 'Account' 

Work with data in Marketo

Extract data from Marketo

Extracting data from Marketo is similar to extracting data from the relational database.

Here's a detailed instruction on how to extract data from Marketo:

Step 1. Create Marketo Connection that will be used as a source (FROM). 

Step 2. Create a destination Connection, for example, a Connection to the relational database, and if needed, a Format (Format is not needed if the destination is a database or well-known API).

Step 3. Create a Flow where the source is a database, and the destination is a Connection created in step 2, for example, a relational database. 

Step 4. Add new source-to-destination transformation.

Step 5. Select the Marketo Connection created in step 1 as a source Connection, and select the Marketo object you are extracting data from.

Step 6. Select TO Connection, Format (if needed), and object (for example database table) to load data into.

Step 7. Click MAPPING and optionally enter Source query (you don't need a query if you are extracting data from the Marketo object unconditionally).

Step 8. Optionally, define the per-field Mapping.

Step 9. Add more transformations, if needed.

Load data in Marketo

Loading data in Marketo is similar to loading data into a relational database.

Here's a step by step instruction on how to load data in Marketo:

Step 1. Create a source Connection and a Format (if needed).

Step 2. Create destination Marketo Connection.

Step 3. Create a Flow where the destination is a database.

Step 4. Add new source-to-destination transformation.

Step 5. Select FROM and TO Connections and objects (also a FROM Format if needed).

Step 6. Optionally, define the per-field Mapping.

Step 7. Add more transformations, if needed.

Manage Marketo API quote

By default, Marketo allows 50K API calls per day, with a rate per instance limited to 100 calls per 20 seconds (https://developers.marketo.com/rest-api/). Contact Marketo support to request a permanent or temporary increase.

Here are some tips on how to stay within the permitted limits:

• Avoid running select * from on any Marketo table. To return the count, the Marketo connector will have to traverse all the records in the table, which can easily cause exceeding the API quote (50K/day). 
• Avoid running transformations in parallel. Running transformations in parallel can cause exceeding the permitted rate of the API calls (100 calls per 20 seconds). 

Browse data in Marketo

You must have a Marketo Connection to browse objects and run SQL queries.

Use the Etlworks Explorer to browse data and metadata in Marketo, as well as execute DML and SELECT queries against Marketo Connection. 

• Startup
• Business
• Enterprise
• On-Premise
• Add-on

Overview

Etlworks Marketo connector allows fast, real-time access to Marketo data. The connector supports most of the objects available through the Marketo API, and it works just like any other database connector.

Etlworks partnered with CData to provide access to the Marketo API using industry-standard JDBC protocol. Read about CData Marketo JDBC connector. 

When to use Marketo connector

Use this connector to create Flows that work with data in Marketo. 

Prerequisites

1. Enable Marketo connector for your Etlworks Integrator account. Contact support@etlworks.com  to enable connector.
2. Obtain the OAuthClientId and OAuthClientSecret values. To do this, navigate to the LaunchPoint option on the Admin area. Click the View Details link for the desired service, and a window containing the authentication credentials will be displayed.
3. Obtain the REST Endpoint URL, which can be found on your Marketo Admin area, specifically on the Integration > Web Services option in the REST API section. The Identity Endpoint will not be needed.

Create a Connection

Follow the steps below in creating a Marketo Connection:

Step 1. In the Connections window, click +, and type in marketo.

Step 2. Enter the following Connection parameters:

• URL: the JDBC Connection string. Typically, you only need to replace <instance> with the Marketo instance name, for example, 177-FXY-123.
• Client ID: the OAuth2 client ID.
• Client Secret: the OAuth2 secret. Read about Marketo API client ID and client secret.
• Other Parameters: optional parameters as key=value pairs. 

Use Other parameters to specify the Connection string options. Read about available Connection string options.

Marketo data model

Read about the Marketo REST API data model, specifically, the filters required to retrieve data from some objects. 

Stored procedures

Stored Procedures are available to complement the data available from the REST Data Model. Sometimes, it is necessary to update the data available from a view using a stored procedure because it does not provide direct, table-like, two-way updates. In these situations, the data retrieval is done using the appropriate view or table, while the update is done by calling a Stored Procedure. Stored Procedures take a list of parameters and return a dataset that contains the collection of tuples that constitute the response.

Read about available Stored Procedures. 

To call stored procure from the SQL Flow or the Before/After SQL, use EXEC sp_name params=value syntax.

For example:

EXEC SelectEntries ObjectName = 'Account' 

Work with data in Marketo

Extract data from Marketo

Extracting data from Marketo is similar to extracting data from the relational database.

Here's a detailed instruction on how to extract data from Marketo:

Step 1. Create Marketo Connection that will be used as a source (FROM). 

Step 2. Create a destination Connection, for example, a Connection to the relational database, and if needed, a Format (Format is not needed if the destination is a database or well-known API).

Step 3. Create a Flow where the source is a database, and the destination is a Connection created in step 2, for example, a relational database. 

Step 4. Add new source-to-destination transformation.

Step 5. Select the Marketo Connection created in step 1 as a source Connection, and select the Marketo object you are extracting data from.

Step 6. Select TO Connection, Format (if needed), and object (for example database table) to load data into.

Step 7. Click MAPPING and optionally enter Source query (you don't need a query if you are extracting data from the Marketo object unconditionally).

Step 8. Optionally, define the per-field Mapping.

Step 9. Add more transformations, if needed.

Load data in Marketo

Loading data in Marketo is similar to loading data into a relational database.

Here's a step by step instruction on how to load data in Marketo:

Step 1. Create a source Connection and a Format (if needed).

Step 2. Create destination Marketo Connection.

Step 3. Create a Flow where the destination is a database.

Step 4. Add new source-to-destination transformation.

Step 5. Select FROM and TO Connections and objects (also a FROM Format if needed).

Step 6. Optionally, define the per-field Mapping.

Step 7. Add more transformations if needed. 

Using Bulk API

The Bulk API is in an interface that allows retrieving and importing large data sets using delimited (CSV, TSV, or SSV) files.

To enable bulk API open the Marketo connection and add the following property to Other Parameters: UseBulkAPI = true.

When UseBulAPI is enabled, the Marketo Bulk API will be used to extract or load data, where applicable.

Currently, the only tables that support the Bulk API are Leads (extract and load) and Activities (extract). For any tables that do not support the Bulk API, this property will be ignored.

The Bulk API causes all the data to be retrieved in a single request and requires the data to be accumulated on the server side prior to sending. Therefore requesting a large amount of data using the Bulk API may be advantageous over using the REST API, and you may see performance improvements. Additionally, the Bulk API requires fewer API requests to be made (which helps preserve your API calls and stay within the API restrictions enforced by Marketo).

How it works

To use the Bulk API to extract records, a job must be created and enqueued. Once enqueued, Marketo will begin processing the job to retrieve the requested data and generate the delimited file. The status of the job can be polled to determine the current status and whether the file is available to be downloaded. Once the status shows that the job is complete and the file is ready, the data can then be downloaded.

When UseBulkAPI is set to true and JobPollingInterval is set to a value greater than 0, the connector will perform all the previously mentioned steps for you when executing a SELECT query on a Leads or Activities table. This will create and enqueue a job with the specified columns and filters. Note that a filter is required when exporting bulk data. For the Activities tables, an ActivityDate range must be specified. For the Leads table, a CreatedAt or UpdatedAt range may be specified, or a Static or Smart list. The driver will poll the job status to identify when the job has been completed, waiting JobPollingInterval seconds in between calls. Once the job is complete, the driver will download the delimited file that was created, parse it, and return the results for the specified query.

Note that job status calls count against your API call limit, and thus it is suggested to space out your status requests based on the amount of data you are requesting. The job status polling interval is configurable via JobPollingInterval. Marketo will only update the status every 60 seconds, and thus it is suggested that your polling interval be larger than 60 seconds. When expecting large datasets, it may be best to increase the polling interval to a value greater than 5 minutes to minimize API calls. It may take a while for the job to be processed, and thus it may seem like the query is exhibiting a hanging behavior when it is actually just waiting for the job to complete.

In the case that you want to issue your own job status polling requests, you can set JobPollingInterval to 0. This will just create and enqueue the job for you when you execute a SELECT query on a Leads or Activities table, returning the JobId in the result set.

Once a job has been enqueued, the status of the job can be polled by calling the GetExportJobStatus stored procedure.

The JobStatus value will be 'Complete' signaling that the job has finished processing and is ready to be downloaded. To finish executing your initial SELECT query, add the JobId filter to the WHERE clause of the initial SELECT statement. This query will download the file for the specified JobId and parse the result set.

Logic/Code Example (JobPollingInterval = 0):

SELECT JobId, Company, FirstName AS fn, LastName AS ln FROM Leads 
WHERE CreatedAt>='10/01/2017' AND CreatedAt<'10/31/2017'
# Retrieve the JobId value from the ResultSet (e.g. c4ebf745-b0e3-4bb8-bfc9-bd8472a28d35). 
# Only one row is returned and JobId will be the only relevant value returned.

loop(desired time interval) {
EXEC GetExportJobStatus @JobId='c4ebf745-b0e3-4bb8-bfc9-bd8472a28d35', @Type='Leads'
if (JobStatus == 'Completed') break;
}

SELECT Company, FirstName AS fn, LastName AS ln FROM Leads 
WHERE CreatedAt>='10/01/2016' AND 
CreatedAt<'10/31/2016' AND JobId='c4ebf745-b0e3-4bb8-bfc9-bd8472a28d35'

Manage Marketo API quote

By default, Marketo allows 50K API calls per day, with a rate per instance limited to 100 calls per 20 seconds (https://developers.marketo.com/rest-api/). Contact Marketo support to request a permanent or temporary increase.

Here are some tips on how to stay within the permitted limits:

• Avoid running select * from on any Marketo table. To return the count, the Marketo connector will have to traverse all the records in the table, which can easily cause exceeding the API quote (50K/day). 
• Avoid running transformations in parallel. Running transformations in parallel can cause exceeding the permitted rate of the API calls (100 calls per 20 seconds). 

Browse data in Marketo

You must have a Marketo Connection to browse objects and run SQL queries.

Use the Etlworks Explorer to browse data and metadata in Marketo, as well as execute DML and SELECT queries against Marketo Connection.