Articles in this section

Exploring Connections and Metadata

  • Updated
Follow

Connections in Etlworks Explorer

Explorer displays all connections available to the current user. A connection represents a data source such as a database, file system, cloud storage, API, web service, Redis, MongoDB, or inbound email inbox.

Connections are shown in the left panel. Expanding a connection reveals metadata objects associated with that connection.

Examples:

  • Databases expose schemas, tables, views, and columns

  • File and cloud storage connections expose folders and files

  • APIs and web services expose endpoints and response fields

  • Redis exposes keys and values

  • MongoDB exposes collections and document fields

  • Inbound email connections expose messages and attachments

Connections That Require Formats

Some connections can be explored immediately. Others require a format to interpret the data.

Connections that do not require formats

These connections expose metadata and data directly:

  • Relational databases

  • Some APIs with fixed schemas

Connections that require formats

These connections require a format to parse and structure the data:

  • File and cloud storage connections

  • Web services returning JSON, XML, HL7, CSV, or other formats

  • Redis

  • MongoDB

When you expand a connection that requires a format and none is assigned, Explorer prompts you to select one.

Once a format is selected:

  • The format is associated with the connection in Explorer

  • Metadata and data become available

  • SQL can be executed against the parsed dataset

Only one format can be associated with a given connection in Explorer. If the same physical connection is used for multiple data structures, create separate connections and assign different formats.

Assign Format

To assign a format to the connection:

1. Right click on a connection

2. Click Select Format

3. Choose a format

The newly selected format is applied immediately and affects metadata discovery, data views, and SQL execution.

Explorer Parameters

Explorer supports configurable parameters that are used to replace {tokens} in connection definitions.

What Explorer parameters are

Explorer parameters are key-value pairs associated with a connection while working in Explorer. When a connection contains tokenized values, such as {tokenName}, Explorer replaces those tokens with parameter values of the same name.

This applies when:

  • Exploring metadata

  • Viewing data

  • Viewing raw data

  • Running SQL queries

Why parameters exist

Previously, connections containing tokens could not be explored because the actual values were unknown. Explorer parameters solve this by allowing you to supply concrete values at exploration time without modifying the connection itself.

This is especially useful for:

  • Tokenized URLs

  • Dynamic file paths

  • API query parameters

  • Environment-specific values

How token replacement works

When Explorer executes an operation:

  • It looks for {tokenName} in the connection definition

  • If a parameter named tokenName exists, its value is substituted

  • Replacement happens at runtime and only within Explorer

The underlying connection configuration remains unchanged.

Example

If a connection URL contains {token}:

https://api.example.com/data/{customerId}

And Explorer parameters include parameter with the same name as {token}:

customerId = 12345

Explorer resolves the URL as:

https://api.example.com/data/12345

The resolved value is used for metadata discovery, data retrieval, and SQL execution.

How to set Parameters

1. Right click on a connection

2. Click Set Parameters

3. Set regular and/or secure parameters as key-value pairs.

3. Optionally enable Remember so the parameters associated with the connection are persisted and used each time you work with this connection in Explorer.

Exploring Metadata

To explore metadata:

  • Expand a connection in the left panel

  • Drill down through available objects

Database metadata

For relational databases, Explorer displays:

  • Databases

  • Schemas

  • Tables and views

  • Columns and data types

SaaS apps metadata

For SaaS apps, Explorer displays:

  • Schemas

  • Objects associated with API endpoints (as Tables and Views)

  • Columns and data types

File and cloud storage metadata

Explorer displays:

  • Files (name, size, last modified timestamp)

  • Parsed fields when a format is assigned

API metadata

Explorer displays:

  • Endpoints

  • Parsed response fields

Redis metadata

Explorer displays:

  • Keys

  • Parsed values when a format is assigned

NoSQL metadata

Explorer displays:

  • Collections

  • Document fields inferred from sample data

Message Queues metadata

Explorer displays:

  • Queues and Topics

  • Fields inferred from sample data

Inbound email metadata

Explorer displays:

  • Messages

  • Attachments and parsed fields

Metadata is discovered dynamically and reflects the current state of the source.

Filtering Connections and Metadata

Filtering connections by name

Use the filter box at the top of the connections panel to filter connections by name. Partial matches are supported.

Filtering connections by tags

Click the Tags field in the filter box to filter connections by one or more tags. 

Sticky filters

When sticky filters are enabled, Explorer remembers the applied name and tag filters and restores them automatically.

Filtering metadata objects

Some metadata groups include a filter icon. Use it to filter objects such as tables, files, or columns by name.

Filtering helps when working with large schemas, deep folder structures, or extensive API definitions.

Summary

In Etlworks Explorer, you can:

  • Browse all available connections

  • Assign and change formats as needed

  • Use Explorer parameters to resolve tokenized connections

  • Explore metadata for databases, files, APIs, Redis, MongoDB, emails, etc.

  • Filter connections and metadata for faster navigation

Related articles