Articles in this section

See more

X12, EDIFACT, NCPDP and VDA Formats

  • Updated
Follow
  • Startup
  • Business
  • Enterprise
  • On-Premise
  • Add-on

Overview

Etlworks provides comprehensive support for most EDI (Electronic Data Interchange) dialects, including X12, EDIFACT, NCPDP, and VDA. Additionally, Etlworks supports healthcare-specific standards like HL7 2.x and HL7 FHIR, widely used by healthcare providers and vendors.

When to use these Formats

Use these formats to read and create EDI messages in X12, EDIFACT, NCPDP, and VDA EDI dialects and generate acknowledgments for messages in X12 and EDIFACT EDI dialects. 

Read how to work with EDI messages in Etlworks.

Differences Between EDI Connectors

Etlworks offers two connectors for EDI processing: EDI Format and X12 Format. Each is designed for different use cases, depending on the file size, EDI dialect, and required transformations.

EDI Format Connector

The EDI Format connector is ideal for small to medium-sized EDI files (up to 100 KB). This connector is optimized for workflows that involve manageable file sizes and simpler transformations, especially with the drag-and-drop UI to streamline data mapping. For larger files or batch processing, consider using the X12 Format connector for better performance.

Supported EDI Dialects: The EDI Format connector supports X12, EDIFACT, NCPDP, and VDA. It also accommodates HL7 versions, though for HL7 data, it’s recommended to use the dedicated HL7 2.x and HL7 FHIR connectors.  

Usage and Access: The EDI Format connector is free to. It requires access to the Etlworks EDI API (Host: https://etlworksedi.azurewebsites.net/api).

Supported Operations:

  • Read and write messages in X12, EDIFACT, NCPDP, and VDA formats.
  • Generate acknowledgments for messages in X12 and EDIFACT formats.
  • Transform EDI messages in any dialect into JSON format, leveraging the open-source OpenEDI format (based on OpenAPI 3). This JSON representation simplifies creating EDI documents in a generic format, making it easier than using the X12 Format, which relies on an XML specification.

X12 Format Connector

The X12 Format connector is designed for batch processing of large X12 documents (up to 100 MB). It is optimized for high-volume data flows requiring performance.

Usage and Access: The X12 Format connector is free and available in all Etlworks versions. For downloading schemas it requires access to the Etlworks CDN (host: https://etlworks-edi-schemas.s3.amazonaws.com)

IMPORTANT: X12 connector is free but it requires installing a license. If you are using self-hosted Etlworks please contact support@etlworks.com for assistance with installing a license. 

Supported Operations:

Schema Requirements:

A schema is required for each version of the X12 dialect. If the schema file does not already exist in the cache under {app.data}/metadata/edi, the connector automatically downloads it from the CDN and stores it in the cache. While users can override the default schema location and install schemas manually (see documentation), this is optional and typically only necessary if the Etlworks instance cannot perform outbound HTTPS requests to https://etlworks-edi-schemas.s3.amazonaws.com.

Configuration Tip: For optimal performance, set the Transaction Set (e.g., 204) when configuring the X12 format.

Generic EDI format

The EDI Format connector is ideal for small to medium-sized EDI files (up to 100 KB). This connector is optimized for workflows that involve manageable file sizes and simpler transformations, especially with the drag-and-drop UI to streamline data mapping. For larger files or batch processing, consider using the X12 Format connector for better performance.

EDI format supports reading and writing EDI messages in X12, EDIFACT, NCPDP, and VDA EDI dialects and creating acknowledgments for messages in X12 and EDIFACT EDI dialects. 

Create a generic EDI format

To create a new generic EDI Format, go to Connections, select the Formats tab, click Add Format, select the EDI tab, and select EDI. You can also just type in edi in the Search field.

Configure generic EDI format

Below are the available parameters for the generic EDI Format.

EDI Dialect

  • EDI dialect: The EDI dialect. Supported dialects are X12, EDIFACT, NCPDP, and VDA.

Read EDI messages

  • Continue on error: whether to continue reading if a corrupt interchange is encountered. The default is true.
  • Ignore null values: whether to ignore all null values. The default is false.

Write EDI messages

  • Preserve blank data elements: Whether to preserve blank data elements so the output contains multiple delimiters instead of omitting any excess delimiters. The default is false.
  • Postix: The postfix is to be applied at the end of each segment, just after the segment separator. This is usually a carriage return (CR), line feed (LF), or both. By default, there is no postfix.
  • Same repetion and data element: EDIFACT only. Sometimes the same delimiter is used to denote the data element separator and repetition separator as in IATA transactions. By default, this is false.
  • S3 is the default syntax for EANCOM transactions: EDIFACT only. The default syntax for EANCOM transactions. By default, all EANCOM transactions will be constructed according to the rules of Syntax 4. Set this flag to true if you need Syntax 3 to be used.
  • Add G1 to header: NCPDP only. Whether to add the G1 to the header or not. The default is false, e.g. it's always produced with a G1.
  • Trailer message: NCPDP only. The optional Message_03 in the transmission trailer. By default, there is no message.

Acknowledgment for X12 and EDIFACT messages

  • Acknowledgement File Name: Acknowledgement for X12 and EDIFACT messages.
  • Detect duplicates: X12 and EDIFACT. If you need to detect duplicates as in functional groups or transactions with the same reference number, set this flag to true. The default is false.
  • Transaction Ref Number: X12 and EDIFACT. The default is 1. Set this to whatever the 997 or 999 X12Interchange.ST.TransactionSetControlNumber_02 needs to be. In case there are multiple acknowledgments (for multiple functional groups), this will be starting reference number and every subsequent acknowledgment will have the previous reference number incremented with 1.
  • Interchange Ref Number: X12 and EDIFACT. The default is 1. Set this to whatever the 997 or 999 X12Interchange.ISA.InterchangeControlNumber_13 needs to be Ack for valid transactions: X12 and EDIFACT. The default is false. Set this to true if you need AK2 loops included for all valid transactions as well. By default, AK2 loops are generated only for invalid transactions.
  • Batch Ack: X12 and EDIFACT. The default is true. Set this to false if you need to generate a separate X12Interchange for each acknowledgment. By default, all acknowledgments are batched in the same X12Interchange
  • Technical Ack: X12 and EDIFACT. The default technical acknowledgment TA1 is generated according to X12Interchange.ISA.AcknowledgementRequested_14. You can either enforce it to always generate TA1s or suppress it to never generate any TA1s. This will override the flag in X12Interchange.ISA.AcknowledgementRequested_14.
  • Ack: X12 only. The default value is 997. The type of acknowledgment being generated. Set this to 999 if you need to generate an implementation instead of functional acknowledgment.
  • The value of the AK9's first element is P: X12 only. The value of the AK9's first element. By default, it is E. Set this to true if you want this value to be P instead.

Common parameters

  • Preprocessor: JavaScript code that modifies the contents of the source document. Read more on how to use a Preprocessor.
  • Column names compatible with SQL: convert column names to SQL-compatible column names by removing all characters, except alphanumeric and spaces.
  • Date and Time Format: a Format for timestamps (date+time).
  • Date Format: a Format for date (date only, no time).
  • Time Format: a Format for time (time only, no date).
  • Parse Dates: if this option is enabled, and the date or time value is not recognized as one of the formats defined above, Etlworks will try to parse it using one of the well-known date+time Formats.
  • Trim Strings: if this option is enabled, Etlworks will trim leading and trailing white spaces from the value.
  • Treat 'null' as null: if this option is enabled, Etlworks will treat string values equal to 'null' as actual nulls (no value).
  • Encoding: the document encoding. The default is UTF-8.

X12 format

The X12 Format connector is designed for batch processing of large X12 documents (100 MB and up). It’s optimized for high-volume, complex data flows that require performance and reliable batch processing.

The X12 format is free and included in every version of Etlworks. It supports reading and creating X12 documents, transforming X12 documents to XML, JSON or CSV, and generating 997 acknowledgments.

Create X12 format

To create a new X12 Format, go to Connections, select the Formats tab, click Add Format, select the EDI tab, and select X12. You can also just type in x12 in the Search field.

Configure X12 format

Below are the available parameters for the X12 Format:

  • Transaction Set: the transaction set is used to provide a structure for transfer of EDI standards, or portions thereof, in an electronic form. IMPORTANT: The transaction set is required to create XML with human-readable fields, like FTX03_TextReference, so it is highly recommended to enter the correct transaction set of the file in this field. For example, for X12, it can be 204.
  • EDI schema Filename or Path: the fully qualified name of the schema file or a base folder with EDI schema files. A schema is required for each version of the X12 dialect. If the schema file does not already exist in the cache under {app.data}/metadata/edi, the connector automatically downloads it from the CDN and stores it in the cache. While users can override the default schema location and install schemas manually (see documentation), this is optional and typically only necessary if the Etlworks instance cannot perform outbound HTTPS requests to https://etlworks-edi-schemas.s3.amazonaws.com.
  • Output File Name for Converted EDI Document: Specifies the name of the output file where the EDI document will be saved in either JSON or XML format. If the file name includes a.json extension, the EDI document will be converted to JSON; if the file name has any other extension (or no extension), it will be saved as XML. This property applies whether the connector is being used in a transformation flow or simply to convert and store files. Read about converting X12 file to XML or JSON.
  • Skip Parsing EDI Document: When selected, this option skips detailed parsing of the EDI document and instead creates an output file directly from the EDI document in either JSON or XML format, based on the file extension specified in the output file name. Use this option if you only need the converted document stored without further processing. Read about converting X12 file to XML or JSON.
  • Strict schema validation: enables strict X12 schema validation.
  • Enable cross-field validation rules: enables cross-field validation rules. The EDI receive pipeline and EDI send pipeline can perform cross-field/segment validation on transaction-set data elements in X12-encoded messages. This validation is called relational conditions in X12.
  • Use long element names: determines whether to use long (human-readable descriptions) or short (codes only) element and segment names in your XML conversions (FTX03_TextReference or FTX03, for example). IMPORTANT: requires valid X12.
  • Include Empty Elements: determines whether to include empty data elements in a composite element.
  • Include Footers: determines whether to include footer information in the XML.
  • Component delimiter: the delimiter character to use to separate components. 
  • Element delimiter: the delimiter character to use to separate data elements.
  • Segment delimiter: the delimiter character to use to separate segments.
  • Release character: the character to use to escape delimiters within values.
  • Repetition character: the repetition character.
  • Template: this is a template in the X12 Format. If this field is not empty, Etlworks will use it to populate column names and data types. The template is an optional field.
  • Preprocessor: JavaScript code that modifies the contents of the source document. Read more on how to use a Preprocessor.
  • Default Extension: the default extension is used when the file name doesn't have an extension. If not entered, x12 is going to be the default extension.
  • Column names compatible with SQL: convert column names to SQL-compatible column names by removing all characters, except alphanumeric and spaces.
  • Date and Time Format: a Format for timestamps (date+time).
  • Date Format: a Format for date (date only, no time).
  • Time Format: a Format for time (time only, no date).
  • Parse Dates: if this option is enabled, and the date or time value is not recognized as one of the formats defined above, Etlworks will try to parse it using one of the well-known date+time Formats.
  • Trim Strings: if this option is enabled, Etlworks will trim leading and trailing white spaces from the value.
  • Treat 'null' as null: if this option is enabled, Etlworks will treat string values equal to 'null' as actual nulls (no value).
  • Encoding: XML document encoding. The default is UTF-8.

Manually Install schemas for different versions of X12 formats

A schema is required for each version of the X12 dialect. If the schema file does not already exist in the cache under {app.data}/metadata/edi, the connector automatically downloads it from the CDN and stores it in the cache. While users can override the default schema location and install schemas manually this is optional and typically only necessary if the Etlworks instance cannot perform outbound HTTPS requests to https://etlworks-edi-schemas.s3.amazonaws.com.

To download and install schemas for different versions of the formats, follow these steps:

Step 1. Request a download link for the selected X12 schema from here.

Step 2. Download the zip archive with the schema to your local drive.

Step 3. Create a Server storage connection that points to the Home folder (if the connection does not exist).

mceclip6.png

Step 5. Create a new Create folder flow.

mceclip0.png

Step 6. Add transformation where the source connection is a server storage connection created in step 3, and the FROM is edi/schema_name, for example edi/00501.

mceclip1.png

Step 7. Execute the Create folder flow. 

Step 8. Create a Server storage connection that points to the Home/edi/schema_name, for example /edi/00501/*.*:

mceclip2.png

Step 8. In Explorer, upload all files from unzipped schema file into this folder:

mceclip3.png

(example for Windows)

mceclip4.png

Step 9. Open EDI format, set Transaction set, set EDI schema to {app.data}/edi.

mceclip5.png

Related articles