Create HL7 messages using scripting – Etlworks Support

Etlworks ships an embedded copy of the HAPI HL7v2 Java library (ca.uhn.hl7v2) and exposes it to JavaScript and Python flows. This article covers when scripting is the right tool, how to reach the HAPI object model from a flow, the common patterns used in practice, and two worked examples drawn from real clinical interfaces.

For the general HL7 reference in Etlworks — format settings, transports, and the visual mapping path — see Working with HL7.

When should I use scripting instead of mapping?

Most HL7 work in Etlworks fits the visual nested mapping editor — segments and fields appear in the tree, you drag source onto destination, and the engine handles the message construction. Reach for scripting when the transformation logic doesn't fit that model:

Deep, per-message conditional logic — e.g., the OBX value depends on which NTE comment kind appeared earlier in the source message, and you need to walk the message to figure that out.
Restructuring repeating segments — e.g., drop NTE segments after extracting their data into OBX segments, or coalesce multiple ORC/OBR groups into a different shape.
Cross-segment lookups within the same message — e.g., propagate a value from MSH into a generated PV1 segment that the source didn't have.
Cloning + selective edits — start with the source message, change a few fields (MSH-3 sending application, MSH-6 receiving facility), keep everything else.
Generating multiple output messages from one input — or one output from many inputs.

If your transformation is "rename these fields, drop these fields, fill these fields from elsewhere", visual mapping is faster to build and easier to maintain. Pick scripting when the logic itself is the hard part.

The HAPI HL7v2 library

HL7 message parsing and construction in Etlworks is implemented on top of HAPI HL7v2, the de-facto Java library for HL7 2.x. The library is bundled with Etlworks — you don't install it separately — and the package ca.uhn.hl7v2 is available to any JavaScript or Python flow.

Supported HL7 versions

HAPI ships separate model classes for each HL7 2.x version. Etlworks bundles all of them. Use the package that matches the version of the message you're working with:

HL7 version	Package prefix
2.1	ca.uhn.hl7v2.model.v21
2.2	ca.uhn.hl7v2.model.v22
2.3	ca.uhn.hl7v2.model.v23
2.3.1	ca.uhn.hl7v2.model.v231
2.4	ca.uhn.hl7v2.model.v24
2.5	ca.uhn.hl7v2.model.v25
2.5.1	ca.uhn.hl7v2.model.v251
2.6	ca.uhn.hl7v2.model.v26
2.7	ca.uhn.hl7v2.model.v27

Package layout

Inside each version's package the structure is the same:

Subpackage	What it contains
ca.uhn.hl7v2.model.vNN.message	Top-level message types — ORM_O01, ORU_R01, ADT_A01, ACK, and so on.
ca.uhn.hl7v2.model.vNN.segment	Segment classes — MSH, PID, PV1, OBR, OBX, NTE, ORC, IN1, etc.
ca.uhn.hl7v2.model.vNN.datatype	HL7 data types — ST (string), FT (formatted text), NM (numeric), TS (timestamp), XPN (extended person name), XAD (extended address), etc.
ca.uhn.hl7v2.model.vNN.group	Repeating group structures within messages, e.g., ORM_O01_ORDER.

Shared utilities used regardless of version:

ca.uhn.hl7v2.util.DeepCopy	Copies a segment from one message into another, field by field.
ca.uhn.hl7v2.parser.PipeParser	Parser / serializer for the standard pipe-delimited HL7 wire format.
ca.uhn.hl7v2.parser.XMLParser	XML parser / serializer for HL7.
ca.uhn.hl7v2.HL7Exception	Base exception type thrown by HAPI APIs.

For the full Javadoc, see the HAPI HL7v2 API documentation.

Accessing the HL7 object model from a flow

The bridge between Etlworks and HAPI is two methods on com.toolsverse.etl.common.DataSet:

Method	What it does
dataSet.getActualData()	Returns the parsed HL7 message as a HAPI object (e.g., an ORM_O01 instance for a 2.3 lab order). Walk it from JavaScript or Python using the same getter / setter API HAPI provides in Java.
dataSet.setActualData(message)	Replaces the underlying HAPI object with one you've built or modified. The serializer emits it back to the wire format on output.

The typical scripting flow:

Read the source HL7 message into the flow. Etlworks parses it through HAPI; the result is reachable as dataSet.getActualData().
In a JavaScript / Python step, get the HAPI object, manipulate it, optionally build a new one.
Call dataSet.setActualData(newMessage) if you've built a new top-level message; or just modify the existing object in place.
The downstream flow writes the result through the HL7 destination format — serialization is automatic.

Importing HAPI packages from JavaScript

Use JavaImporter to make HAPI's Java packages available without the full path on every reference:

var javaImports = new JavaImporter(
    Packages.ca.uhn.hl7v2.model.v23.message,
    Packages.ca.uhn.hl7v2.model.v23.segment,
    Packages.ca.uhn.hl7v2.model.v23.datatype,
    Packages.ca.uhn.hl7v2.util,
    java.io);

with (javaImports) {
    var message = dataSet.getActualData();   // HAPI ORM_O01 (or whatever the source is)

    // … your transformation code …

    // For a brand-new top-level message:
    // var destMessage = new ORU_R01();
    // dataSet.setActualData(destMessage);
}

For other HL7 versions, change v23 to v25, v251, v26, etc.

From a Python flow the equivalent is direct import of the Java packages:

from ca.uhn.hl7v2.model.v23.message import ORM_O01, ORU_R01
from ca.uhn.hl7v2.util import DeepCopy

message = dataSet.getActualData()
# … transform …

Common scripting patterns

Get a field value

HAPI exposes getters at every level — message → segment → field → component → subcomponent. Long names with the segment-field number are the most readable choice:

var sendingApp = message.getMSH().getSendingApplication().getNamespaceID().getValue();
var patientId  = message.getPID().getPatientIDInternalID(0).getID().getValue();
var orderingMd = message.getORDER(0).getORDER_DETAIL().getOBR()
                       .getOrderingProvider(0).getFamilyName().getValue();

Set a field value

message.getMSH().getSendingApplication().getNamespaceID().setValue("Integrator");
message.getMSH().getMessageControlID().setValue(newControlId);

Iterate repeating segments

HL7 segments and groups can repeat. HAPI exposes both an indexed accessor and a count:

var numOrders = message.getORDERReps();

for (var i = 0; i < numOrders; i++) {
    var order = message.getORDER(i);
    var obr   = order.getORDER_DETAIL().getOBR();
    // … do something with each ORDER group …
}

For "all of them as a list" use the getXxxAll() variants:

var allNte = order.getORDER_DETAIL().getNTEAll();
for (var i = 0; i < allNte.length; i++) {
    var nte = allNte.get(i);
    var commentSource = nte.getNte2_SourceOfComment().getValue();
    // …
}

Insert / remove segments

// Insert a new OBSERVATION group at index 0
var obx = order.getORDER_DETAIL().insertOBSERVATION(0).getOBX();
obx.getObx1_SetIDOBX().setValue(1);
obx.getObx2_ValueType().setValue("FT");

// Remove NTE segments after extracting what you need from them
var detail = order.getORDER_DETAIL();
while (detail.getNTEReps() > 0) {
    detail.removeNTE(0);
}

Clone a message

Start from the source and edit a few fields:

var sourceMessage = dataSet.getActualData();
var destMessage   = ClassUtils.clone(sourceMessage);

destMessage.getMSH().getSendingApplication().getNamespaceID().setValue("Integrator");

dataSet.setActualData(destMessage);

Copy a segment between messages

Use DeepCopy to move a segment from one message to another field-for-field:

var sourceMessage = dataSet.getActualData();
var destMessage   = new ORU_R01();

DeepCopy.copy(sourceMessage.getMSH(), destMessage.getMSH());

destMessage.getMSH().getSendingApplication().getNamespaceID().setValue("Integrator");

Build a destination message from scratch

var destMessage = new ORU_R01();

destMessage.getMSH().getFieldSeparator().setValue("|");
destMessage.getMSH().getEncodingCharacters().setValue("^~\\&");
destMessage.getMSH().getSendingApplication().getNamespaceID().setValue("Integrator");
destMessage.getMSH().getMessageType().getMessageType().setValue("ORU");
destMessage.getMSH().getMessageType().getTriggerEvent().setValue("R01");
// …

dataSet.setActualData(destMessage);

Example: lab order processing

A hospital sends an HL7 2.3 ORM_O01 order via SFTP to a vendor folder. Etlworks reformats the message to the AP Easy specification and writes the result to the lab's destination folder.

What the transformation does

Change MSH-3 (Sending Application) from the source application to Integrator.
For each ORDER group in the message, derive OBX observations from the NTE comments and insert them into the ORDER_DETAIL.
Use the NTE comment kind to decide whether each OBX represents "Site" or "Procedure".
Map NTE-3 comment text to a controlled vocabulary (SHAVE, PUNCH, EXCISION, …).
Remove the NTE and DG1 segments after the data has been extracted.

Flow shape

Stage	What happens
Hospital → source-orders folder	SFTP drop. Etlworks polls.
source-orders → processing	Move-files flow.
processing → JavaScript reformat → destination-orders	HL7 to HL7 flow with a JavaScript transformation step using HAPI.
On error	Move to the failed folder for inspection.

The transformation script

The key piece is the JavaScript step in the HL7-to-HL7 flow. It clones the source, edits MSH, and rewrites each ORDER's observations:

var javaImports = new JavaImporter(
    Packages.ca.uhn.hl7v2.model.v23.message,
    Packages.ca.uhn.hl7v2.model.v23.segment,
    Packages.ca.uhn.hl7v2.model.v23.datatype);

with (javaImports) {
    var message     = dataSet.getActualData();
    var destMessage = ClassUtils.clone(message);

    // change sender
    destMessage.getMSH().getSendingApplication().getNamespaceID().setValue("Integrator");

    var numOrders = destMessage.getORDERReps();

    for (var num = 0; num < numOrders; num++) {
        var order  = destMessage.getORDER(num);
        var detail = order.getORDER_DETAIL();
        var allNte = detail.getNTEAll();

        // build first OBX from OBR-13
        var obx = detail.insertOBSERVATION(0).getOBX();
        obx.getObx1_SetIDOBX().setValue(1);
        obx.getObx2_ValueType().setValue("FT");
        obx.getObx3_ObservationIdentifier().getIdentifier().setValue("03");
        obx.getObx3_ObservationIdentifier().getText().setValue("Clinical");

        var ft = new FT(destMessage);
        ft.setValue(detail.getOBR().getObr13_RelevantClinicalInformation().getValue());
        obx.insertObservationValue(0).setData(ft);

        // turn selected NTEs into Site / Procedure observations
        var index = 1;
        for (var i = 0; i < allNte.length; i++) {
            if (i == 0 || i == 2 || i == 4 || i == 5) continue;   // skip NTEs that don't map to OBX

            var nte = allNte.get(i);
            var obx = detail.insertOBSERVATION(index).getOBX();
            obx.getObx1_SetIDOBX().setValue(index);
            obx.getObx2_ValueType().setValue("FT");
            obx.getObx3_ObservationIdentifier().getIdentifier().setValue("0" + index);

            var qualifier = nte.getNte2_SourceOfComment().getValue().toLowerCase();
            obx.getObx3_ObservationIdentifier().getText().setValue(
                qualifier.contains("specimen") ? "Site" : "Procedure");

            // map NTE-3 free text to a controlled vocabulary
            var raw = nte.getNte3_Comment(0).getValue().toLowerCase();
            var coded = raw.contains("punch")  ? "PUNCH"
                      : raw.contains("shave")  ? "SHAVE"
                      : raw.contains("excision") ? "EXCISION"
                      : nte.getNte3_Comment(0).getValue();

            var ft = new FT(destMessage);
            ft.setValue(coded);
            obx.insertObservationValue(0).setData(ft);

            index++;
        }

        // remove the NTE segments (and DG1 segments, omitted here for brevity)
        while (detail.getNTEReps() > 0) {
            detail.removeNTE(0);
        }
    }

    dataSet.setActualData(destMessage);
}

Flow setup

Create SFTP connections for source-orders, processing, failed, destination-orders. The processing folder is typically server storage on the Etlworks host.
Create an HL7 2.3 format.
Build the pipeline as a nested flow:
- Step 1: Move from source-orders to processing.
- Step 2: HL7-to-HL7 flow that reads from processing, runs the script above as the transformation step, writes to destination-orders.
- Step 3: Move source files out of processing on success.
- On error step: Move to failed.
Schedule the nested flow continuously or on a short interval (every 30 s is typical for clinical drops).

Example: lab results processing

The mirror of the order flow. The lab sends an HL7 2.3 ORU_R01 with optional PDF attachment via SFTP to the client folder. Etlworks builds a reformatted ORU_R01 and writes it back toward the hospital's destination folder.

What the transformation does

Start from a blank ORU_R01 in the destination.
Copy MSH from the source via DeepCopy.
Change MSH-3 to Integrator.
Iterate over each RESPONSE / ORDER_OBSERVATION in the source, transform the result content, and place it in the destination.
Output is a different ORU_R01 structure tuned to the receiving system's expectations.

Flow shape

Stage	What happens
Lab → source-results folder	SFTP drop. May include an HL7 ORU plus a PDF attachment.
source-results → processing	Files copied to a remote processing folder and to a local processing folder on the Etlworks host.
processing → JavaScript reformat → destination-results	HL7-to-HL7 flow with a JavaScript transformation step that builds a fresh ORU_R01 using DeepCopy.
PDF passthrough	Copy the associated PDF to the same destination folder.
On error	Move to failed.

The transformation script

The key technique is build-from-scratch + DeepCopy instead of clone: when the output structure isn't the same as the input, building fresh and copying selectively is cleaner.

var javaImports = new JavaImporter(
    Packages.ca.uhn.hl7v2.model.v23.message,
    Packages.ca.uhn.hl7v2.model.v23.segment,
    Packages.ca.uhn.hl7v2.model.v23.datatype,
    Packages.ca.uhn.hl7v2.util,
    java.io);

with (javaImports) {
    var message     = dataSet.getActualData();
    var destMessage = new ORU_R01();

    // Copy MSH from the source
    DeepCopy.copy(message.getMSH(), destMessage.getMSH());

    // Change sender
    destMessage.getMSH().getSendingApplication().getNamespaceID().setValue("Integrator");

    var responses = message.getRESPONSEAll();

    for (var resNum = 0; resNum < responses.length; resNum++) {
        var response     = responses.get(resNum);
        var destResponse = destMessage.getRESPONSE(resNum);

        // Copy PID and ORDER_OBSERVATION sub-pieces, applying any
        // per-field transformations the receiving system needs.
        DeepCopy.copy(response.getPATIENT().getPID(),
                      destResponse.getPATIENT().getPID());

        var orderObs     = response.getORDER_OBSERVATIONAll();
        for (var ooNum = 0; ooNum < orderObs.length; ooNum++) {
            var srcOrderObs  = orderObs.get(ooNum);
            var destOrderObs = destResponse.getORDER_OBSERVATION(ooNum);

            DeepCopy.copy(srcOrderObs.getORC(), destOrderObs.getORC());
            DeepCopy.copy(srcOrderObs.getOBR(), destOrderObs.getOBR());

            // Reformat OBX observations per the receiving system's spec
            var obxAll = srcOrderObs.getOBSERVATIONAll();
            for (var obxNum = 0; obxNum < obxAll.length; obxNum++) {
                DeepCopy.copy(obxAll.get(obxNum).getOBX(),
                              destOrderObs.getOBSERVATION(obxNum).getOBX());
                // … field-level edits here …
            }
        }
    }

    dataSet.setActualData(destMessage);
}

Notes and limitations

Pick the right version package. HL7 2.3, 2.5, 2.5.1, and 2.6 messages each have their own classes — ORM_O01 in v23.message is a different class from ORM_O01 in v25.message. Importing the wrong package gives ClassCastException at runtime.
Mixing versions across a flow is allowed (parse 2.3, emit 2.5) but the script has to translate — you can't simply DeepCopy a 2.3 segment into a 2.5 segment instance.
Strict validation on the HL7 format may reject messages your script built if you forgot to set a required field. Disable strict validation temporarily while developing.
Error handling — HAPI throws HL7Exception from many APIs. Wrap in a try/catch when accessing optional segments / fields that may not exist.
Python flows can use the same HAPI APIs via direct package imports (from ca.uhn.hl7v2.model.v23.message import ORM_O01). JavaScript with JavaImporter is more common in the examples online.
For simple field-level edits the visual mapping editor is faster to build and easier to maintain — reserve scripting for cases where the logic is more than a rename / reorder.

Working with HL7 — the general HL7 reference: format settings, transports, the visual mapping path, ACK / NACK handling.
Nested mapping — the visual alternative to scripting for most HL7 transformations.
Execute any JavaScript — the JavaScript flow type the examples above use.
HAPI HL7v2 project page — upstream documentation, source code, and Javadoc.

Articles in this section

When should I use scripting instead of mapping?

The HAPI HL7v2 library

Supported HL7 versions

Package layout

Accessing the HL7 object model from a flow

Importing HAPI packages from JavaScript

Common scripting patterns

Get a field value

Set a field value

Iterate repeating segments

Insert / remove segments

Clone a message

Copy a segment between messages

Build a destination message from scratch

Example: lab order processing

What the transformation does

Flow shape

The transformation script

Flow setup

Example: lab results processing

What the transformation does

Flow shape

The transformation script

Notes and limitations

Related articles

Related articles