Back to Blog
Healthcare

Lessons from Building Healthcare Interoperability at the U.S. Department of Veterans Affairs

Sandeep Reddy Kaidhapuram · Founder & Lead ArchitectApril 21, 202610 min
HealthcareVAHL7Interoperability

The Invisible Problem That Affects Millions

Before I worked on the FDA's drug review modernization, I spent formative years as an integration engineer at the U.S. Department of Veterans Affairs. The VA is the largest integrated healthcare system in the United States, serving over 9 million enrolled veterans across more than 1,200 healthcare facilities. On paper, it's one system. In practice, when I arrived, it was dozens of disconnected systems that happened to share a logo.

The problem was deceptively simple to describe and enormously difficult to solve: a veteran who received care at a VA facility in San Diego had medical records that were essentially invisible to a VA facility in Boston. A clinician in Boston treating that same veteran for a new condition couldn't see the medications prescribed in San Diego, the lab results from a visit in Phoenix, or the surgical history documented in Atlanta. They were making treatment decisions with incomplete information — and in healthcare, incomplete information can be the difference between effective treatment and a dangerous drug interaction.

My job was to help fix this. The technology we used was HL7v2 messaging over IBM Integration Bus. The lessons I learned there about interoperability, data semantics, and the gap between protocol compliance and genuine integration have shaped every system I've built since — including the agentic AI architectures I design today.

Understanding the Technical Landscape

The VA's electronic health record landscape was a product of decades of organic growth. Different facilities had adopted different EHR systems at different times, with different configurations and customizations. Some facilities ran versions of VistA (the VA's homegrown EHR system) that had been locally modified over years. Others had begun migrating to newer commercial systems. The result was a patchwork of systems that each contained a piece of a veteran's health story but couldn't share that story with each other.

HL7v2 — Health Level Seven version 2 — was the messaging standard we used to bridge these systems. HL7v2 defines message formats for clinical events: patient admissions, lab results, medication orders, discharge summaries, and dozens of other healthcare data types. The idea is straightforward — when something clinically significant happens, the source system generates an HL7v2 message and sends it to systems that need to know.

The reality is considerably more complex. HL7v2 is not a strict specification — it's a framework that allows extensive local customization. The standard defines message structures with required and optional segments, but implementations routinely extend, modify, or reinterpret those segments. Two systems can both claim HL7v2 compliance while producing messages that are structurally different in ways that break naive parsing.

Building the Integration Pipelines

My work centered on building HL7v2 message-based integration pipelines using IBM Integration Bus (now called IBM App Connect Enterprise). These pipelines received HL7v2 messages from source EHR systems, transformed them into a normalized format, validated the data, and routed the normalized messages to a shared clinical data repository that any authorized facility could query.

The architecture had three layers:

  • Ingestion layer: received raw HL7v2 messages from source systems over TCP/IP connections (the MLLP protocol — Minimal Lower Layer Protocol — that HL7v2 typically runs over). Each source system had its own connection with its own message format variations, so the ingestion layer included source-specific parsing logic.
  • Transformation layer: converted source-specific message formats into a canonical data model. This was the hardest layer to build and maintain, because every source system had its own interpretation of HL7v2 fields, its own code sets for diagnoses and procedures, and its own conventions for optional segments.
  • Distribution layer: routed normalized messages to the shared clinical data repository and to any subscribing systems that needed real-time clinical data updates. This layer also handled acknowledgment tracking — confirming that every message was received and processed by every intended recipient.

The Complexity of HL7v2: Where Theory Meets Reality

HL7v2 is notoriously difficult to work with, and my experience at the VA taught me exactly why. The challenges weren't in the protocol specification — they were in the gap between what the specification allows and what implementations actually do.

Field interpretation varies wildly. The HL7v2 PID (Patient Identification) segment has a field for "patient name." Sounds simple. But one system might put the legal name there while another puts the preferred name. One system formats it as "Last^First^Middle" while another uses "Last^First^Middle^Suffix^Prefix." One system populates the "name type code" component to distinguish legal from alias names; another leaves it blank. Every one of these variations required specific transformation logic.

Code sets are inconsistent. A diagnosis code in one system might use ICD-10-CM, while the same diagnosis in another system uses an internal code set that maps loosely to ICD-10-CM but not exactly. Medication codes might be NDC in one system and a local formulary code in another. Each of these inconsistencies required building and maintaining translation tables — and those tables needed constant updates as code sets evolved.

Optional segments are used inconsistently. HL7v2 defines many optional message segments for additional clinical context. One system might include detailed allergy information in an AL1 segment; another might omit it entirely and encode allergies in a free-text note within a different segment. Building a complete patient picture required understanding not just what each system sent but what each system omitted and where that omitted information might be found elsewhere.

Error handling must be bulletproof. When a message fails to parse or validate, the failure mode matters enormously. In e-commerce, a failed message might mean a delayed order confirmation. In healthcare, a failed message might mean a clinician doesn't see a critical lab result. We built multi-tier error handling: automatic retry for transient failures, dead-letter queues for persistent failures with automated alerting, and manual review workflows for messages that failed validation in ways that suggested data quality issues at the source.

Monitoring and Reliability: Clinical Data Demands Zero Tolerance

One of the most important systems we built wasn't part of the message pipeline at all — it was the monitoring and alerting infrastructure that ensured the pipeline was working correctly at all times.

We implemented real-time dashboards showing message throughput, processing latency, error rates, and queue depths for every integration point. We set up automated alerts for anomalies: if a source system that normally sends 500 messages per hour suddenly drops to zero, that's not a quiet day — it's likely a connectivity failure that needs immediate investigation.

We also built reconciliation reports that ran on a scheduled basis, comparing the messages sent by source systems against the messages received by destination systems. Any discrepancy triggered an investigation. These reconciliation reports were arguably the most important quality assurance tool in the entire system — they caught issues that real-time monitoring missed, particularly edge cases where messages were received but processed incorrectly.

The Impact: Seeing the Whole Patient

When the integration pipelines were operational, the impact was immediate and tangible. Clinicians at any VA facility could access a veteran's complete clinical history — medications, lab results, diagnoses, procedures, allergies — regardless of where that care was delivered. A clinician in Boston could see that the patient sitting in front of them had been prescribed a specific blood thinner in San Diego two weeks ago, and could make informed decisions about new prescriptions that wouldn't create dangerous interactions.

This sounds like basic functionality — and it should be. The fact that it required years of integration engineering to achieve tells you something important about the state of healthcare IT. The technology to share clinical data existed. The protocols existed. What was missing was the hard, unglamorous work of bridging the gap between systems that technically spoke the same protocol but practically spoke different dialects.

The Interoperability Lesson That Changed My Thinking

The deepest lesson from my VA work was this: connecting systems isn't a technology problem — it's a semantics problem. Two systems can speak the same protocol, exchange messages successfully, and acknowledge receipt correctly — and still be completely wrong about what those messages mean.

HL7v2 compliance didn't guarantee interoperability. A system that sent a perfectly formatted ADT^A01 (patient admission) message could be interpreted completely differently by the receiving system if the two systems had different conventions for populating the message fields. The protocol standardized the envelope; it did not standardize the meaning of the contents.

Protocol compliance is step one. Semantic alignment — ensuring that all systems agree on what the data means, not just how it's formatted — is step two. And step two is where most interoperability efforts succeed or fail.

This lesson applies far beyond healthcare. Every integration project I've worked on since — including the FDA modernization and my current work with MCP and agent communication — has reinforced the same principle. The hard problem is never the protocol. The hard problem is the meaning.

From HL7v2 to MCP: The Evolution Is Surprisingly Direct

When I first encountered the Model Context Protocol, I experienced a strong sense of recognition. MCP is, at its core, an interoperability protocol for connecting disparate systems — just like HL7v2. The systems being connected are different (AI agents and tools, rather than EHR systems), but the architectural challenges are remarkably similar:

  • Context passing: HL7v2 messages carried clinical context — patient identity, encounter information, ordering provider. MCP messages carry task context — what the agent is trying to accomplish, what permissions it has, what prior results are relevant. In both cases, the quality of the context determines the quality of the outcome.
  • Identity management: HL7v2 messages included sending facility, receiving facility, and message control IDs for tracking and audit. MCP needs agent identity, delegation chain, and permission scope. The identity challenge is more complex for agents (because delegation chains are deeper and more dynamic), but the underlying requirement is the same.
  • Audit trails: In healthcare, every message exchange must be auditable for regulatory compliance. In agentic AI, every tool invocation must be auditable for governance and debugging. The architectural pattern — persist every event with full context — is identical.
  • Error handling and reliability: HL7v2 integration required guaranteed delivery, retry logic, and reconciliation because lost clinical messages could harm patients. MCP integration in enterprise settings requires the same guarantees because lost or corrupted tool responses can lead to incorrect agent decisions with business consequences.

The difference — and it's a significant one — is that MCP connects intelligent agents rather than passive endpoints. An HL7v2-receiving system processes a message according to deterministic rules. An MCP-using agent processes a tool response according to probabilistic reasoning. This makes the interoperability challenge simultaneously more powerful and more dangerous. More powerful because agents can handle ambiguity and adapt to unexpected responses. More dangerous because agents can misinterpret responses in ways that deterministic systems cannot.

What the VA Taught Me About Building for the Future

My years at the VA taught me that interoperability is a long game. The systems we built didn't solve healthcare data fragmentation overnight — they established a foundation that subsequent teams continued to build on. The canonical data model we defined, the transformation patterns we created, the monitoring infrastructure we deployed — all of these became building blocks for the next generation of integrations.

I carry the same philosophy into my current work with agentic AI. The MCP servers, agent governance frameworks, and integration patterns we're building today at StackAhead.ai aren't finished products — they're foundations. The specific AI models and tools will evolve rapidly. But the principles of reliable message delivery, semantic clarity, comprehensive audit trails, and governance-first architecture will remain constant. They worked for HL7v2 at the VA. They worked for middleware at the FDA. And they'll work for agentic AI in the enterprise.

The veterans who received better care because their clinicians had access to complete information — that's the outcome that makes integration engineering meaningful. And it's the same outcome we're pursuing with every agentic system we design: connecting systems so that the people who depend on them get better results.

Stay ahead of the stack

Weekly insights on agentic architecture, protocol updates, and governance patterns. No fluff — just what architects need.