In the blog entry “GlassFish ESB v2.2 Field Notes – Exercising Load Balanced, Highly Available, Horizontally Scalable HL7 v2 Processing Solutions”, at https://blogs.czapski.id.au/?p=13, I present the GlassFish ESB v2.2-based load balanced, highly available, horizontally scalable solution for HL7 v2.x delimited messaging, using both the HL7 Binding Components, Web Services and JMS in request/reply mode. The one and a half hour recording of me discussing and demonstrating this solution is available as a Flash Movie (SWF), “GFESB_LB_HA_Demo_Session SWF ” at https://blogs.czapski.id.au/wp-content/uploads/2010/03/GFESB_LB_HA_Demo_Session_SWF.swf (62.7Mb download)
Jan 05
It seems frequently assumed that architecting and deploying Highly Available (HA) solutions requires Application Server and/or Operating System clustering. When it comes to SOA and Integration solutions this is not necessarily a correct assumption. Load Balanced (LB) and Highly Available HA) SOA and Integration solutions may not require that degree of complexity and sophistication. Frequently, protocol, binding component, JBI and architectural application design properties can be exploited to design highly available solutions. Testing LB and HA solutions requires infrastructure consisting of multiple hosts and the ability to “crash” hosts at will. With virtualization technologies available now it is far easier to use multiple virtual machines then to use physical machines. It is also easier and potentially less destructive to “crash” virtual machines then it is to do so with physical machines.
In this Note a heterogeneous, non-clustered collection of hosts will be used to implement and exercise three load balanced, highly available GlassFish ESB-based solutions. The environment consists of a number of independent “machines”, which are not a part of an Operating System Cluster. Each “machine” hosts a GlassFish Application Server. Application Servers are independent of one another and are not clustered. This is to demonstrate that load balanced, highly available, horizontally scalable solutions, based on the GlassFish ESB software alone, can be designed and implemented.
The specific class of solutions to which this discussion applies is the class of solutions which:
1. are exposed as request/reply services
a. HL7 messaging with explicit Application Acknowledgment
or
b. Request/Reply Web Services
or
c. JMS in Request/Reply mode
2. implement business logic as short lived processes
3. are
a. atomic
or
b. are idempotent
or
c. tolerant of duplicate messages
Classes of solutions with characteristics different from these named above require different approaches to high availability and horizontal scalability, and are not discussed here.
In this Note only high availability and scalability of receiver solutions is addressed. This aspect is the focus because a failure to process a message by a receiver may result in message loss –generally a bad thing.
Paradoxical as it may sound; senders are special cases of receivers. Just as a receiver is triggered by arrival of a message so too is a sender. Making sure that the sender trigger message does not get lost is much the same as making sure the message a receiver receives does not get lost. This means that the same considerations apply to senders and to receivers.
This note discusses an exercise involving an example load balanced, highly available, horizontally scalable healthcare environment, processing HL7 v2 messages. Discussion includes customization of generic GlassFish ESB v2.2 VMware Virtual Appliances for a specific Load Balancing and High Availability exercise and deploying ready-made GlassFish ESB solutions. The exercise for HL7 BC-based, Web Service-based and JMS-based highly available, load balanced, and horizontally scalable receivers, processing HL7 v2.3.1 messages, will be conducted and discussed.
At the end of the Note we will have three GlassFish ESB VMware Appliances with GlassFish ESB v2.2 Runtime infrastructure, ready to use for further GlassFish ESB Load Balancing and High Availability exercises.
The reader will be convinced, one hopes, that for the applicable class of GlassFish ESB-based solutions, load balancing and dynamic failover without message loss work. For that class of solutions this provides for high availability and horizontal scalability without resorting to Application Server or Operating System clustering.
The complete Note is available as 03_Conducting_HL7_LB_and_HA_Exercise_v1.0.0.1.pdf at https://blogs.czapski.id.au/wp-content/uploads/2010/03/03_Conducting_HL7_LB_and_HA_Exercise_v1.0.0.1.pdf
Sep 16
As a healthcare enterprise looks after patients, information is gathered about various events that take place. Information about notable events, Admissions and Discharges, for example, is recorded in Hospital Information Systems or Patient Administration Systems. These systems typically broadcast event information in a form of HL7 messages for use by other enterprise systems, for example laboratory or diagnostic imaging. A stream of HL7 messages can be intercepted and processed to derive all sorts of interesting information.
The solution developed in this walkthrough deals with Excessive Length of Stay. Length of stay is defined as the period between patient’s admission to and discharge from the hospital. Statistical average expected length of stay is typically available for different kinds of patients presenting with different kinds of conditions. A significant variation from the average length of stay for specific patients may indicate complications, treatment errors, infections and other kinds of issues that the hospital needs to investigate. Notification of such incidents may help the hospital in addressing these issues and prevent future occurrences.
In this solution the Intelligent Event Processor is used to calculate the continuously updated average length of stay over a period of time and use it to compare against each event’s length of stay. It passes, to the downstream component, all events where the length of stay exceeds the average by 1 ½ times and ignores all others.
In the initial iteration, the solution reads a stream of discharge messages, containing admission date, discharge date, length of stay, and a bunch of other fields from a file and passes them to the IEP process. The IEP process keeps the window on the last 10 seconds worth of records and continuously calculates the average length of stay over all records in that window. As records are added to and removed from the window the average is recalculated. As each record is seen its length of stay is compared to the average length of stay of all records in the window at the time. If the length of stay in the current record is less then or equal to 1 ½ times the average at the same time the record is discarded. If the average is greater the record is ejected to the output and ultimately written to a file of exception records.
In a subsequent iteration the solution is modified to accept messages from a JMS Queue. This modification allows the solution to use the stream of discharge messages produced by the HL7 Processor solution, discussed in “HL7 Processor Demonstration – GlassFish ESB v2.1”, https://blogs.czapski.id.au/?p=23.
In a further modification the solution is configured to send notification messages to another JMS Queue. Notification messages are processed by a different solution and sent to an email recipient.
The document, “Excessive length of Stay Healthcare IEP Demonstration”, can be found at https://blogs.czapski.id.au/wp-content/uploads/2010/03/Combinned_Intelligent_Event_Processor_Demonstration_v1.0.pdf
The pre-built projects in the “Excessive length of Stay Healthcare IEP Demo Companion Archive” can be found at https://blogs.czapski.id.au/wp-content/uploads/2010/03/Healthcare_Demo_Combined_v1.0.zip
Sep 04
This Note walks the reader through development of a GlassFish ESB v2.1based solution that addresses a Healthcare-related business problem. The Note elaborates on the healthcare background necessary to get a notion of what is being done and why, and provides detailed steps required to implement and exercise the solution to the business problem.
We will use the HL7 Binding Component, the File Binding Component, the JMS Binding Component, the SOAP/HTTP Binding Component, the BPEL 2.0 Service Engine, the JavaEE Service Engine, the HL7 Encoder and EJB-based Web Services in a JBI-based solution.
This note is an update, for GlassFish ESB v2.1, of the original note “HL7 Processor Demonstration – Java CAPS 6/JBI and OpenESB”, to be found at https://blogs.czapski.id.au/wp-content/uploads/2010/03/00_HL7_Example_Development_Instructions_Final.pdf.
Updated note is available at https://blogs.czapski.id.au/wp-content/uploads/2010/03/00_HL7_Example_Development_Instructions_Final_1.1.2.pdf
Jul 02
“Progress” notwithstanding, Healthcare environments still extensively use the HL7 v2.x Delimited messages for conveyance of patient and patient-related information between applications. The GlassFish ESB provides support for HL7 v2.x messaging in the form of the HL7 Encoder, which allows conversion between HL7 v2 Delimited and HL7 v2 XML message formats, and in the form of the HL7 Binding Component, which allow connectivity between the GlassFish ESB-based healthcare solutions and healthcare applications that support HL7 over TCP connectivity.
In this document I will walk through the process of generating HL7 v2.3.1 delimited messages from pipe-delimited records containing patient information, sending and receiving HL7 v2.3.1 delimited messages using the HL7 Binding Component, parsing HL7 v2.3.1 delimited messages and writing HL7 v2 delimited messages to a file. To create and process HL7 messages I show how create a custom ADT A04 XML Schema and a custom “any HL7 v2 message” XML Schema. This gives me an opportunity to use the File Binding Component (File BC), the HL7 BC, the HL7 Encoder, the Custom Encoder and the BPEL Service Engine (BPEL SE). This also gives me an opportunity to demonstrate a HL7 v2.3.1 delimited message sender solution and to demonstrate a HL7 v2.3.1 delimited message receiver solution. At the end of the process we will have a file containing HL7 v2 delimited ADT A04 messages, which we will use in related writeups.
Here is the document: 02_PatientSvc_MakeHL7v2DelimDataFromCustomDelimRecords_0.4.pdf
Here is the companion archive containing input files, the output file and the projects: 02_PatientSvc_MakeHL7v2DelimDataFromCustomDelimRecords_data.zip
The writeup document has been updated and version changed to 0.4.
Feb 28
This Quick Note discusses a solution to the use case provided by Marcus Davies.
I am trying to read HL7 from JMS (preferably stcms) and populate an outbound XML data structure (different to the XML generated by the decoder).
I have been thinking of doing one of the following […]:
1. Use a Concrete JMS WS using the HL7 encoders to unmarshal the HL7 and use JAXB to populate the outbound XML. Unfortunately this does not appear to connect to the stcms queue as I can not see any receivers
2. Use a JCA MDB to read from the stcms JMS queue – this works but I don’t think I can use the HL7 encoder like this
3. Use and MDB to read from JMS, manually unmarshal the HL7 and use JAXB to populate the data structure
Ideally I would like to use the HL7 encoders. Do you think the first approach should work?
Number 1 will not work as at end of February 2009 because the JMS BC does not properly decode the HL7 delimited message. This is a know issue. I don’t know what the status of this is. The only BCs that know how to deal with HL7 delimited, that I know of, are the File BC and the HL7 BC.
Number 2 should work. I did not personally try it. You can invoke an encoder library from Java. Have a look at http://wiki.open-esb.java.net/Wiki.jsp?page=UseEncodersInJavaSE.
Number 3 should work but it will be very laborious.
I have a Number 4, which uses a HL7 OTD and a custom XSD-based OTD in a JCA EJB. You may or may not like it but it’s the best thing to do if you can not use BPEL 2.0 to do the mapping and you don’t want to build a repository-based solution (which would be the best for your case anyway).
The solution involves the use of:
1. HL7 2.3.1 OTD Library (Java CAPS 6 Repository)
2. JMS JCA to trigger a MDB with a HL7 Delimited message
3. JMS JCA to write result message out
4. JCA MDB to do the processing
5. OTDImporter to provide HL7 2.3.1 OTD and custom XSD-based OTD to the EJB for “convenient” mapping
Brief steps to implement this solution are given in Quick Note 002 at QuickNote002_For_Marcus_Davies.pdf. Archive containing project exports and sample data is provided at QuickNote002.zip. The code will work in Java CAPS 6 Update 1.
Jan 08
The Note “HL7 Processor Demonstration – Java CAPS 6/JBI and OpenESB” walks the reader through development of a Java CAPS 6/JBI-based / OpenESB-based solution that addresses a Healthcare-related business problem. The Note elaborates on the healthcare background necessary to get a notion of what is being done and why, and provides detailed steps required to implement and exercise the solution.
I recorded a screencast of a session during which I discuss the business side of the Note, then discuss, implement, deploy and exercise all components of the solution documented in the Note.
The screencast is here: HL7Processor_Exercise_Screencast.avi. The associated archive, 00_HL7Processor_example_screencast_companion.zip, contains code fragments and other bits and pieces which are used, or referred to, in the screencast. Of some interest are the Note itself, in documents/00_HL7_Example_Development_Instructions_Final.pdf, and the brief example implementation instructions, in documents/00_HL7_ExampleBrief.pdf. I followed the brief instructions while I was building the projects when recording the screencast.
The screencast, which is over 320 Mb in size and takes 2 hours and 50 minutes to play, may require a TechSmith Compression Codec on your machine to allow your player to play the media. You can get one from the TechSmith site: http://www.techsmith.com/download/codecs.asp. Information on the codec can also be found here: http://www.movavi.com/codec/TSCC.html. If you prefer, and you are on Windows, you can get the CamPlay.exe from here: CamPlay.zip and use it instead.
Jan 01
This Note walks the reader through development of a Java CAPS 6/JBI-based / OpenESB-based solution that addresses a Healthcare-related business problem. The Note elaborates on the healthcare background necessary to get a notion of what is being done and why, and provides detailed steps required to implement and exercise the solution to the business problem.
Updated note, where GlassFsish ESB v2.1 is used instead of Java CAPS 6, is available at 00_HL7_Example_Development_Instructions_Final_1.1.2.pdf
We will use the HL7 Binding Component, the File Binding Component, the JMS Binding Component, the SOAP/HTTP Binding Component, the BPEL 2.0 Service Engine, the JavaEE Service Engine, the HL7 Encoder and EJB-based Web Services in a JBI-based solution.
In the process we will create XML Schema Documents (XSDs), Web Services Description Language Documents (WSDLs), a BPEL 2.0 Business Process, an EJB-based “Implementation First” web service, an EJB- and WSDL-based “Interface First” web service, a bunch of Composite Applications, BPLE 2.0 mapping, BPEL 2.0-based Web Service orchestration, on-the-fly conversion of HL7 version 2.3.1 delimited messages to their XML equivalents. We will get a pretty good exposure to what OpenESB and Java CAPS 6/JBI components look like, how they work and how they can be used to create real business solutions. Above all, we will develop and test a solution that is more sophisticated then the customary “Hello World” examples but not so complex as to take too long to build and become too hard to comprehend by a novice user.
The particular business problem and the particular solution came about because once upon a time there was intent to build a series of related OpenESB projects – HL7 Processor, MDM Processor and IEP Processor – that would:
- receive HL7 v2.x delimited messages
- convert HL7 v2.x messages to their equivalent XML format
- split message stream into ADT A01s, ADT A03s and other
- convert A01s to an abbreviated Custom Patient XML format
- convert A03s to an abbreviated Custom Discharge format
- send Custom Patients to a JMS Queue for processing by a MDM solution
- send Custom Discharges to a JMS Queue for processing by an IEM solution
- have the MDM process Custom Patients into a Master Patient Index
- have the IEP process Custom Discharges to flag excessive length of stay
The MDM Processor and the IEP Processor made it to the Sun CEC 2008 as demonstrations, with associated Tutorials by Tom Barrett, and demonstration recordings by me. The HL7 Processor did not make it. With the appearance of Java CAPS 6 Update 1 more JBI components made it into the officially supported Sun product. While the HL7 BC and the HL7 Encoder did not make it into this Update they will, eventually. Both components are already available from the OpenESB site and can be installed into the Java CAPS 6 Update 1 installation as unsupported components. This is what we will do for this Note.
The complete Note is to be found here.
The archive containing supplementary material you can use to save yourself the trouble of downloading component and, above all else, containing test files with HL7 delimited records, is here.
For these interested in processing HL7 using GlassFish ESB v2.1 there is a blog entry, “GlassFish ESB v2.1, MySQL v5.1 – Make HL7 v2.3.1 Delimited Messages from Custom Delimited Records with HL7 Encoder“, which discusses, amongst other things, how to create a custom HL7 v2 ADT A04 XSD and a “match any” XSD.
