Java CAPS 6 has the 5.x compatibility infrastructure which allows one to import 5.x projects right into Java CAPS 6, build, deploy and run without changes. One can also develop repository-based projects in Java CAPS 6 – that’s the 5.x-style projects. This is the old way of developing Java CAPS solutions – still good and valid.
If one were to decide to not use the old way there is the JBI infrastructure, which allows development of solutions that use BPEL Service Engine, XSLT Service Engine, IEP Service Engine, Java EE Service Engine, etc., and a variety of Binding Components. The implication is that business logic is implemented in BPEL 2.0, which is used to orchestrate other services and resources, including interaction with external systems through Binding Components. This is the new way of developing Java CAPS solutions – 100% compatible with the Open Source OpenESB project since it uses the OpenESB project-developed container and components.
Someone might ask “so what happened to eGate?”. “eGate” meaning Java Collaboration Definition-like logic components, eWays and the JMS messaging backbone.
While the facility seems underadvertised/downplayed, Java CAPS 6 provides a number of 5.1 eWay-based JCA Adapters and a moderately easy means of developing JCA Message-Driven Beans that can use these adapters to implement JCD-like logic components and, effectively, eGate-like solutions that do not use BPEL or the JBI infrastructure.
This Note discusses and illustrates the implementation of a mixed Java CAPS 5.x-like integration solution that retrieves a file from the local file system using JCA Adapters and passes its content to a BPEL 2.0 process executing in the JBI container. This requirement I have seen and heard of being implemented in 5.x many times by many customers.
Most of the material in the first 16 pages of this Note is the same as in Note 2.
The JCA Message-Driven Bean, the piece of JCD-like Java logic, will be triggered by a Batch Inbound Adapter (what one would have called the Batch Inbound eWay in 5.1), will read the content of the file using the Batch Local File Adapter (eWay) and will send the payload as a string to a BPEL 2.0 Business Process, which will be triggered by this message and will execute in the JBI container. The batch Inbound Adapter will be configured to use a regular expression to match the name of the file. Once it finds the file it will rename the file by prepending the GUID to the name and will pass the new name, the original name and the directory path to the Java code. This is exactly what the 5.1 Batch Inbound does. The JCA MDB will use the new name, the original name and the directory path to dynamically configure the Batch Local File Adapter to retrieve the file content and rename the file (post transfer) to the original name with some string appended to indicate that the file was processed. This, too, is exactly what one would do in a 5.1 JCD in the same circumstance. Once the payload is available the JCA MDB will use the OneWay WSDL interface and the JBI NMR to send it, as a String, to a BPEL 2.0 process. Both the JCA MDB and the BPEL process will be a part of the same JBI Composite Application and will communicate with one another using the Normalized Message Router (NMR).
The entire Note is available in 03JCA-BInboundThroughBLFToBPEL2.0.pdf
Hi Michael,
The various solution patterns discussed by you in this blogs are excellent source of information. They are helping me to understand the OpenESB style integration using JBI specs.
I would love to see some usecases showing different parsing methods available within Java EE5 and JBI supported components.
Thanks,
Mayank
Thanks Mayank. I appreciate the good word.
The only thing I have so far is http://blogs.sun.com/javacapsfieldtech/entry/java_caps_6_jbi_note2
I will see what else I can do 🙂
Regards
Michael
Hi Michael,
We are using Batch Inbound and Batch Local adapters as you suggest in this tutorial.
Do you have a solution how to configure the file names and directory names for the Batch Inbound and Batch Local adapters without a need to recompile and redeploy the project everytime they change.
I want to specify different configurations for development, test and production environments.
The polling directory of the Batch Inbound adapter can be specified from the Netbeans -> EJB-project -> Java Collaborations -> Edit JCA Activation -> Configuration, but then I have to recompile and redeploy the project separately for all environments.
Regards,
Kimmo
Hello, Kimno.
Alas, my colleagues in Engineering advise that there is no mechanism to modify directory path and file name pattern of the Batch Inbound JCA without rebuilding and redeploying the application. A workaround would be to develop a scheduler-triggered JCA which uses the Batch Local File JCA to check if the file exists and load and process it when it does. The Batch Local File JCA, used in this way, has its configuration properties settable through the sub node of the CAPS node of the GlassFish Application Server Console by virtue of the Connection Pool being configured through the GF Console. For changes to take effect you still need go through the disable/enable application cycle using the GF App Console.
Regards
Michael
Hi Michael,
Thank you for this information.
I try to develop a workaround as you suggested.
I think that Batch Inbound adapter is quite restricted if the configuration can’t be modified through Glassfish admin console. Any plans to develop another tutorial that describes the replacement for Batch Inbound?
Regards,
Kimmo
Hello, Kimo.
I understand from my Engineering colleagues that there is a plan to provide the ability to externalise the Batch Inbound configuration though of course there is no date promised.
I don’t have a plan to do a tutorial like that in the foreseeable future. Perhaps the scheduler example.combined with the JCA that uses a Batch Local File rather then the JMS JCA, will do the trick for you. See http://blogs.sun.com/javacapsfieldtech/entry/java_caps_6_scheduler_for
Regards
Michael
Hi Michael,
I look at the scheduler example and try to combine it with the Batch Local File.
Thank you!
Regards,
-Kimmo
Hello Michael,
First of all – sorry about my english… 😉
We use CAPS 5.1.3 and the BatchLocalFile-Adapter. This works greate in case there are only a fiew files waiting to be processed.
If you put several thousands file in den Source-Directory, CAPS no longer process this files as it should – because it consumes a lot of time from one file to the other… we don’t no the exact reason – may be caps constructs a new file-List each time we call the getifexists.
Is there a way to get the file-list and walk through all the files waiting in the list – before calling/rebuilding the list?
Thank you
Thomas
Hello, Thomas.
I assume that you are using the Batch Inbound to find the files in the directory.
Unfortunately, Batch Inbound only deals with one file at a time and waits for the pooling interval time before it checks again.
One possible solution would be to:
1. have a JCA MDB triggered by a JMS message, which would be deposited in a JSM queue by a scheduling component.
2. Have the JCA MDB use the Batch Local File JCA to find and process all files that fit the pattern.
The difficulties will be in getting a scheduling component together to trigger the JCA MDB and making sure that the component does not get triggered again until it processed all qualifying files.
Having thousands of files in a directory for processing suggests to me that the integration architecture needs to be looked at seriously. File transfer-based solutions are typically very brittle and inefficient. File transfer-based solutions with thousands of files are likely to cause more trouble as the time goes by. Perhaps the applications that generate the files can be integrated in a more efficient manner.
Regards
Michael
Hello Michael,
Thank you for your answer.
I agree with you – but in switzerlands healthcare fileinterfaces are very common – no chance to eliminate them at this time 😉 .
Normaly there would be only a few files in that directory at a time.
Because we produce a initial-export of all patients in the source-db, there are thousends of files in that directory at this time…
We do it as you describe in den second case.
We have a Scheduler that fires every 5 sec. – the messages have a lifetime of 3.5sec.
If I did it correct – only one process runs at a time.
And if the process takes 5minutes, there will be only one fire-message waiting.
In the JCD I wrote a loop that checks with getifexists() if there are any files to
process.
After each file processed, I check again with getifexists() for waiting files.
It seams to me, that this check ‘getifexists()’ consumes a lot of time to collect all the thousands of files waiting.
How should I process all files waiting at a time?
Regards
Thomas
Hello, Thomas.
Section 7.2.1.2, "Poll for Multiple Local Files", in Part II of the "Java CAPS Basics" book, which is available on the CD-ROM accompanying Part I, the hardcopy book, shows the JCD code for this case reproduced from the book.
Here it is, without explanation.
public void start
(com.stc.schedulerotd.appconn.scheduler.FileTextMessage input
,com.stc.eways.batchext.BatchLocal G_BatchLocalFile
,com.stc.connectors.jms.JMS W_toJMS )
throws Throwable
{
logger.debug( "\n===>>> Trying for a file" );
G_BatchLocalFile.getClient().getIfExists();
;
// payload will be null if file did not exist for getIfExists!
;
while (G_BatchLocalFile.getClient().getPayload() != null) {
;
com.stc.eways.batchext.TransferNamesAndCommands tnc
= G_BatchLocalFile.getClient().getResolvedNamesToGet();
logger.debug( "\n===>>> Target Directory: "
+ tnc.getTargetDirectoryName() );
logger.debug( "\n===>>> Target File : "
+ tnc.getTargetFileName() );
;
logger.debug( "\n===>>> File found. Passing payload on …." );
W_toJMS.setTimeToLive( 2 * 60 * 1000 );
W_toJMS.sendText(
new String( G_BatchLocalFile.getClient().getPayload() )
);
;
G_BatchLocalFile.reset();
G_BatchLocalFile.getClient().getIfExists();
}
}
Based on your comment I expect you to have no difficulties adapting the code for your needs.
Regards
Michael
Project exports of all projects from the Java CAPS Book are available on-line. See http://blogs.sun.com/javacapsfieldtech/entry/java_caps_basics_book_project
Michael
Hello Michael,
Thanks a lot of your assistenc!
I’m very surprised – I recognized that I have a book of you … Java CAPS Basics!
;-))
We do it exactly as you wrote:
G_BatchLocalFile.getClient().getIfExists();
while (G_BatchLocalFile.getClient().getPayload() != null) {
// do what ever…
G_BatchLocalFile.reset();
G_BatchLocalFile.getClient().getIfExists();
}
At my point of understanding the getifExists() takes about 5minutes to get the next file – if there halfe a million of files in the directory.
I am right?
Should I try with get() and handle the exception if no file is present?
Thanks a lot.
Thomas
Hello, Jakob.
Do try plain get.
I am working, in the background, on a blog entry that will discuss the issue of processing large number of files. If time permits, a week or two should see it done.
Regards
Michael
Hello Michael,
Thanks for your help ‘in the background’.
I’ve found your solution and will implement it in the future.
Best regards
Thomas
My name is Piter Jankovich. oOnly want to tell, that your blog is really cool
And want to ask you: is this blog your hobby?
P.S. Sorry for my bad english
Hello, Peter.
Thanks Peter.
I blog to help people and to record my own “discoveries” as I work with the products about which I write. I do this in my own time so you could call it a “hobby” 🙂
Regards
Michael