Jul 24

Java CAPS 6, JCA, Note 4 – Streaming FTP Inbound

By Michael Czapski Sun No Comments »

As at now there does not seem to be a way to stream FTP payloads to the local file system, or stream local files to FTP servers in the JBI environment. This kind of functionality may or may not appear in the OpenESB project, perhaps in conjunctions with the FTP BC or the File BC or both. In the meantime, a Java CAPS 6 developer can use the JCA Adapters-based solution to stream payloads of arbitrary size between FTP servers and local file systems in either direction.

This Note walks through the implementation of a Batch FTP JCA solution, which is triggered by a JMS Message, and performs a streaming FTP transfer of an arbitrarily large payload between a remote FTP server and the local file system. This Note re-implements the inbound part of the “Java CAPS 5.1 and Java CAPS 6 – Streaming Large FTP Transfers” Note available at http://blogs.sun.com/javacapsfieldtech/entry/streaming_large_ftp_transfers_with.
It should be easy enough to re-implement the outbound part of that Note using the material in this Note.

The complete Note is available in 04JCAStreamingFTPIn.pdf.

Tagged with: • • • • • • • • • • • • •
Apr 05

Java CAPS 5.1 and Java CAPS 6 – Streaming Large FTP Transfers

By Michael Czapski Sun 34 Comments »

Transferring large payloads, on the order of tens or hundreds of megabytes, between a FTP server and a local file system, in either direction, requires selection of appropriate features of the Batch FTP and Batch Local File eWays, and tuning certain timing parameters.

Default timing parameter values result in timeout exceptions when transferring large payloads.

The Batch FTP eWay and the Batch Local File eWas are typically used to receive the entire payload before writing it out. This results in attempts to allocate memory many time the size of the payload being transferred and, for large files, causes memory exhaustion and application server failures.

Discussion in the attached document points out which timing parameters need to be tuned to facilitate transfer of large payloads. It also presents sample Java code that uses facilities of the Batch FTP and Batch Local File for streaming payload between the FTP server and the local file systems without using excessive amount of memory.

The material covered in the document was prepared using Java CAPS projects developed and tested in Java CAPS 5.1.0, exported, imported into Java CAPS Release 6 and tested again. It is expected that the code will work in all versions of Java CAPS from 5.1.0 up.

Streaming Large FTP Transfers with CAPS 5.1 and 6.pdf

FTPtoLocalFileStreaming_5.1.0_project_export.zip

LocalFiletoFTPStreaming_JC6_project_export.zip

Tagged with: • • • • • • • • •
preload preload preload