The Reference Message sample programs

The Reference Message samples allow a large object to be transferred from one node to another (usually on different systems) without the need for the object to be stored on WebSphere MQ queues at either the source or the destination nodes.

A set of sample programs is provided to demonstrate how Reference Messages can be 1) put to a queue, 2) received by message exits, and 3) taken from a queue. The sample programs use Reference Messages to move files. If you want to move other objects such as databases, or if you want to perform security checks, you must define your own exit, based on our sample, amqsxrm. The following sections describe the Reference Message sample programs.

There are four versions of the Reference Message exit sample program. The one to use depends on the platform on which the channel is running. If the sender channel is running on:

MQSeries or WebSphere MQ Version 5 products (excluding WebSphere MQ for iSeries and WebSphere MQ for z/OS): Use amqsxrma at the sending end. Use amqsxrma at the receiving end if the receiver is running under MQSeries or WebSphere MQ Version 5 products (excluding WebSphere MQ for iSeries and WebSphere MQ for z/OS) or amqsxrm4 if the receiver is running under WebSphere MQ for iSeries.

Notes for OS/400 users

To receive a Reference Message using the sample message exit, specify a file in the root file system of IFS or any sub-directory so that a stream file can be created. The sample message exit on OS/400 creates the file, converts the data to EBCDIC, and sets the code page to your system code page. You then have the option of copying this file to the QSYS.LIB file system using the CPYFRMSTMF command. For example:

CPYFRMSTMF FROMSTMF('JANEP/TEST.TXT')
           TOMBR('qsys.lib.janep.lib/test.fie/test.mbr') MBROPT(*REPLACE)
           CVTDTA(*NONE)

Note that the CPYFRMSTMF command does not create the file. You must create it before running this command.

If you send a file from QSYS.LIB no changes are required to the samples. For any other file system ensure that the CCSID specified in the CodedCharSetId field in the MQRMH structure matches the bulk data you are sending.

When using the integrated file system, create program modules with the SYSIFCOPT(*IFSIO) option set. If you want to move database or fixed-length record files, define your own exit based on the supplied sample AMQSXRM4.

Running the Reference Message samples

The Reference Message samples run as follows:

Figure 34. Running the Reference Message samples

The figure shows a diagramatic view of two queue managers (QMGR1 and QMGR2) set up to run the Reference Message sample applications. The details of the configuration and the flows between objects are described in the text following the figure.

Set up the environment to start the listeners, channels, and trigger monitors, and define your channels and queues.

For the purposes of describing how to set-up the Reference Message example this refers to the sending machine as MACHINE1 with a queue manager called QMGR1 and the receiving machine as MACHINE2 with a queue manager called QMGR2.

Note:: The following definitions allow a Reference Message to be built to send a file with an object type of FLATFILE from queue manager QMGR1 to QMGR2 and to recreate the file as defined in the call to AMQSPRM (or AMQSPRMA on OS/400). The Reference Message (including the file data) is sent using channel CHL1 and transmission queue XMITQ and placed on queue DQ. Exception and COA reports are sent back to QMGR1 using the channel REPORT and transmission queue QMGR1.

The application that receives the Reference Message (AMQSGRM or AMQSGRMA on OS/400) is triggered using the initiation queue INITQ and process PROC. You need to ensure the CONNAME fields are set correctly and the MSGEXIT field reflects your directory structure, depending on machine type and where the WebSphere MQ product is installed.

The MQSC definitions have used an AIX style for defining the exits, so if you are using MQSC on OS/400 you will need to modify these accordingly. It is important to note that the message data FLATFILE is case sensitive and the sample will not work unless it is in uppercase.

On machine MACHINE1, queue manager QMGR1

MQSC syntax

define chl(chl1) chltype(sdr) trptype(tcp) conname('machine2') xmitq(xmitq)
msgdata(FLATFILE) msgexit('/usr/lpp/mqm/samp/bin/amqsxrm(MsgExit)')
 
define ql(xmitq) usage(xmitq)
 
define chl(report) chltype(rcvr) trptype(tcp) replace
 
define qr(qr) rname(dq) rqmname(qmgr2) xmitq(xmitq) replace

OS/400 command syntax

Note:

If you do not specify a queue manager name the system uses the default queue manager.

CRTMQMCHL  CHLNAME(CHL1) CHLTYPE(*SDR) MQMNAME(QMGR1) +
             REPLACE(*YES) TRPTYPE(*TCP) +
             CONNAME('MACHINE2(60501)') TMQNAME(XMITQ) +
             MSGEXIT(QMQM/AMQSXRM4) MSGUSRDATA(FLATFILE)
 
CRTMQMQ    QNAME(XMITQ) QTYPE(*LCL) MQMNAME(QMGR1) +
             REPLACE(*YES) USAGE(*TMQ)
 
CRTMQMCHL  CHLNAME(REPORT) CHLTYPE(*RCVR) +
               MQMNAME(QMGR1) REPLACE(*YES) TRPTYPE(*TCP)
 
CRTMQMQ    QNAME(QR) QTYPE(*RMT) MQMNAME(QMGR1) +
             REPLACE(*YES) RMTQNAME(DQ) +
             RMTMQMNAME(QMGR2) TMQNAME(XMITQ)

On machine MACHINE2, queue manager QMGR2

MQSC syntax

define chl(chl1) chltype(rcvr) trptype(tcp)
msgexit('/usr/lpp/mqm/samp/bin/amqsxrm(MsgExit)')
        msgdata(flatfile)
 
define chl(report) chltype(sdr) trptype(tcp) conname('MACHINE1')
        xmitq(qmgr1)
 
define ql(initq)
 
define ql(qmgr1) usage(xmitq)
 
define pro(proc) applicid('/usr/lpp/mqm/samp/bin/amqsgrm')
 
define ql(dq) initq(initq) process(proc) trigger trigtype(first)

OS/400 command syntax

Note:

If you do not specify a queue manager name the system uses the default queue manager.

CRTMQMCHL  CHLNAME(CHL1) CHLTYPE(*RCVR) MQMNAME(QMGR2) +
             REPLACE(*YES) TRPTYPE(*TCP) +
             MSGEXIT(QMQM/AMQSXRM4) MSGUSRDATA(FLATFILE)
 
CRTMQMCHL  CHLNAME(REPORT) CHLTYPE(*SDR) MQMNAME(QMGR2) +
             REPLACE(*YES) TRPTYPE(*TCP) +
             CONNAME('MACHINE1(60500)') TMQNAME(QMGR1)
 
CRTMQMQ    QNAME(INITQ) QTYPE(*LCL) MQMNAME(QMGR2) +
             REPLACE(*YES) USAGE(*NORMAL)
 
CRTMQMQ    QNAME(QMGR1) QTYPE(*LCL) MQMNAME(QMGR2) +
             REPLACE(*YES) USAGE(*TMQ)
 
CRTMQMPRC  PRCNAME(PROC) MQMNAME(QMGR2) REPLACE(*YES) +
             APPID('QMQM/AMQSGRM4')
 
CRTMQMQ    QNAME(DQ) QTYPE(*LCL) MQMNAME(QMGR2) +
             REPLACE(*YES) PRCNAME(PROC) TRGENBL(*YES) +
             INITQNAME(INITQ)

Once the above WebSphere MQ objects have been created:
1. Where applicable to the platform, start the listener for the sending and receiving queue managers
2. Start the channels CHL1 and REPORT
3. On the receiving queue manager start the trigger monitor for the initiation queue INITQ

Invoke the put Reference Message sample program AMQSPRM (AMQSPRMA on OS/400) from the command line using the following parameters:

-m	Name of the local queue manager, this defaults to the default queue manager
-i	Name and location of source file
-o	Name and location of destination file
-q	Name of queue
-g	Name of queue manager where the queue, defined in the -q parameter exists This defaults to the queue manager specified in the -m parameter
-t	Object type
-w	Wait interval, that is, the waiting time for exception and COA reports from the receiving queue manager

For example, to use the sample with the objects defined above you would use the following parameters:

-mQMGR1 -iInput File -oOutput File -qQR -tFLATFILE -w120

Increasing the waiting time will allow time for a large file to be sent across a network before the program putting the messages times out.

amqsprm -q QR -m QMGR1 -i d:\x\file.in -o d:\y\file.out -t FLATFILE

OS/400 users:

Use the following command:

CALL       PGM(QMQM/AMQSPRM4) PARM('-mQMGR1' +
             '-i/refmsgs/rmsg1' +
             '-o/refmsgs/rmsgx' '-qQR' +
             '-gQMGR1' '-tFLATFILE' '-w15')

This assumes that the original file rmsg1 is in IFS directory /refmsgs and you want the destination file to be rmsgx in IFS directory /refmsgs on the target system.

Create your own directory using the CRTDIR command rather than using the root directory.

When you call the program that puts data, remember that the output file name will need to reflect the IFS naming convention; for instance /TEST/FILENAME will create a file called FILENAME in the directory TEST.

Note:: You can use either a forward slash (/) or a dash (-) when specifying parameters.

For example:

  amqsprm /i d:\files\infile.dat /o e:\files\outfile.dat /q QR
  /m QMGR1 /w 30 /t FLATFILE

Note:

For UNIX platforms, you must use two slashes (\\) instead of one to denote the destination file directory. Therefore, the above command looks like this:

  amqsprm -i /files/infile.dat -o e:\\files\\outfile.dat -q QR
  -m QMGR1 -w 30 -t FLATFILE

Running the put Reference Message program does the following:

The Reference Message will be put to queue QR on queue manager QMGR1.
The source file and path is d:\files\infile.dat and exists on the system where the example command is issued.
If the queue QR is a remote queue, the Reference Message is sent to another queue manager, on a different system, where a file is created with the name and path e:\files\outfile.dat. The contents of this file are the same as the source file.
amqsprm waits for 30 seconds for a COA report from the destination queue manager.
The object type is flatfile, so the channel used to move messages from the queue QR must specify this in the MsgData field.

When you define your channels, select the message exit at both the sending and receiving ends to be amqsxrm. This is defined on MQSeries for OS/2 Warp, and WebSphere MQ for Windows as follows:
```
  msgexit('<pathname>\amqsxrm.dll(MsgExit)')
```
This is defined on WebSphere MQ for AIX, WebSphere MQ for HP-UX, MQSeries for DIGITAL UNIX (Compaq Tru64 UNIX), and WebSphere MQ for Solaris as follows:
```
  msgexit('<pathname>/amqsxrm(MsgExit)')
```
If a pathname is specified, the complete name must be specified (including the extension .dll on OS/2). If a pathname is not specified, it is assumed that the program is in the path specified in the qm.ini file (or, on WebSphere MQ for Windows, the path specified in the registry). This is explained fully in WebSphere MQ Intercommunication.
The channel exit reads the Reference Message header and finds the file that it refers to.
It can then choose to segment the file before sending it down the channel along with the header. On WebSphere MQ for AIX, WebSphere MQ for HP-UX, MQSeries for DIGITAL UNIX (Compaq Tru64 UNIX), and WebSphere MQ for Solaris, you must change the group owner of the target directory to 'mqm' so that the sample message exit can create the file in that directory. Also, change the permissions of the target directory to allow mqm group members to write to it. The file data is not stored on the WebSphere MQ queues.
When the last segment of the file is processed by the receiving message exit, the Reference Message is put to the destination queue specified by amqsprm. If this queue is triggered (that is, the definition specifies Trigger, InitQ, and Process queue attributes), the program specified by the PROC parameter of the destination queue is triggered. The program to be triggered must be defined in the ApplId field of the Process attribute.
When the Reference Message reaches the destination queue (DQ), a COA report is sent back to the putting application (amqsprm).
The Get Reference Message sample, amqsgrm, gets messages from the queue specified in the input trigger message and checks the existence of the file.

Design of the Put Reference Message sample (amqsprma.c, AMQSPRM4)

This sample creates a Reference Message that refers to a file and puts it on a specified queue:

The sample connects to a local queue manager using MQCONN.
It then opens (MQOPEN) a model queue which is used to receive report messages.
The sample builds a Reference Message containing the values required to move the file, for example, the source and destination file names and the object type. As an example, the sample shipped with WebSphere MQ builds a Reference Message to send the file d:\x\file.in from QMGR1 to QMGR2 and to recreate the file as d:\y\file.out using the following parameters:
```
  amqsprm -q QR -m QMGR1 -i d:\x\file.in -o d:\y\file.out -t FLATFILE
```
Where QR is a remote queue definition that refers to a target queue on QMGR2.
Note:
For UNIX platforms, you must use two slashes (\\) instead of one to denote the destination file directory. Therefore, the above command looks like this:
```
  amqsprm -q QR -m QMGR1 -i /x/file.in -o d:\\y\\file.out -t FLATFILE
```
The Reference Message is put (without any file data) to the queue specified by the /q parameter. If this is a remote queue, the message is put to the corresponding transmission queue.
The sample waits, for the duration of time specified in the /w parameter (which defaults to 15 seconds), for COA reports, which, along with exception reports, are sent back to the dynamic queue created on the local queue manager (QMGR1).

Design of the Reference Message Exit sample (amqsxrma.c, AMQSXRM4)

This sample recognizes Reference Messages with an object type that matches the object type in the message exit user data field of the channel definition. For these messages, the following happens:

At the sender or server channel, the specified length of data is copied from the specified offset of the specified file into the space remaining in the agent buffer after the Reference Message. If the end of the file is not reached, the Reference Message is put back on the transmission queue after updating the DataLogicalOffset field.
At the requester or receiver channel, if the DataLogicalOffset field is zero and the specified file does not exist, it is created. The data following the Reference Message is added to the end of the specified file. If the Reference Message is not the last one for the specified file, it is discarded. Otherwise, it is returned to the channel exit, without the appended data, to be put on the target queue.

For sender and server channels, if the DataLogicalLength field in the input Reference Message is zero, the remaining part of the file, from DataLogicalOffset to the end of the file, is to be sent along the channel. If it is not zero, only the length specified is sent.

If an error occurs (for example, if the sample is unable to open a file), MQCXP.ExitResponse is set to MQXCC_SUPPRESS_FUNCTION so that the message being processed is put to the dead-letter queue instead of continuing to the destination queue. A feedback code is returned in MQCXP.Feedback and returned to the application that put the message in the Feedback field of the message descriptor of a report message. This is because the putting application requested exception reports by setting MQRO_EXCEPTION in the Report field of the MQMD.

If the encoding or CodedCharacterSetId (CCSID) of the Reference Message is different from that of the queue manager, the Reference Message is converted to the local encoding and CCSID. In our sample, amqsprm, the format of the object is MQFMT_STRING, so amqsxrm converts the object data to the local CCSID at the receiving end before the data is written to the file.

The format of the file being transferred should not be specified as MQFMT_STRING if the file contains multibyte characters (for example, DBCS or Unicode). This is because a multibyte character could be split when the file is segmented at the sending end. To transfer and convert such a file, the format should be specified as something other than MQFMT_STRING so that the Reference Message exit does not convert it and the file should be converted at the receiving end when the transfer is complete.

Compiling the Reference Message Exit sample

To compile amqsxrma, use the following commands:

Note:: Throughout this section the "\" character is used to split long commands over more than one line. Do not enter this character, enter each command as a single line.

On AIX

  $xlc_r -c -I/usr/mqm/inc amqsxrma.c
  $ld -o amqsxrm amqsxrma.o -H512 -T512 -e MQStart \
    -bM:SRE -bE:amqsxrm.exp -lc_r -ls_r -lmqm_r -lpthreads

On Compaq Tru64 UNIX Version 4.0

In a non-threaded environment:

  $ cc -std1 -c -I /opt/mqm/inc amqsxrma.c
  $ cc -std1 -shared -o amqsxrma amqsxrma.o -L /opt/mqm/lib -lmqm -e MQStart -lc

In a threaded environment:

  $ cc -std1 -c -I /opt/mqm/inc amqsxrma.c
  $ cc -std1 -shared -pthread -o amqsxrma_r amqsxrma.o -L /opt/mqm/lib \
    -lmqm_r -e MQStart -lc

On Compaq Tru64 UNIX Version 5.0

  $ cc -std1 -c -I /opt/mqm/inc amqsxrma.c
  $ cc -std1 -shared -pthread -o amqsxrma amqsxrma.o -L /opt/mqm/lib \
    -lmqm -e MQStart -lc

On HP-UX

  $ cc -c -Aa +z -I/opt/mqm/inc amqsxrma.c
  $ ld -b -o amqsxrm amqsxrma.o -z +b : -lmqm -lc

On Linux

  $ gcc -c -I/opt/mqm/inc amqsxrma.c
  $ gcc -o amqsxrm amqsxrma.o -shared \
      -L/opt/mqm/lib -lmqm

On OS/400

To create the module use the following command:

 CRTCMOD MODULE(MYLIB/AMQSXRMA) SRCFILE(QMQMSAMP/QCSRC)
   TERASPACE(*YES *TSIFC)

Notes:

To create your module so that it uses the IFS file system add the option SYSIFCOPT(*IFSIO)

To create the program for use with nonthreaded channels use the following command: CRTPGM PGM(MYLIB/AMQSXRMA) BNDSRVPGM(QMQM/LIBMQM)

To create the program for use with threaded channels use the following command: CRTPGM PGM(MYLIB/AMQSXRMA) BNDSRVPGM(QMQM/LIBMQM_R)

On Solaris

  $ cc -c -KPIC -I/opt/mqm/inc amqsxrma.c
  $ ld -G -o amqsxrm amqsxrma.o -dy -lmqm -lc -lnsl -ldl

On Windows systems

  cl amqsxrma.c -o amqsxrm.dll -LD -DEFAULTLIB mqm.lib mqmvx.lib amqsxrm.def

Design of the Get Reference Message sample (amqsgrma.c, AMQSGRM4)

The program logic is as follows:

The sample is triggered and extracts the queue and queue manager names from the input trigger message.
It then connects to the specified queue manager using MQCONN and opens the specified queue using MQOPEN.
The sample issues MQGET with a wait interval of 15 seconds within a loop to get messages from the queue.
If a message is a Reference Message, the sample checks the existence of the file that has been transferred.
It then closes the queue and disconnects from the queue manager.