HCL Workload Automation, Version 9.4

File transfer jobs

A file transfer job runs programs to transfer files to and from a server reachable using FTP, SSH, or other protocols

A description of the job properties and valid values are detailed in the context-sensitive help in the Dynamic Workload Console by clicking the question mark (?) icon in the top-right corner of the properties pane.
This section describes the required and optional attributes for file transfer jobs. Only single file transfers are allowed. Each job definition has the following format and arguments:
Table 1. Required and optional attributes for the definition of a file transfer job
Attribute Description and value Required
application name filetransfer ✓
File transfer type (upload or download) Specifies whether the file transfer is an upload or a download.

Enclose all the file transfer attributes between jsdlfiletransfer:uploadInfo or jsdlfiletransfer:downloadInfo tags as shown in the example.

 ✓
Permissions (Octal Notation) If downloading, specify file permissions for the user on the local system. File permissions are expressed as octal notation.  
Delete source files after transfer If uploading, specify if source files must be deleted after transfer.  
server The address of the file transfer server (and the port number, if other than the standard port, when you choose FTP as protocol). ✓
localfile and remotefile
  • If uploading, localfile is the fully qualified path and name of the file to be uploaded, while remotefile is the fully qualified path and name of the file to be created on the remote target.

    You can use asterisks (*) or question marks (?) as wildcard characters in localfile.

  • If downloading, localfile is the fully qualified path and name of the file to be created on the local target, while remotefile is the fully qualified path and name of the file to be downloaded.

    You can use asterisks (*) or question marks (?) as wildcard characters in remotefile.

 ✓
localCredentials and remoteCredentials The names and passwords of the authorized users on the local and remote systems. As an alternative to hard-coding actual values, you can parametrize in one of the following ways:
  • Enter a username specified in the database with the user definition (it is applicable to all operating systems on this job type) and key the statement:
    <jsdl:password>${password:username}</jsdl:password>
    The password is retrieved from the username user definition in the database and resolved at runtime. See Using user definitions on job types with advanced options for further details.
    You can also specify the user of a different workstation and use the following syntax for the password:
    <jsdl:password>${password:workstation#username}</jsdl:password>
  • Enter a user and password defined with the param utility command locally on the dynamic agent that will run the job (if the job is to be submitted to a pool or to a dynamic pool, the definition must be present on all the agents of the pool). Provided you defined the user name with the variable user and a password, the corresponding credential statements would be:
    <jsdl:userName>${agent:user}</jsdl:userName>
<jsdl:password>${agent:password.user}</jsdl:password>
    The user and password variables are resolved on the agent at runtime. See Defining variables and passwords for local resolution on dynamic agents for further details.
Tip: The password of the remote user is not required if a keystore file path and password are specified when using the SSH protocol.
 ✓
keystore file path The fully qualified path of the keystore file containing the private key used to make the connection. A keystore is a database of keys. Private keys in a keystore have a certificate chain associated with them which authenticates the corresponding public key on the remote server. A keystore also contains certificates from trusted entities. Applicable to SSH protocol only.  
keystore password The password that protects the private key and is required to make the connection. This attribute is required only if you specify a keystore file path. If the keystore file path and keystore password combination fail to make a connection, then an attempt is made using the userName and password that correspond to the user authorized to start a connection on the remote computer. ✓
protocol Can be:
WINDOWS
The Microsoft file sharing protocol. If you do not specify a protocol, WINDOWS is assumed. Specify the shared directory in the remotefile keyword, without specifying any paths where the shared directory is nested. Specify the address of the workstation hosting the shared directory in the server keyword.

The default working directory is: C:\Program Files\HCL\TWA\TWS\ITA.

SSH
A network protocol that provides file access, file transfer, and file management functions over any reliable data stream. If you do not specify a protocol, SSH is assumed as default. To access remote Windows workstations, install Cygwin or MKS on each target Windows workstation.
FTP
A standard network protocol used to exchange files over a TCP/IP-based network, such as the Internet.
FTPS
An extension to the File Transfer Protocol (FTP) that adds support for the Tranport Layer Security (TLS) cryptographic protocol. Specifically, the file transfer is performed using implicitly the TLS security protocol for the FTP sessions, providing a private security level for the data connection. TLS protocol version 1 is supported. The SSL session reuse configuration is not supported.
FTPES
An extension to the File Transfer Protocol (FTP) that adds support for the Tranport Layer Security (TLS) cryptographic protocol. Specifically, the file transfer is performed using explicitly the TLS security protocol for the FTP sessions, providing a private security level for the data connection. TLS protocol version 1 is supported. The SSL session reuse configuration is not supported.
AUTO
The protocol is selected automatically between the WINDOWS and SSH protocols. The product tries using the WINDOWS protocol first. If this protocol fails, the SSH protocol is used. When using SSH, the path has to be in the SSH format. In this case the Cygwin ssh server is mounted on /home/Administrator.
  
transferMode Can be binary (the default) or ascii.  
remoteCodepage The codepage used on the remote workstation.
If you want to use a custom code page, define the remoteCodepage parameter as follows:
<jsdlfiletransfer:remoteCodepage>USER:CUSTOM_CP
</jsdlfiletransfer:remoteCodepage>
where CUSTOM_CP is the code page defined by the user.
For example, to use the tcpip.ftpd.ftpxlbin.frence3 custom code page, define the remoteCodepage parameter as follows:
<jsdlfiletransfer:remoteCodepage>USER:tcpip.ftpd.ftpxlbin.frence3
</jsdlfiletransfer:remoteCodepage>
 Required if you specify localCodepage
localCodepage The codepage used on the local workstation. Required if you specify remoteCodepage
Timeout Specifies the number of seconds to be used for the file transfer operation. The default value is 60 seconds.  
portsRange When the active mode is enabled, the portsRange section restricts the port numbers sent by the FTP PORT command. This option accommodates highly restrictive firewall rules. The portsRange section defines the port range to use on the client side of TCP data connections. If you do not specify the portsRange section, the operating system determines the port numbers to be used.
The portsRange parameter requires the following parameters:
min (min_port)
The minimum port value to use on the client side of TCP data connections. The values allowed range from 0 to 65535. For example, if you set this value to 1035, HCL Workload Automation restricts the port numbers to be greater than or equal to port 1035
max (max_port)
The maximum port value to use on the client side of TCP data connections. The values allowed range from 0 to 65535. For example, if you set this value to 1038, HCL Workload Automation restricts the port number to be less than or equal to port 1038.
For example, to limit the port range to use on the client side between port 1035 and port 1040, specify the following:
<jsdlfiletransfer:portsRange>
<jsdlfiletransfer:min>1035</jsdlfiletransfer:min>
<jsdlfiletransfer:max>1040</jsdlfiletransfer:max>
</jsdlfiletransfer:portsRange>
  
passiveMode Specifies whether the server is passive or active in establishing connections for data transfers.

If you set this option to NO, the server establishes the data connection with the client (active mode).

If you set this option to YES, the client establishes the data connection with the server (passive mode).

The default value is NO.		  
The following xml file shows the JSDL “application” section of a sample job definition for a file transfer job type: 
<jsdl:application name="filetransfer">
<jsdlfiletransfer:filetransfer>
<jsdlfiletransfer:downloadInfo>
<jsdlfiletransfer:server>FTP_SERVER</jsdlfiletransfer:server>
<jsdlfiletransfer:localfile>LOCAL_FILE</jsdlfiletransfer:localfile>
<jsdlfiletransfer:remotefile>REMOTE_FILE</jsdlfiletransfer:remotefile>
<jsdlfiletransfer:remoteCredentials>
<jsdl:userName>USERNAME</jsdl:userName>
<jsdl:password>PASSWORD</jsdl:password>
</jsdlfiletransfer:remoteCredentials>
<jsdlfiletransfer:protocol>PROTOCOL</jsdlfiletransfer:protocol>
<jsdlfiletransfer:transferMode>ASCII_BINARY</jsdlfiletransfer:transferMode>
<jsdlfiletransfer:codepageConversion>
<jsdlfiletransfer:remoteCodepage>RM_CP</jsdlfiletransfer:remoteCodepage>
<jsdlfiletransfer:localCodepage>LC_CP</jsdlfiletransfer:localCodepage>
</jsdlfiletransfer:codepageConversion>
<jsdlfiletransfer:timeout>CONNECTION_TIMEOUT</jsdlfiletransfer:timeout>
<jsdlfiletransfer:portsRange>
<jsdlfiletransfer:min>MIN_PORT</jsdlfiletransfer:min>
<jsdlfiletransfer:max>MAX_PORT</jsdlfiletransfer:max>
</jsdlfiletransfer:portsRange>
<jsdlfiletransfer:passiveMode>YES_NO</jsdlfiletransfer:passiveMode>
</jsdlfiletransfer:downloadInfo>
</jsdlfiletransfer:filetransfer>
</jsdl:application>
The following example shows a generalized task that downloads a file from a remote workstation with address 10.0.0.8:
$JOBS
AGENT#FILE_TRANSFER
TASK
<?xml version="1.0" encoding="UTF-8"?>
<jsdl:jobDefinition xmlns:jsdl="http://www.ibm.com/xmlns/prod/scheduling/1.0/jsdl"
 xmlns:jsdlfiletransfer="http://www.ibm.com/xmlns/prod/scheduling/1.0/jsdlfiletransfer" 
name="FILETRANSFER">
<jsdl:application name="filetransfer">
<jsdlfiletransfer:filetransfer>
<jsdlfiletransfer:downloadInfo>
<jsdlfiletransfer:server>10.0.0.8</jsdlfiletransfer:server>
<jsdlfiletransfer:localfile>c:\MyTextFile.txt</jsdlfiletransfer:localfile>
<jsdlfiletransfer:remotefile>./MyRemoteFile.txt</jsdlfiletransfer:remotefile>
<jsdlfiletransfer:localCredentials>
<jsdl:userName>${agent:locluser}</jsdl:userName>
<jsdl:password>${agent:password.${agent:locluser}}</jsdl:password>1
</jsdlfiletransfer:localCredentials>
<jsdlfiletransfer:remoteCredentials>
<jsdl:userName>remuser</jsdl:userName>
<jsdl:password>${password:remuser}</jsdl:password>2
</jsdlfiletransfer:remoteCredentials>
<jsdlfiletransfer:protocol>FTP</jsdlfiletransfer:protocol>
<jsdlfiletransfer:transferMode>ascii</jsdlfiletransfer:transferMode>
<jsdlfiletransfer:codepageConversion>
<jsdlfiletransfer:remoteCodepage>IBM-280</jsdlfiletransfer:remoteCodepage>
<jsdlfiletransfer:localCodepage>ISO8859-1</jsdlfiletransfer:localCodepage>
</jsdlfiletransfer:codepageConversion>
</jsdlfiletransfer:downloadInfo>
</jsdlfiletransfer:filetransfer>
</jsdl:application>
</jsdl:jobDefinition>
Note: in the credentials sections,
  1. The local user name was defined on the agent that runs the job with a variable named locluser through the param utility command. So, the value defined for locluser will be retrieved at runtime from the variables file located in the agent. Likewise, the password for the value represented by locluser was defined on the agent with the param command and will be resolved at runtime from the local variables file.
  2. The remote user name was defined with the composer user command and is stored in the database together with its password as user name remuser. The password for remuser will be retrieved from the database at runtime.
The following example shows a job definition to be used to download a text file using the FTPES protocol with an active mode, a timeout of 10 seconds and a port range to use between 1035 and 1038:
<?xml version="1.0" encoding="UTF-8"?>
<jsdl:jobDefinition xmlns:jsdl="http://www.ibm.com/xmlns/prod/scheduling/1.0/jsdl" 
xmlns:jsdlfiletransfer="http://www.ibm.com/xmlns/prod/scheduling/1.0/
jsdlfiletransfer" name="FTPES_DOWNLOAD_TEXT">
<jsdl:application name="filetransfer">
<jsdlfiletransfer:filetransfer> 
<jsdlfiletransfer:downloadInfo> 
<jsdlfiletransfer:server>myServerFtp</jsdlfiletransfer:server>
<jsdlfiletransfer:localfile>d:\MyLocalFile.txt</jsdlfiletransfer:localfile>
<jsdlfiletransfer:remotefile>/tmp/MyRemoteFile.txt</jsdlfiletransfer:remotefile>
<jsdlfiletransfer:remoteCredentials> 
<jsdl:userName>myUser</jsdl:userName> 
<jsdl:password>myPassword</jsdl:password> 
</jsdlfiletransfer:remoteCredentials> 
<jsdlfiletransfer:protocol>FTPES</jsdlfiletransfer:protocol>
<jsdlfiletransfer:transferMode>ascii</jsdlfiletransfer:transferMode>
<jsdlfiletransfer:timeout>10</jsdlfiletransfer:timeout>
<jsdlfiletransfer:portsRange>
<jsdlfiletransfer:min>1035</jsdlfiletransfer:min> 
<jsdlfiletransfer:max>1038</jsdlfiletransfer:max> 
</jsdlfiletransfer:portsRange>
</jsdlfiletransfer:downloadInfo> 
</jsdlfiletransfer:filetransfer>
</jsdl:application>
</jsdl:jobDefinition>
The following example shows a job definition to be used to transfer a text file using the SSH protocol:
<?xml version="1.0" encoding="UTF-8"?>
<jsdl:jobDefinition xmlns:jsdl="http://www.ibm.com/xmlns/prod/scheduling/1.0/jsdl" 
xmlns:jsdlfiletransfer="http://www.ibm.com/xmlns/prod/scheduling/1.0/
jsdlfiletransfer" name="SSH_UPLOAD">
<jsdl:application name="filetransfer">
<jsdlfiletransfer:filetransfer> 
<jsdlfiletransfer:uploadInfo> 
<jsdlfiletransfer:server>myServer</jsdlfiletransfer:server>
<jsdlfiletransfer:localfile>d:\MyLocalFile.txt</jsdlfiletransfer:localfile>
<jsdlfiletransfer:remotefile>/tmp/MyRemoteFile.txt</jsdlfiletransfer:remotefile>
<jsdlfiletransfer:remoteCredentials> 
<jsdl:userName>myUser</jsdl:userName> 
<jsdl:password>myPassword</jsdl:password> 
</jsdlfiletransfer:remoteCredentials> 
<jsdlfiletransfer:protocol>SSH</jsdlfiletransfer:protocol>
<jsdlfiletransfer:transferMode>ascii</jsdlfiletransfer:transferMode>
</jsdlfiletransfer:uploadInfo> 
</jsdlfiletransfer:filetransfer>
</jsdl:application>
</jsdl:jobDefinition>
The following example shows a job definition to be used to transfer a text file using the WINDOWS protocol:
<?xml version="1.0" encoding="UTF-8"?>
<jsdl:jobDefinition xmlns:jsdl="http://www.ibm.com/xmlns/prod/scheduling/1.0/jsdl" 
xmlns:jsdlfiletransfer="http://www.ibm.com/xmlns/prod/scheduling/1.0/
jsdlfiletransfer" name="WINDOWS_DOWNLOAD">
<jsdl:application name="filetransfer">
<jsdlfiletransfer:filetransfer> 
<jsdlfiletransfer:downloadInfo> 
<jsdlfiletransfer:server>myServer</jsdlfiletransfer:server>
<jsdlfiletransfer:localfile>d:\MyLocalFile.txt</jsdlfiletransfer:localfile>
<jsdlfiletransfer:remotefile>mySharedFolder\MyRemoteFile.txt
</jsdlfiletransfer:remotefile>
<jsdlfiletransfer:remoteCredentials> 
<jsdl:userName>myUser</jsdl:userName> 
<jsdl:password>myPassword</jsdl:password> 
</jsdlfiletransfer:remoteCredentials> 
<jsdlfiletransfer:protocol>WINDOWS</jsdlfiletransfer:protocol>
<jsdlfiletransfer:transferMode>ascii</jsdlfiletransfer:transferMode>
</jsdlfiletransfer: downloadInfo > 
</jsdlfiletransfer:filetransfer>
</jsdl:application>
</jsdl:jobDefinition>
The file transfer job type provides the following return codes. Return codes are available only for completed jobs.
RC=0
File transfer completed successfully.
RC=-1
File transfer not performed. The job failed with the following error code: 
AWKFTE007E
Explanation: An error occurred during the file transfer operation.

Possible reasons: Remote file not found or permission denied.

RC=-2
File transfer not performed. The job failed with the following error code: 
AWKFTE020E
Explanation: Only for SSH or WINDOWS protocols. An error was returned while attempting to convert the code page.

Possible reasons: For SSH or WINDOWS protocols, the code page is automatically detected and converted. In this case, there is an error in the code page of the file to be transferred, which is not compliant with the code page of the local system.

RC=-3
File transfer not performed. The job failed with the following error code: 
WKFTE015E
Explanation: An error occurred during the file transfer operation.

Possible reasons: Local file not found.

RC=-4
File transfer performed with the default code page. The job failed with the following error code: 
AWKFTE023E
Explanation: The specified code page conversion was not performed. The file transfer was performed with default code pages.

Possible reason: The specified code page is unavailable.

Scheduling and stopping a job in HCL Workload Automation

You schedule HCL Workload Automation file transfer jobs by defining them in job streams. Add the job to a job stream with all the necessary scheduling arguments and submit it.

You can submit jobs using the Dynamic Workload Console, Application Lab, or the conman command line.

After the submission, if the job that is running contains more files to transfer and the job is reported in EXEC status in HCL Workload Automation, you can stop it if necessary by using the kill command.

Restarting the job

If the HCL Workload Automation agent stops when you submit the file transfer job, or while the job is running, the job restarts automatically as soon as the agent restarts.

Job properties

You can see the job properties by running conman sj <File transfer_job_name>;props, where<File transfer_job_name> is the file transfer job name.

For a file transfer job that, for instance, transfers four files, in the Extra Information section of the output command, you see the following properties:
Extra Information
File.0.LocalFile = ./MyFile3.log
File.0.LocalUser = agtParis
File.0.Protocol = FTP
File.0.RemoteFile = /export/home/FileTransferRep/MyFile3.log
File.0.RemoteUser = ftpuser
File.0.Size = 983
File.0.TotalTransferTime = 0
File.1.LocalFile = ./MyFile.log
File.1.LocalUser = agtParis
File.1.Protocol = FTP
File.1.RemoteFile = /export/home/FileTransferRep/MyFile.log
File.1.RemoteUser = ftpuser
File.1.Size = 983
File.1.TotalTransferTime = 0
File.2.LocalFile = ./MyFile2.log
File.2.LocalUser = agtParis
File.2.Protocol = FTP
File.2.RemoteFile = /export/home/FileTransferRep/MyFile2.log
File.2.RemoteUser = ftpuser
File.2.Size = 983
File.2.TotalTransferTime = 0
File.3.LocalFile = ./MyFile1.log
File.3.LocalUser = agtParis
File.3.Protocol = FTP
File.3.RemoteFile = /export/home/FileTransferRep/MyFile1.log
File.3.RemoteUser = ftpuser
File.3.Size = 983
File.3.TotalTransferTime = 0
Number of transferred files = 4
where, for each trasferred file, you have:
File.counter.LocalFile
For the file number #, the name of the local file that you want to transfer.
File.counter.LocalUser
For the file number #, the user name for accessing the local workstation.
File.counter.Protocol
For the file number #, the protocol to be used for the file transfer.
File.counter.RemoteFile
For the file number #, the name of the remote file that you want to transfer.
File.counter.RemoteUser
For the file number #, the user name for accessing the remote workstation.
File.counter.Size
For the file number #, the size of the transferred file.
File.counter.TotalTransferTime
For the file number #, the total transfer time, in seconds.
Number of transferred files
The number of the files transferred with the job. You have only one value of this property for each job.

You can export the file transfer job properties that you can see in the Extra Information section, to a successive job in the same job stream instance.

For more information about the list of job properties that you can export, see Table 9.

See also

From the Dynamic Workload Console you can perform the same task as described in

Creating job definitions.

For more information about how to create and edit scheduling objects, see

Designing your Workload.