Audience

The intended users of this document are the users with administrator privileges to the Workflow system.

Dependencies

The Workflow system is mostly built in Java 1.5.x and Perl 5.6.x , hence they are required to use this system. It requires log4j and log4Perl libraries to support cross-language logging services, Castor, Xml-rpc nd JDOM libraries to support XML operations and the JUnit libraries for validating the installation.

The workflow distribution comes with all the required libraries except the perl related modules. Following is the list of perl modules that are required by the Workflow system. These modules can be found and installed from www.cpan.org.

The Workflow system also depends on two other scripts epilog and prolog, which come with distribution and exist under the bin directory.  These scripts have to installed on all the queues of the SGE, which may execute jobs that are submitted through the Workflow system.  The prolog and epilog entries in the SGE queue configuration shoud look like below:
prolog:     <Path>/prolog $host $job_owner $job_id $job_name $queue
epilog: <Path>/epilog $host $job_owner $job_id $job_name $queue

Deployment

The software for the Workflow system is distributed through a compressed tar ball called wf-3.0.tar.gz . This tar ball includes all the required third party libraries except JRE and Perl libraries. Refer to the Dependencies section for the details on JRE and Perl dependencies.

To deploy the workflow system from the tar ball:

Testing

The Workflow distribution comes with a tool called 'RunTestSuite' validates the newly deployed workflow system. RunTestSuite runs all the necessary tests against the installed Workflow system.

Summary of Command Line Options for RunTestSuite

Param Brief Description Required Default Valid values
mode Specifies the interface to run the tests Required   swing, text, awt
log Log level Optional 3, Info 0,1,2,3,4,5,6
logfile Log file to be used Optional stdout  
delete A true value for this parameter indicates that data that is created during the tests needs to be deleted. A false value indicates that the data generated need not to be deleted. Optional true true, false
truncate

A true value for this parameter indicates that data that the tables that were created or updated during the tests need to be truncated at the end of the tests. A false value indicates that the same database tables do not need to be truncated the end of the tests.

Optional true true, false
help Prints a brief help message Optional    


Details

mode: This parameter sets the interface to run the tests. The valid modes are swing, awt and text. It is a required parameter.

log: This parameter is used to specify a log level. This is an optional parameter All log messages logged at levels at or below the specified
log level will be logged to the file. If the user does not specify the log level explicitly, a default log level of 3 is used. There are 6 different 
log levels that can be specified, see the logging section of the Workflow User Guide for details.

delete: This parameter indicates whether to delete the data created during the tests or not, at the end of the tests. 'By default, all the data created during the tests is deleted. It is an optional parameter.

truncate: This parameter indicates whether to truncate the tables that are created or updated during the tests or not, at the end of the tests. By default, all the tables that created/updated during the tests are truncated. It is an optional parameter.

logfile: This parameter is used to specify the name of the log file. If this parameter is not specified, program sends log messages to stdout
This is an optional parameter.

help : This option produces a short help / usage message before exiting the program.

Usage and Examples

Usage:
RunTestSuite mode=<swing|awt|text> [log=<LogLevel (default 3)>] 
[delete=<true|false (default true)>]
[truncate=<true|false (default true)>]
[logfile=<log file name>] [--help]
Examples:
RunTestSuite mode=swing log=4 delete=false truncate=false logfile=test.log















Configuration / Properties files

Certain aspects of the Workflow system are configured by a number of configuration and properties files that are part of Workflow distribution. Following table lists these files with brief descriptions and the location of files.

File Brief Description location
workflow.config This is the main configuration file used by the Workflow system. It configures several different variables with right paths to the processor, dispatcher look up files and mapping files, sets the thread throttling limits and sets the database server information. Deployment directory.
log4j.properties The logger configuration file used by the Workflow system. Deployment directory.
log4perl.properties The logger configuration file used by the perl components of the Workflow system. Deployment directory.
command_proc_factory_lookup.prop A lookup up file used to locate the appropriate CommandProcessor name. properties directory under the deployment directory.
dispatcher_factory_lookup.prop A lookup file used to locate the appropriate CommandDispatcher name. properties directory under the deployment directory.
distributable_command_lookup.prop A lookup file used to check if a Command is a distributable command or not. properties directory under the deployment directory.
log4j_server.conf The log4j configuration file used by the htcservice. server-conf directory under the deployment directory.
sge_mockserver.conf Configuration file to support job submissions 'SGE' grid. server-conf directory under the deployment directory.









Thread Regulation

Each Command or CommandSet of a workflow in a Workflow engine is executed either by a CommandProcessor or a CommandDispatcher. Each CommandProcessor or CommandDispatcher is in turn run by a corresponding thread. When executing large workflows, creating and executing large number of concurrent threads may pose a risk of crashing the workflow engine or the host systems due to certain software / hardware limitations. To avoid this risk, the Workflow engine employs a mechanism to enforce a maximum limit on the number of concurrent Command Processors or Dispatchers that can run with in a workflow engine. It also restricts the number of concurrent DCE jobs that can be submitted to the grid by a Workflow engine.

These limits can only be changed by the workflow administrator by editing the file 'workflow.config'. The appropriate values for these limits may be determined by the administrator based on the type and the size of workflows that are executed using the Workflow engine.

Following are the various limits that are enforced in the current release:

Reporting Problems

Please address all problems, and concerns about workflow system by sending mail to antware@tigr.org.