HCL Workload Automation, Version 9.4

Managing external follows dependencies for jobs and job streams

During the creation of the preproduction plan, all external follows dependencies to job streams and jobs are resolved using four different possible matching criteria:
Same day
Considering the job or job stream instances planned to run on the same day. In this case you set the clause follows...sameday in the object definition. Figure 1 shows a job stream named Js1 which has an external follows dependency on the instance of the job stream Js2 that is scheduled to start on the same day.
Figure 1. Sameday matching criteria
The figure displays the predecessor job stream when using the sameday matching criteria
Below is an example of how to define the involved job streams.
schedule Js2 schedule Js1
on everyday on everyday
at 0700 at 1000
:job2 follows wk1#Js2 sameday
end :job1
  end
The job stream Js1 in not launched until the job stream instance of Js2 on the workstation wk1 completes successfully.
Closest preceding
Using the closest job or job stream instance (earlier or same time). The job or job stream instance that HCL Workload Automation uses to resolve the dependency is the closest in time before the instance that includes the dependency. In this case you set the follows ... previous clause in the object definition. Figure 2 shows a job stream named Js1 which has an external follows dependency on the closest earlier instance of job stream Js2. The time frame where the predecessor is searched is greyed out in the figure.
Figure 2. Closest preceding matching criteria
The figure displays the predecessor job stream when using the closest preceding matching criteria
Below is an example of how to define the involved job streams.
schedule Js2 schedule Js1
on Th on Fr
at 0700 at 1000
:job2 follows wk1#Js2 previous
end :job1
  end
The job stream Js1 in not launched until the closest preceding job stream instance of Js2 on the workstation wk1 completes successfully.
Within a relative interval
Considering the job or job stream instances defined in a range with an offset relative to the start time of the dependent job or job stream. For example, from 25 hours before the dependent job stream start time to 5 hours after the dependent job stream start time. In this case you set the follows ... relative from ... to ... clause in the object definition. Figure 3 shows a job stream named Js1 which has an external follows dependency on the job stream instance of Js2 that starts with an offset of 2 hours with respect to Js1. The job or job stream instance that HCL Workload Automation considers to resolve the dependency is the closest one within the relative time interval you chose.
Figure 3. Within a relative interval matching criteria
The figure displays the predecessor job stream when using the within a relative interval matching criteria
Below is an example of how to define the involved job streams.
schedule Js2 schedule Js1
on everyday on everyday
at 0900 at 1000
:job2 follows wk1#Js2 relative from 0200 to 0200
end :job1
  end
The job stream Js1 in not launched until the job stream instance of Js2 on the workstation wk1 that runs in the 08:00 to 12:00 time frame completes successfully.
Within an absolute interval
Using only the job or job stream instances defined in a range. For example from today at 6:00 a.m. to the day after tomorrow at 5:59 a.m. In this case you set the follows ... from ... to ... clause in the object definition. Figure 4 shows a job stream named Js2 which has an external follows dependency on the instance of job stream Js1 that is positioned in the preproduction plan between 7 a.m. and 9 a.m. The job or job stream instance that HCL Workload Automation considers to resolve the dependency is the closest one within the absolute time interval you chose. The time interval specifies the time of the day on which the interval starts and ends, either on the same day as the instance that include the dependency or on a day defined relative to that day.
Figure 4. Within an absolute interval matching criteria
The figure displays the predecessor job stream when using the within an absolute interval matching criteria
Below is an example of how to define the involved job streams.
schedule Js1 schedule Js2
on everyday on everyday
at 0800 at 1000
:job1 follows wk1#Js1 from 0700 to 0900
end :job2
  end
The job stream Js2 in not launched until the job stream instance of Js1 on the workstation wk1 that runs in the 07:00 to 09:00 time frame on the same day completes successfully.
Regardless of which matching criteria are used, if multiple instances of potential predecessor job streams exist in the specified time interval, the rule used by the product to identify the correct predecessor instance is the following:
  1. HCL Workload Automation searches for the closest instance that precedes the depending job or job stream start time. If such an instance exists, this is the predecessor instance.
  2. If there is no preceding instance, HCL Workload Automation considers the correct predecessor instance as the closest instance that starts after the depending job or job stream start time.
This behavior applies for external follows dependencies between job streams. For external follows dependencies of a job stream or job from another job the criteria are matched by considering the start time of the job stream hosting the predecessor job instead of the start time of the predecessor job itself. Figure 5 shows in bold the instances of job1 the successor job or job stream is dependent on.
Figure 5. Closest preceding predecessor job
The figure displays the closest preceding predecessor job

External follows dependencies are identified between jobs and job streams stored in the database whose instances are added to the preproduction plan when the preproduction plan is automatically created or extended. Job and job stream instances submitted in production from the conman command line are written in the preproduction plan but they are not used to recalculate predecessors of external follows dependencies already resolved in the preproduction plan.

The scheduler classifies follows dependencies as internal when they are specified only by their job name within the job stream. It classifies them as external when they are specified in the jobStreamName.workstationName.jobName format.

When a job stream includes a job with a follows dependency that shares the same job stream name (for example, job stream schedA includes a job named job6 that has a follows dependency on schedA.job2), the dependency is added to the plan as an external follows dependency. Since Version 8.3, unlike in previous versions, because the scheduler uses the sameday matching criteria to resolve external dependencies, dependencies originated in this way are never added the first time the object is submitted.

A job or job stream not yet included in the production plan, but that can be a potential predecessor of instances of jobs and job streams added to the production plan as the production plan is extended, is called a pending predecessor. A pending predecessor is like a dummy occurrence created by the planning process to honor a dependency that has been resolved in the preproduction plan, but that cannot be resolved in the current production plan because the predecessor's start time is not within the current production plan end time. Figure 6 shows how a pending predecessor and its successor are positioned in the preproduction plan.
Figure 6. Pending predecessor instance
The figure displays the pending predecessor instance
The way in which pending predecessors are managed is strictly linked to whether or not the successor job or job stream is carried forward:
  • If the successor is carried forward when the production plan is extended, the predecessor is included in the new production plan and the dependency becomes current. A pending predecessor job or job stream is marked with a [P] in the Dependencies column in the output of the conman showjobs and conman showschedules commands.
  • If the successor is not carried forward when the production plan is extended, the predecessor is included in the new production plan, but the dependency becomes orphaned. This can happen, for example, if, when extending the production plan, the successor is carried forward and the pending predecessor is not added to the plan because it was flagged as draft in the database. The orphaned dependencies are marked with a [O] in the Dependencies column in the output of the conman showjobs command. When dealing with an orphaned dependency you must verify if it can be released and, if so, cancel it.

Note that when an HCL Workload Automation network includes agents running on IBMIBM® Workload Scheduler versions older than 8.3 and the enLegacyId option is set to yes on the master domain manager, having multiple instances of a job stream as pending predecessors produces errors caused by identification problems at submission time.

External follows dependency resolution and status transition examples

This section includes examples for each of the four matching criteria described in the previous paragraphs. In all the examples, the start of day time (SOD) is set to 06:00 AM.

Same day
The job or job stream instance to be considered in resolving the dependency is the closest one on the same day in which the instance that includes the dependency is scheduled to run. In this example, two job streams, Js1 and Js2, each have one job. Job stream Js1 is scheduled to run every day at 08:00 and on Thursdays also at 07:00. Js1.Job1 runs at 09:00. Job stream Js2 has no time restrictions and is scheduled by default at the defined start of day time. Js2.Job2 is scheduled to run at 15:00 and has an external follows dependency on the closest earlier instance of the job stream Js1 running on the same day. The two job streams are defined in this way: 
SCHEDULE MY_MASTER#JS1
ON RUNCYCLE RULE1 "FREQ=WEEKLY;BYDAY=TH" 
(AT 0700)
ON RUNCYCLE RULE2 "FREQ=DAILY"
(AT 0800)
:
MY_MASTER#JOB1
AT 0900
END

SCHEDULE MY_MASTER#JS2
ON RUNCYCLE RULE2 "FREQ=DAILY;"
FOLLOWS MY_MASTER#JS1.@ SAMEDAY
:
MY_MASTER#JOB2
AT 1500
END
When the schedules are included in the plan, the sequence of graphics illustrate how the dependency is resolved:
  1. On Thursdays, the instance of Js2 scheduled at 06:00 depends on the instance of Js1 scheduled to run at 07:00. On any other day of the week, Js2 has a dependency on the instance of Js1 scheduled at 08:00. Figure 7 shows the status of the two job streams in the plan at 06:00 (SOD) on Thursday:
    Figure 7. Sameday matching criteria - Step 1: at Start of Day (SOD) on a Thursday
    The figure displays the status of the job streams for the sameday matching criteria at start of day time on Thursday
  2. At 09:00, Js1.job1 starts and Js1 changes status. Js2.job2 is held until its scheduled time.Figure 8 shows the status of the job streams in the plan at 09:00.
    Figure 8. Sameday matching criteria - Step 2: at 9:00
    The figure displays the status of the job streams at 09:00 on Thursday
  3. On Thursdays at 15:00, Js2 changes to ready status and Js2.job2 starts. Figure 9 shows the status of the two job streams in the plan at 15:00.
    Figure 9. Sameday matching criteria - Step 3: at 15:00
    The figure displays the job streams at 15:00 for the sameday matching criteria
Closest preceding
  • In this example, two job streams, Js1 and Js2, each have one job. The job in Js2 has an external follows dependency on the closest preceding instance of the job in Js1. The two job streams are defined in this way: 
    SCHEDULE MY_MASTER#JS1
ON RUNCYCLE RULE1 "FREQ=DAILY;"
(AT 0800)
ON RUNCYCLE RULE2 "FREQ=WEEKLY;BYDAY=TH,FR"
(AT 0900)
:
MY_MASTER#JOB1
END

SCHEDULE MY_MASTER#JS2
ON RUNCYCLE RULE1 "FREQ=DAILY;"
(AT 1200)
FOLLOWS MY_MASTER#JS1.@ PREVIOUS
:
MY_MASTER#JOB2
AT 1500
END
    Job stream Js1 runs every day at 0800 and on Thursdays and Fridays also at 0900. Job stream Js2 runs every day at 1200, and has an external dependency on the closest preceding instance of Js1. When the job streams are included in the plan, the sequence of graphics illustrates how the dependency is resolved:
    1. Before 12:00 on Thursdays and Fridays, there are two instances of Js1.Job1. Job stream Js2 has a dependency on the instance of Js1.Job1 that is scheduled to run at 09:00, because it is the closest preceding in terms of time. Figure 10 shows the status of the two job streams in the plan on Thursdays and Fridays.
      Figure 10. Closest preceding matching criteria - Step 1: before 08:00
      The figure displays the status of the job streams before 08:00 on Thursdays and Fridays for the closest preceding matching criteria
    2. On any other day of the week, the only instance of Js1.Job1 in plan, is the one scheduled to run at 08:00. In this case, Js2 has a dependency on this instance. When Job1 completes successfully, the status of Js2 becomes Ready. Figure 11 shows the status of the two job streams in the plan on any other weekday except Thursdays and Fridays.
      Figure 11. Closest preceding matching criteria - Step 2: at 08:00 on weekdays except Thursdays and Fridays
      The figure displays the status of the job streams at 8:00 on all weekdays except Thursdays and Fridays for the closest preceding matching criteria
    3. On Thursdays and Fridays at 09:00, the second instance of Js1.Job1 completes successfully. Job stream Js2 changes to Ready. Js2.Job2 is held until its scheduled start time. Figure 12 shows the status of the two job streams in the plan.
      Figure 12. Closest preceding matching criteria - Step 3: at 09:00 on Thursdays and Fridays
      The figure displays the status of the job streams at 9:00 on Thursdays and Fridays for the closest preceding matching criteria
    4. At 15:00 the time dependency of Js2.Job2 is satisfied and Job2 starts. Figure 13 shows the status of the two job streams in the plan at 15:00.
      Figure 13. Closest preceding matching criteria - Step 4: at 15:00 on every day
      The figure displays the status of the job streams at 15:00 on every day for the closest preceding matching criteria
    In the job stream definition, run cycle Rule1 can be substituted by the keywords ON EVERYDAY.
  • In this second example, the difference between the use of sameday and closest preceding matching criteria in a plan is described. Job stream Js1 runs every Friday at 0900, while job stream Js2 and Js3 run every Saturday at 0900. The three job streams are defined in this way:
    SCHEDULE ACCOUNTING#JS1
ON RUNCYCLE RULE1 "FREQ=WEEKLY;BYDAY=FR"
:
ACCOUNTING#JOB1
 AT 0900
END

SCHEDULE ACCOUNTING#JS2
ON RUNCYCLE RULE2 "FREQ=WEEKLY;BYDAY=SA"
FOLLOWS ACCOUNTING#JS1.@ PREVIOUS
:
ACCOUNTING#JOB1
 AT 0900
END

SCHEDULE ACCOUNTING#JS3
ON RUNCYCLE RULE2 "FREQ=WEEKLY;BYDAY=SA"
FOLLOWS ACCOUNTING#JS1.@
:
ACCOUNTING#JOB1
 AT 0900
END
    Job stream Js2 has an external dependency on the closest preceding instance of Js1, which is resolved as described in the previous example. Job stream Js3 is defined with sameday matching criteria, so it does not have any dependency on job stream Js1, because Js1 is not defined to run on the same day as Js2.
Within a relative interval

In this example, the job or job stream instance considered to resolve the dependency is the closest one in a time interval of your choice, which is defined relatively to the time when the instance that includes the dependency is scheduled to run. Job stream Js1 is scheduled to run every day at 15:00 and on Thursdays also at 08:00. Js2 is scheduled to run every day at 13:00 and on Thursdays also at 06:00, because no specific time is defined in the run cycle, it is scheduled at start of day time. Js2 uses the relative interval criteria (-04:00 to +04:00) to determine which instance is used to solve the dependency. The interval is based on the time the job stream enters the plan. The job streams are defined as follows:

SCHEDULE MY_MASTER#JS1
ON RUNCYCLE RULE1 "FREQ=WEEKLY;BYDAY=TH" 
(AT 0800)
ON RUNCYCLE RULE2 "FREQ=DAILY"
(AT 1500)
:
MY_MASTER#JOB1
END

SCHEDULE MY_MASTER#JS2
ON RUNCYCLE RULE3 "FREQ=WEEKLY;BYDAY=TH"
ON RUNCYCLE RULE2 "FREQ=DAILY;"
(AT 1300)
FOLLOWS MY_MASTER#JS1.@
RELATIVE FROM -0400 TO 0400
:
MY_MASTER#JOB2
AT 1300
END
At plan creation time, conman showjobs produces the following output:
%sj @#@
                                               (Est)  (Est)
CPU      Schedule SchedTime  Job      State Pr Start  Elapse RetCode Deps
MY_MASTER#JS1      0800 11/13 ******** READY 10       (00:06)
                             JOB1     HOLD  10       (00:06)
MY_MASTER#JS1      1500 11/13 ******** READY 10       (00:06)
                            JOB1     HOLD  10       (00:06)
MY_MASTER#JS2      0600 11/13 ******** HOLD  10              JS1(0800 11/13/09).@
                             JOB2     HOLD  10(13:00)
MY_MASTER#JS2      1300 11/13 ******** HOLD  10(13:00)       JS1(1500 11/13/09).@
                             JOB2     HOLD  10(13:00)
Figure 14 shows the status of the job streams in the plan at start of day on Thursday.
Figure 14. Relative Interval matching criteria - at start of day on Thursday
The figure displays the job streams at start of day on Thursday.
The instance of Js2 scheduled at 06:00 has a dependency on Js1.job1 which is scheduled at 08:00, within the relative interval based on the scheduled time (06:00). Js2.job2 depends on the instance of Js1.job1 within the relative interval based on the scheduled time (13:00). When the instance of Js1.job1 starts at 08:00, the status ofJs2 changes to Ready. From this point onwards, the sequence in which the job streams and jobs run follows the typical process.
Within an absolute interval
In this example, the job or job stream instance considered to resolve the dependency is the closest one in a fixed time interval of your choice. The time interval specifies the time of day on which the interval begins and the time of day on which it ends, either on the same day as the instance that includes the dependency, or on a day defined relatively to that date. Js1 is scheduled to run every day at 08:00 and on Thursdays also at 07:00. Job Js1.job1 is scheduled to run at 09:00. Job stream Js2 is scheduled every day at 10:00 and on Thursdays also at start of day (06:00) and has a dependency on Js1 based on the absolute interval occurring on the same day between 06:00 and 11:00. The job streams are defined as follows:
SCHEDULE MY_MASTER#JS1
ON RUNCYCLE RULE1 "FREQ=WEEKLY;BYDAY=TH" 
(AT 0700)
ON RUNCYCLE RULE2 "FREQ=DAILY"
(AT 0800)
:
MY_MASTER#JOB1
AT 0900
END

SCHEDULE MY_MASTER#JS2
ON RUNCYCLE RULE3 "FREQ=WEEKLY;BYDAY=TH"
ON RUNCYCLE RULE2 "FREQ=DAILY;"
(AT 1000)
FOLLOWS MY_MASTER#JS1.@ FROM 0600 TO 1100
:
MY_MASTER#JOB2
AT 1300
END
At plan creation time, conman showjobs produces the following output:
%sj @#@
                                               (Est)  (Est)
CPU      Schedule SchedTime  Job      State Pr Start  Elapse RetCode Deps
MY_MASTER#JS1      0700 11/13******** READY 10       (00:06)
                             JOB1     HOLD  10(09:00)(00:06)
MY_MASTER#JS1      0800 11/13 ******** READY 10       (00:06)
                             JOB1     HOLD  10(09:00)(00:06)
MY_MASTER#JS2      0600 11/13 ******** HOLD  10               JS1(0700 11/13/09).@
                             JOB2     HOLD  10(15:00)
MY_MASTER#JS2      1000 11/13 ******** HOLD  10(10:00)        JS1(0800 11/1309).@
                             JOB2     HOLD  10(15:00)
Figure 15 shows the status of the job streams in the plan at start of day on Thursday.
Figure 15. Absolute interval matching criteria - at start of day on Thursday
The figure displays the job streams using the absolute interval matching criteria at start of day on Thursday.
At 09:00, Js1.job1 starts, and at 10:00 the dependency is released and Js2 becomes ready. From this point onwards, the sequence is the same as described in the previous matching criteria.