HCL Workload Automation, Version 9.4

The solution

About this task

Fine Cola decides that one important step toward improving their process execution is to adopt a solution based on automatic and dynamic workload scheduling. The solution is based on a choice that strengthens integration with business applications to run the following tasks:
  • Read data from one database to update other databases.
  • Read data from external applications, process it, and add it to the appropriate databases.
  • Provide the information necessary for the operation of every phase.
  • Trigger some of the phases when predetermined thresholds are reached.
  • Back up their data without interrupting production.
  • From a capacity management perspective, understands the size of an application and what resources it requires, models that against the existing resources and is able to predict and forecast the capacity that the new application needs as it is defined in the enterprise.
  • From an availability management perspective, use the resources available in the environment to support the application and understand out how to work to effectively schedule, monitor, and manage that application as it is submitted. Then if the resources are not available, interact with the change management and provisioning processes to dynamically allocate the necessary resources.
  • Have a business management process monitoring all the various policies and driving a consistent view of the policies for the application.
After analyzing the workload management products available on the market, Fine Cola has chosen to use HCL Workload Automation and specifically the dynamic domain manager to:
  • Optimize and automate the tasks to process their applications and dynamically adapt their processing in response to changes in the environment.
  • Plan, choreograph, and schedule required changes to applications to minimize the impact of changes on critical production workloads, and ensure that workload processes are updated to reflect changes throughout asset life cycles.
  • Minimize the total amount of time that is required to deliver the output of the task resolution processes.
  • Handle dependencies between tasks, data, and external applications so that the entire workload can be managed homogeneously in the same process flow.
  • Create a policy-based view of workflow automation, not just workload automation, but cross-enterprise workflow, and direct that workflow across the enterprise while planning, scheduling, managing, and monitoring all these things. Dynamically tuning the cross-enterprise capacity to support this dynamic view of workloads.
  • Automatically transfer entire workloads across multiple platforms, and update policies across multiple platforms.
  • Balance between the ability to provide sophisticated planning, testing, choreographing, monitoring, and adaptation of workload processes with fault tolerance and redundancy for high availability of the scheduling infrastructure, while minimizing server and network resource requirements.
  • Perfectly integrate with each other.
HCL Workload Automation operates at both a macro-level and micro-level to prepare work schedules and to preprocess work items where necessary so that the delivery resources can be matched to the demands of the flow of work in an optimal fashion.

The dynamic domain manager dynamically routes workload to the best available resources based on application requirements and business policies. Moreover it optimizes the IT computing resource use according to SLAs.

Fine Cola's applications are mapped to what in HCL Workload Automation terminology are units of work called jobs. Some of these jobs are statically allocated to dedicated resources to run (static job definition), others are dynamically allocated to physical or virtual resources according to the job importance, requirements, scheduling policies, and based on the environment resource characteristics, relationships, availability, load, and performance (dynamic job definition). They drive the resource allocation to meet the job SLA and the resource optimization.

Jobs that run as a unit (such as a weekly backup application), along with times, priorities, and other dependencies that determine the exact order of the jobs are grouped into job streams.

Fine Cola's job streams are collections of jobs that are grouped for organizational purposes. The jobs of any particular job stream are related because they:
  • Operate toward the completion of related tasks. For example, the jobs of Jobstream100 run tasks designed to convert incoming customer orders into operational data.
  • Might be dependent on each other. Some jobs might have to wait for the completion of predecessor jobs before they can start running. The jobs are usually laid out in a sequence where the outcome of a predecessor is fed to a successor job.
  • Share the same programs, applications, and databases.
  • Share the same time-frames within the planning period.
Using HCL Workload Automation, Fine Cola's business process is structured in the following way:
  1. At the start of each day, Jobstream100:
    1. Extracts the new incoming orders from the Customer Orders database.
    2. Checks an external application where a number of selected customers can place unforeseen orders. If there are orders, they are extracted and merged with the other data.
    3. Copies the consolidated orders into a separate database view.
    4. Sorts them by due delivery date and by quantity and makes a report.
  2. As soon as the report is available, Jobstream200 extracts the numbers from the report and compares them with relevant data in the Inventory database. The goal is to determine the production volume required in the next production cycle to satisfy the orders.
  3. Jobstream300 extracts the production volume data and updates the Production Volumes database with the quantities of each type of soda that is to be manufactured in the next cycle.
  4. Jobstream400 reads the data in the Production Volumes database and:
    1. Calculates the quantities of raw materials required to run the upcoming production cycle.
    2. Flags these quantities as allocated to next cycle in the Raw Materials database.
    3. Checks the quantities to see if they have reached the minimum stock levels and triggers orders to Fine Cola's raw material suppliers if necessary.
  5. Jobstream500 reads the report with upcoming due orders from the Customer Orders database and:
    1. Produces the transportation schedules and destinations.
    2. Updates the To Supply database.
    3. Sends the delivery schedules to the distribution centers.
  6. Jobstream600 reads the distribution center databases and:
    1. Extracts the orders that have been filled.
    2. Updates the Customer Orders database so that invoices can be prepared and sent.
  7. Jobstream700 makes a backup of every database.

Fine Cola sets up a long term plan that encompasses the entire workload, spanning job streams that run on a daily basis and job streams that have other reoccurrences. From the long term plan, a current plan is extracted at the beginning of every time unit. The time period of the current plan can be chosen to vary from some hours to several days. Fine Cola has chosen to set their current plan on a daily basis. At the start of every day a new daily plan is built by their workload scheduling software: data is taken from the long term plan and from the daily plan of the previous day to include any jobs that might not have completed.

The company must also ensure that during peek periods the jobs in the critical path are run in the required time frame. To ensure this they converted some jobs from static definition to dynamic definition to manage the extra orders using the dynamic domain manager. With the dynamic domain manager, the company can:
  • Manage the automatic discovery of available resources in the scheduling environment with their characteristics and relationships.
  • Assign to the job the appropriate resources for running based on the job requirements and on the administration polices.
  • Optimize the use of the resources by assigning to the job the required resources based on the SLA.
  • Manage and control the resource consumption and load.
  • Dispatch jobs to target resources that meet the requirements to run the job.

The HCL Workload Automation relational database contains the information related to the jobs, the job streams, the workstations where they run, and the time specifications that rule their operation. It also contains data used by the dynamic domain manager, such as information about the current IT environment, the resource real time performance, and load data. It also stores the job definitions and keeps track of resources assigned to each job.

In this way, Fine Cola's scheduling analyst can create and change any of these objects at any time and Fine Cola's IT administrator can dynamically assign the best set of resources to match allocation requests based on the defined policies, without any impact on the business.

The IT administrator can also ensure the correct concurrent or exclusive use of the resources across the different jobs according to resource characteristics. If the resource request cannot be immediately carried out, he can use dynamic scheduling to automatically queue the resource until changes in the resource utilization or in the environment lead to its fulfillment.

The workload scheduling plan can be changed as quickly and dynamically as the business and operational needs require. The scheduling analyst makes full use of the trial and forecast planning options available in the scheduler to adjust and optimize workload scheduling and, as a consequence, Fine Cola's line of operations.

To respond to any unexpected and unplanned-for demands, individual jobs can be added ad hoc to the scheduling plan at any time.

Moreover, the company can use dynamic scheduling to rapidly adapt to the increase of workload during peak periods driving the requirement for workload virtualization, that is the ability to manage and control the workload so that it can be slit, routed to appropriate resources and capacity, and dynamically moved around in logical resource pools.

If a resource is not available, the SLA defined continues to be met because the job processing is restarted from the point where the failure happens.