The goal of TIBCO Performance Benchmark is to understand the performance capabilities and limitations of a system. Every system has limitations, and TIBCO Performance Benchmark characterize the system in a way that you can understand these limitations.
Benchmarks are complicated if the system capabilities and limitations vary depending on the demands placed on the system. TIBCO Performance Benchmark also vary based on the resources that are available to the system, for example, CPU, memory, network bandwidth, and disk bandwidth. The set of benchmark measurements must be carefully designed so that the impact of the factors can be understood clearly .
Shape of the performance curve informs us a lot about the system under test. If each input results in single output, then over the normal operating range, the output rate will exactly match the input rate and within statistical bounds. If each input results in more than one output or you are measuring data rates instead of input and output rates, there may be a scale factor relating the two values. In any case, over the normal operating range there should be a linear relationship between the two rates – that is the curve is a straight line.
One of the initial fundamental requirements before performing any kind of tuning exercise is to carefully eliminate all external factors that can potentially affect any performance issues.
TIBCO Performance Analysis and Tuning is an iterative process consisting of following:
- Establish TIBCO Performance Benchmark marking criteria
- Review ActiveMatrix Business Works 6.x performance architecture
- Establish performance best practices guidelines
- Identify and review tuneable parameters for the use case
TIBCO Performance Benchmark Criteria
The initial step when measuring performance is to identify the Service Level Agreement. Performance targets are determined by the user response time and message processing requirements.
Examples of performance requirements include:
- Engine throughput or the number of messages processed per second
- Processing speed or the average process instance duration and latency
- Web response time. The response and request time
- Resource utilization
- Concurrent request, sleep time, registered and if applicable, concurrent users
Defining the minimum, desired, peak targets for each requirement helps identifying the type of data to collect and to evaluate the test results.
Adding to these normal load expectations, abnormal error-recovery scenarios under unusually high loads should also be considered. For example, the Active Matrix Business Works process might be receiving or polling messages from a TIBCO Enterprise Message Service queue, and the above targets reflect the normal flow of messages through the queue.
However, if communication to the TIBCO Enterprise Message Service server has been disrupted for an extended period of time, or if the Active Matrix Business Works Engine shuts down, a much high load may be experienced when communication is re-established or when the engine restarts.
The scenarios must be addressed when considering the engine performance under load, and to ensure that the throughput does not deteriorate below the target in such situations.
The Business requirements also control the decision to use reliable or certified messaging. Certified messaging has an impact on performance.
The input rate that marks the end of the linear region marks the operating capacity of the system. This might mark the true limits of the system design, or it may indicate that some type of resource limit has been hit. This might be the result of the available physical memory or the bandwidth available on the NIC card or the available CPU cycles getting exhausted. It is important to determine the nature of the limit, as this may indicate ways to alter the environment and increase capacity either by tuning the system or adding resources.
Beyond the operating capacity, further increase in input rate exceeds the capacity of the system to perform work. When this occurs, increasing the input rate will no longer produce the same level of increase in the output. Inputs increase faster than the system is producing output. When the input rate continues to increase it will reach a point where the output rate begins to decline. The system takes the resources away from completing work and applying them to accepting the inputs.
Collecting Performance Data
Begin by creating a set of processes for testing purposes. These can be actual processes that will be used in production or more basic processes that represent production scenarios. Granularity and the scope of your performance requirements should determine how processes are used for performance testing.
Configure the operating system tool to measure memory, disk, CPU usage during all tests. Identify the Activematrix Business Works metrics that can measure conformance with requirements. General strategy is beginning with summary metrics, and then progress to detailed metrics as areas for improvement are identified.
Specific performance goals has to be defined, so that you can tailor the data collection to provide only the required information. Understand where the process instance lifetime is spent and collect detailed process instance metrics while processing a few representative process instances. Calculate the total throughput, collect summary metrics while generating a typical number of incoming messages of typical size and frequency.
Conduct a separate test for minimum, desired, and peak target numbers. Wherever possible, try restricting other local network, operating system, and engine activities during this time. If the average metrics are used, restrict the test window to ensure that data collected is relevant.
Locus IT provides TIBCO Performance Management services including Performance Analysis and Tuning, Performance Management upgradation and support. We have the expertise to provide unique solutions to any requirements. For more information please contact us.