LUALoriotPro WEB site

LoriotPro scripting documentation
Extensions for LUA language

Table of contents Search function

Print the current page Mail this  link UTUBE Channel version française

Global Object - Introduction & Concepts

Introduction

An innovative functionality was added to the monitoring software LoriotPro in June 2012. This greatly improves the performance of SNMP collection and increases the refresh speed of the display and this even with important volumes of data to collect. The number of collection can grow, collection processing times can increase, the capacity and speed of processing and display will remain optimal. Parallel processing (multithreading) and the many tasks performed enables scalability by adding new process of collection and possibly more CPU in the system.

Real Time Monitoring with LoriotPro WAN simulation and Global object in LoriotPro

With this internal new architecture of LoriotPro, hundreds of SNMP collects can be considered with periods of the order of seconds, and the visuals (Active View) can reach refresh frequencies of this level. It is now possible to have extremely dynamic visuals (Active View) that can alert you in a blink of an eye of a malfunction in your network or broadcast playout.

The technical concept

As a reminder, the data collection equipment on the system to monitor, mainly operates through SNMP. This protocol allows the retrieval of status indicators and performance via SNMP agents present on the equipment and systems. Response times to queries SNMP agents are quite unpredictable, so you cannot really predict how long a SNMP query for LoriotPro will get a response. Each collection process has a maximum limit of waiting (called timeout) beyond which it considers that the agent is not responding. If a single process is responsible for the collection so that it performs the sequential, performance may not be the appointment, knowing that a hundred collections can take several seconds to several minutes.

Let's summarize  the context: We know that what we call the collections "tasks" have very random execution times ranging from a few milliseconds to several seconds. Furthermore we wish to achieve the collection periodically and on tight intervals (polling period), the order of seconds for some performance indicators.

Implementation principle: All tasks (collection) to be performed are realized by simple LUA script programs. This one details for each task, the type of collect to achieve (GET SNMP MIB object). Generally we use SNMP objects but other types of collection can be made from log files extraction, from SQL databases queries, TRAP meter reading, etc.. Note that the collects may come from global Global Objects already in memory, which allows to treat by correlation.

To accomplish these tasks, a variable number of Audit processes can be launched. In principle, higher is the number of tasks to perform and higher is their repetition frequency, higher the number of Audit processes must be. The processes in question are instantiated as needed (Audit Process Plugin LoriotPro) to assume a quasi-parallel processing.

To simplify the setting, a standard audit process (902) is provided. One or more can be reponsible for the processing of the GLOBAL OBJECTS that are defined in a group of GLOBAL OBJECTS

Audit Process for a Group of Objects

Here's a simplified example with two Audit processes responsible of the execution of three Global Object LUA function (collection). Collections are made ​​at different time intervals, collection durations are also supposed to be variable. Both processes (audit process) support collections based on their availability. Once they have finished their job, they go through the list of Global Object in this Group and attributes to which the first polling period (polling) has expired and who is not already assigned.

Example with 3 GLOBAL OBJECTS in the same group and two Audit process.

Concept of Global Object of LoriotPro

This example shows the ratios between polling period and duration of disproportionate treatment. Usually the ratio thereof is from 1 to about 100. In the event that an SNMP agent is working properly, collection is of the order of several tens of milliseconds and the polling intervals between 1 and 15 seconds. Some delay (jitter) in treatment may occur if the collection duration increases or if the number of collections increases.

Ideally it is necessary that the sum of the ratios between the execution time and the polling period of all collections is less than the number of processes available for processing.

The values ​​of the collection are then stored in a block of GLOBAL OBJECTS directly in memory. These Global Objects are accessible everywhere within LoriotPro especially in a visual Active View.

The components of the architecture

The components of the architecture are presented in the diagram below:

Basic concept of Global Object

On one side the equipment on which collections are made, on the other a visual which should reflect as quickly as possible the working statuts of this devices.

 The Audit Process are constantly looking for tasks to perform (LUA function to execute). The Global Object LUA function describes how to programmatically perform the collect on the device but not only (cf. correlation) stored in the corresponding GLOBAL OBJECTS.

The ActiveView for their part have their own data collection internal process that gather, based on a collection interval positioned on each graphical object, the current values of GLOBAL OBJECTS. Depending of the returned values of the Global Object, graphic objects are modified (background color, text, position, clipart, etc..) to notify the administrator of a state change.

Correlation of Global Objects

 We use the term "correlation" to identify a Global Object resulting from the treatment of several other Global Objects. There are several possible levels of correlation in the solution.

The correlation can be achieved by collecting several indicators sequentially, a set of SNMP Global Object for example, then by the processing of their values by some mathematical operations and finally by storing the result in another global Global Object.

The other option consists of taking existing GLOBAL OBJECTS, correlating them and to storing the result in another global Global Object  (Example below).

Correlation of GLOBAL OBJECTS

The combination of these two options is also possible.

The correlation process is a task similar to the task of collecting and is inscribed in the Job List. Unlike a collection, treatment is extremely fast because it manipulates Global Objects in memory and does not include network access time or treatment by remote devices.

Structure of the collection processes

The collection program is multi-instance capable because it can be launched simultaneously by separate processes (multi-task). To avoid any contention during update time and guaranty that a Global Object is manipulated at a given time by a single Audit process, a lock (semaphore) is assigned to it. The simplified logical diagram given hereafter follows the logic of the code. At first, GLOBAL OBJECTS manipulated are checked one by one. If they do not exist, they can be initialized with default values.

Then for each Global Object the audit process checks if a lock is present, if this is the case it goes to the next Global Object in the list if it sets a lock on it. The lock prohibits the change of the Global Object by another process. It then checks the polling interval for this Global Object, if it is expired it call the Global Object LUA function that will collect the SNMP data on the device (if SNMP). The Global Object values are updated and the lock released.

The audit process that scans the list of jobs, monopolizes all jobs whose Global Objects are not locked.

Audit Algo GLOBAL OBJECTS

Global Global Objects are structures containing multiple information.

 Here under are the details of this structure

Name

A name that must be unique within the list and easily identifies the Global Object.

String

The value of the Global Object when it is a string

Double

The value of the Global Object when it is numeric

Interval

The time interval required between each Global Object LUA function execution

Time

The date and time when the last collection was done

Clock

The timestamp of the last made ​​collection

Param1

Parameter used by the collection process, such as an SNMP object, an IP address etc ...

Param2

Same as Param1, parameter used by the collection process

 



www.loriotpro.com