At the end of the last century I had the opportunity to help in a very ambitious computer project: the search for radio messages emitted by extraterrestrial civilizations… And what the hell does it have to do with Distributed Systems?

Recently my colleagues wrote an interesting article on distributed network visibility, which I really liked and I came up with the idea of taking it to the next level. If this post tries to offer full knowledge of the different components in operation within our network, Distributed Systems go “further”; they reach where we lack control over the devices that comprise it.

I am going to exemplify both at the social science level, comparing a union versus a confederation (as a central of workers and unioI am going to exemplify both at the social science level, comparing a union versus a confederation (as a central of workers and unions and not from a political point of view).

*Confederacy

According to Merriam-Webster

1. A group of people, countries, organizations, etc. joined together for a common purpose or by a common interest: LEAGUE, ALLIANCE

Distributed computing, distributed systems, are they the same?

Distributed Systems

If you look for the concept of Distributed Systems on Wikipedia (that magical place), you will be redirected to the article called Distributed Computing and, I quote:

“Distributed computing also refers to the use of distributed systems to solve computational problems. In distributed computing, a problem is divided into many tasks, each of which is solved by one or more computers, which communicate with each other via message passing.”

Without going any further: Wikipedia, if we consider ourselves as computers, it is a very high-level Distributed System, since we comply with its intrinsic characteristics… And what are they?

Features of Distributed Systems

A Distributed System (or Distributed Computing) has:

•   Concurrence: Which in the case of computers is a distributed program and in Wikipedia they are people… who use specialized software distributed by web browsers.

•   Asynchronous: Each computer (or Wikipedian) works independently without waiting for a result from the other, when it finishes its batch of work, it delivers it and it is taken in and saved.

•   Resilience: A computer device that breaks down or loses connection, or a person who dies, withdraws or is expelled from Wikipedia, in both environments does not mean stopping the work or global task. There will always be new resources, machines or humans, ready to join the Distributed System.

The aliens

Right, I started this article talking about them. In today’s -unfortunately- destroyed radio telescope in Arecibo, Puerto Rico, astronomers Carl Sagan and Frank Drake sent a message to the Hercules cluster, a group of galaxies 25,000 light years away from our planet.

“Hercules Globular Cluster (https://commons.wikimedia.org/wiki/File:Hercules_Globular_Cluster,_EVscope-20211008.jpg) ”

That means that it will take 50 thousand years to get an answer, if there is life out there, but what if it is us who were already sent messages thousands or millions of years ago?

Well, this was the program Seti@home  about: it collected radio signals and chopped them into two-minute pieces that were sent to each person who wanted to collaborate in the analysis with their own computer. At the end of the calculation according to a special algorithm, the result was sent and a new piece of code was requested. If a computer after a reasonable time did not return an answer, then the same piece was sent to another computer that wanted to collaborate: the “prize” consisted in publicly recognizing the collaborator as a discoverer of life and intelligence outside this world.

I installed the program and put it as a screensaver, so I calculated while I was working on something else or resting.

“Seti@home (imagen de setiathome.berkeley.edu) ”

There you have it! A distributed system for analyzing the radio signals of the universe!

Distributed monitoring

Distributed monitoring depends on the network topology used, and I bring it up as an introduction or approach to monitoring a distributed system. If you are new to Pandora FMS, I recommend you take some time to read this post.

Essentially it is about distributed environments that give service to a company or organization but do not execute a common software and have very different areas or purposes between departments, supported in communication with a distributed network topology accompanied by a well planned security architecture in monitoring.

Pandora FMS offers in this field service monitoring, very well described in the official documentation.

Observability

It would be an attribute of a system, and the topic is worth a full blog post, but, in summary, I expose observability as a global concept that includes more alert monitoring and alert management activities, visualization and trace analysis for distributed systems, and log analysis.

Companies like Twitter have taken observability very seriously and, as you may have guessed, that addictive social network is a distributed system but with a diffuse end product (increase our knowledge and facts about the real world).

Transaction monitoring

How can we monitor a distributed system if it consists of very heterogeneous components and, as we saw, can reach any part of our known universe?

Pandora FMS has Business Transactional Monitoring, a tool that I consider the most appropriate for distributed systems since we can configure transactions, as many as we need, and then use the necessary transactional agents to do so.

It is a difficult topic to take in but our documentation starts with a simple and practical example, with which, as you experiment, you may add “blocks” of more complex transactions until you reach a point where you can have a panorama of the distributed system.

All this is possible with Pandora FMS since it has standard monitoring, remote checks, transaction synthetic monitoring and the Satellite server for distributed environments that can be used with transactional monitoring for distributed systems.

Present and future

The question is no longer whether we need distributed systems. That is a fact. Today people use distributed systems in computing services in the cloud or in data centers and the Internet.

Distributed systems can offer impossible functions in monolithic systems or take advantage of computer processes, such as performing restorations from backups by asking other systems for chunks that are missing or have deteriorated in the local system.

For all these cases, and in any case, the flexibility of Pandora FMS will always be useful and adaptable for current or future challenges.

There are different positions on whether observability and monitoring are two sides of the same coin.

We will analyze and explain what the observability of a system is, what it has to do with monitoring and why it is important to understand the differences between the two.

What is observability?

Following the exact definition of the concept of observability, observability is nothing more than the measure that determines how internal states can be inferred through external outputs.

That is, you may guess the status of the system at a given time if you only know the outputs of that system.

But let’s look at it better with an example.

Observability vs monitoring: a practical example

Some say that monitoring provides situational awareness and the capacity for observation (observability) helps determine what is happening and what needs to be done about it.

So what about the root cause analysis that has been provided by monitoring systems for more than a decade?

What about the event correlation that gave us so many headaches?

Both concepts were essentially what observability promises, which is nothing more than adding dimensions to our understanding of the environment. Be able to see (or observe) its complexity as a whole and understand what is happening.

Let’s look at it with an example:

Suppose our business depends on an apple tree. We sell apples, and our tree needs to be healthy.

We can measure the soil pH, humidity, tree temperature and even the existence of bad insects for the plant.

Measuring each of these parameters is monitoring the health of the tree, but individually they are only data, without context, at most with thresholds that delimit what is right or what is wrong.

When we look at that tree, and we also see those metrics on paper, we know that it’s healthy because we have that picture of what a healthy tree is like and we compare it with things that we don’t see.

That is the difference between observing and monitoring.

You may have blood tests, but you will only see a few specific metrics of your blood.

If you have doubts about your health, you will go to a doctor to look at you and help you with the analysis data, do more tests or send you home with a pat on your back.

Monitoring is what nourishes observation.

We’re not talking about a new concept, we’re rediscovering gunpowder.

Although being fair, gunpowder can be a powerful weapon or just used for fireworks.

The path to observability

One of the endemic problems with monitoring is verticality.

Have isolated “silos” of knowledge and technology that barely have contact with each other.

Networks, applications, servers, storage.

Not only do they not have much to do with each other, but sometimes the tools and equipment that handle them are independent. 

Returning to our example, it is as if our apple tree were dying and we asked each expert separately:

• Our soil expert would tell us it’s okay.
• Our insect expert would tell us it’s okay.
• Our expert meteorologist would tell us that everything is fine.

Perhaps the worm eating the tree reflected a strange spike in soil pH and it all happened on a day of subtropical storm.

By themselves the data did not trigger the alarms, or if they did, they corrected themselves, but the ensemble of all the signals should have portended something worse.

The first step to achieving observability is to be able to put together metrics from different domains/environments in one place. So you may analyze them, compare them, mix them and interpret them.

Basically what we’ve been saying at Pandora FMS for almost a decade: a single monitoring tool to see it all.

But it’s only the first step, let’s move on.

Is Doctor House wrong when he says everyone is lying?

Or rather, everyone tells what they think they know.

If you ask a server at network level if it’s okay, it will say yes.

If there is no network connectivity and the application is in perfect condition, and you ask at application level whether it is OK, it will tell you that it is OK.

In both cases, no service is provided.

And we’ll say, but how is it okay? it doesn’t work!

Therein lies the reason that observability and monitoring are not the same.

It is processing all the signals what produces a diagnosis and a diagnosis is something that brings much more value than data.

Is it better to observe or monitor?

Wrong.

If you’re asking yourself that question, we haven’t been able to understand each other.

Is it better to go to the doctor or just have an analysis?

It depends on what you’re risking.

If it is important, you should observe with all available data.

If what you’re worried about is something very specific and you know well what you’re talking about, it might be worthwhile to monitor a group of isolated data.Although, are you sure you can afford only to monitor?

Finding the needle in the haystack

Among so many data, with thousands of metrics, the question is how to get relevant information among so many shrouds. Right?

AIOPS, correlation, Big Data, root cause analysis…

Are we looking at another concocted word to sell us more of the same?

It may, but deep down it is a deeper and more meaningful reflection:

What is the use of so much data (Big Data) if I don’t have the capacity for its analysis to be useful to me for something practical?

What good is technology like AIOPS if we can’t have all the data together from all our systems, together and accessible?

Before developing black magic, the ingredients must first be obtained, if not, everything remains in promises and expensive investments that entail wasting time and the unpleasant feeling of having been deceived.

From monitoring to observability

In order to elevate monitoring to the new observability paradigm, we must gather all possible data for analysis.

But how do we get them?

With a monitoring tool.

Yes, a tool like Pandora FMS that can gather all the information together, in one piece, without different parts that make up a Frankenstein that we do not know either what it costs or how it is assembled.

And we’re not talking about a monitoring IKEA, made up of hundreds of pieces that require time and… a lot of time.

This is not new.

Nor is it new that we need a monitoring tool that can collect data from any domain.

For example, switch data, crossed with SAP concurrent user data.

Latency data with session times of a web transaction. 

Temperature in Kelvin dancing next to euro cents, positive heartbeats looking closely at the number of slots waiting in a message queue.

LThe only thing that matters is business.

Just the final view.

Observe, understand and above all, resolve that everything is okay, and if it is wrong, know exactly who to call.

What is real observability?

We call it service views.

It is not difficult, we provide tools so that you, who know your business, can identify the critical elements and form a service map that gets feedback from the available information, wherever it comes from.

FMS means for us FLEXIBLE Monitoring System, and it was designed to get information from any system, in any situation, however complex it was and store it to be able to do things with it.

Today our best customers are those who have such a large amount of information that other manufacturers do not know what to do with it.

We don’t know what to do with it either, I won’t fool you, but our customers with our simple technology do.

We help them process it and make sense of it. Make it observable. 

We would like to say that we have a kind of magic that others do not, but the truth is that we have no secret.

We take the information from wherever it comes from, whatever it is, and make it available to design service maps.

Some are semi-automatic, but customers who know what to do with it prefer to define very well how to implement them. I insist, they do it themselves, they don’t even ask us for help.

If you want to observe, you need to monitor everything first. 

And there we can help you.

Let’s check out together the features and improvements related to the new Pandora FMS release: Pandora FMS 760.

What’s new in the latest Pandora FMS release, Pandora FMS 760

NEW FEATURES AND IMPROVEMENTS

New histogram graph in modules

Added the ability to display a histogram graph for modules. This graph is exclusive for Boolean modules or for modules that have their criticality thresholds defined, it is very useful to see crash periods.

Alert templates with multiple schedules

The possibility of including several schedules for the execution of both module alerts and events is incorporated. With this new feature, different time slots can be generated within the same day or week, where alerts can be generated.

New Zendesk integration plugin

A Zendesk integration plugin has been added to the module library. Thanks to this plugin you may create, update and delete tickets from this system from the terminal or from Pandora FMS.

New inventory plugin for Mac OS X

Just as there were inventory tools for Linux and Windows, you may use this tool to obtain inventory in Mac OS X. You may get information on CPU, Memory, Disks and Software installed on machines of that OS.

New mass deletion section in the Metaconsole

With the latest changes in the process of combination and centralization in the Metaconsole, it was necessary to start including mass operations in it. For now, deleting and editing agents from the Metaconsole have been included.

New internal audit view in the Metaconsole

As part of the continuous improvements to Pandora FMS Metaconsole, the internal audit feature that already existed in the node has been added to supervise the accesses to the Metaconsole, as well as some of the actions carried out from it.

Forcing remote checks on Visual Consoles

In order to carry out a real-time monitoring control in the visual consoles, a button has been generated to be able to force the remote checks that are included in the visual consoles, just as it can be done from the detailed view of a node.

New alert macros

The following alert macros have been added to be able to include more details in the notices:

• _time_down_seconds_
• _time_down_human_
• _warning_threshold_min_
• _warning_threshold_max_
• _critical_threshold_min_
• _critical_threshold_max_

Support for MySQL8 and PHP8

We have included support to be able to use MySQL8 without any type of modification or previous adjustment. We are also preparing the console to work on PHP8 due to the PHP7.4 support time ending on 28th Nov 2022. 

Support for OS RHEL 8.x, RockyLinux 8.x, AlmaLinux 8.x

Due to recent changes to what was our base system so far (CentOS), we have decided to use RockyLinux 8 and AlmaLinux 8, as well as continue to support RHEL8 as the base OS. We recommend to all our users who have to migrate from other unsupported Linux versions (such as Centos6) to do so to one of these systems. However, we will continue to provide installers in RPM and Tarball format that can be used to run Pandora FMS on such systems.

KNOWN CHANGES AND LIMITATIONS

• New installations using ISO have been removed. From now on, the default installer will be the online installer, which, by means of a single command, prepares and installs the entire system from a Linux RHEL8, Rocky Linux or Alma Linux OS.

• Pandora FMS integration with the new plugin library has been improved, in order to use the new plugin library you need to be updated to version 760.

Resources

Download the release note 

Pandora FMS plugin library

Pandora FMS Online Community

I want to learn more!

Our Trial

We love uploading this kind of post to our blog. Articles in which we boast about our work and where all the effort of our team throughout the year comes to light. Because yes, we are rewarded once more, Pandora FMS is proclaimed winner in several categories in the SourceForge Awards.

• Award in the Community Leader category
• Award in the Community Choice category
• Award in the Open Source Excellence category
• Award in the category Users Love us
• SourceForge Favorite

No more and no less than five awards, including the Open Source Excellence 2022 award, possibly one of the most desired and disputed in the industry in this specific sector. 

Pandora FMS wins the Open Source Excellence 2022 award

As a message to the world from this podium, we want to make clear that it is an honor to know that these awards are only given to selected projects that have reached significant milestones in terms of downloads and participation within the SourceForge user community.

A great achievement to keep in mind, since Pandora FMS, one of the most complete monitoring software on the market, has been considered for these awards from more than 500,000 open source projects throughout the whole SourceForge platform. 

“We are very proud of what our team at Pandora FMS is achieving.  An effort of our entire workforce, users and customers that makes Pandora FMS better every day. This award is a recognition of our entire career and shows that Opensource is still alive and that we are one of the leading and pioneering projects in Europe”, states with satisfaction, Sancho Lerena, founder and CEO of Pandora FMS.

SourceForge is an open source software community devoted especially to helping open source projects be as successful as possible. Currently the platform has about 502,000 Open Source projects in progress, more than 2.6 million downloads per day and a community of 30 million monthly users, who search and develop open source software, and who, from now on, will be able to find the badges achieved by Pandora FMS within its projects page in SourceForge. 

As many of you already know, Pandora FMS is a very comprehensive monitoring solution: cost-effective, scalable and covering most infrastructure deployment options. Find and solve problems quickly, no matter if you come from on-premise, multi cloud or a mix of both. A flexible solution that can unify data display for full observability of your organization. With more than 500 plugins available you may control and manage any application and technology, such as SAP, Oracle, Lotus, Citrix, Jboss, VMware, AWS, SQL Server, Redhat, Websphere and a long etc. A flexible tool able to reach everywhere and unify data display to make management easier. Ideal for hybrid environments where technologies, management processes and data are intertwined. And now, moreover, backed and rewarded by the wide expert community of SourceForge.

How have we come so far?

Let’s go back a little. Pandora FMS is licensed under GNU GPL 2.0 and the first line of code was written in 2004 by Sancho Lerena, the company’s current CEO. At that time, free software was in full swing and the Free Software Foundation in Spain had an active group of which Sancho was a part.

In those days there was no Github, but there was something that united us all: SourceForge. From the beginning of the current century this platform served to unify and enhance thousands of developers who wanted to share their creations with users around the world. Pandora FMS was there from its inception in 2004, although initially it was not called that, but “Pandoramon.” 

*If you are curious about our beginnings, you may read this article about our history.

As of this date, there are several thousand free version users who download Pandora FMS updates through their update system and use it daily. 

Pandora FMS has been uploading every release with its corresponding source file for over 18 years, every day to its Sourceforge project and we are very proud to say that not only do we continue to believe in it, but we have not stopped doing so in almost twenty years of history.

Beyond code, we believe in the power of community, sharing, and growing together. That is why we maintain a very extensive documentation of more than 1000 pages in four languages: Spanish, English, French and Russian. 

Our community website includes a system of forums, an extensive knowledge base with more than 500 articles and a blogh more than 1,900 articles translated into four languages.

Of course, we also offer a wide range of professional services and commercial versions of our software. But, as Stallman himself said:

“Free software” means software that respects users’ freedom and community. Roughly, it means that the users have the freedom to run, copy, distribute, study, change and improve the software. Thus, “free software” is a matter of liberty, not price. To understand the concept, you should think of “free” as in “free speech,” not as in “free beer.” 

Now yes, after all that has been said, we invite you to check that freedom is much more than a slogan. Thank you again for this award. And don’t take too long, join us!

Do you already know what tasks the QA department performs? Would you like to discover what each QA tester does on a daily basis? You don’t know what the hell we’re talking about but you’re intrigued and can’t stop reading because my prose is enigmatic and addictive? Well, you’ve come to the right place! We’ll tell you how our QA department manages so you can learn what yours does without having to ask. Read on and don’t forget to propose me for the Nobel Prize for literature when the time comes!

Do you already know what tasks the QA department performs?

Starting with the functions of the department, QA is in charge of testing Pandora FMS and making sure that we offer the best possible quality to our clients and the community. It is an extremely complex task because Pandora FMS is very large and it could be said that chaos theory is well applied, since inserting an “&” character in a form field can cause a report that had nothing to do with it to fail. So be careful, any day your building could burn for the wrong character! From Pandora FMS we recommend hiring only professionals

Currently, our QA team is made up of Daniel Ródriguez, Manuel Montes and Diego Muñoz, although from time to time, colleagues from other departments support them to carry out specific tests. They are thick as thieves. They always sit together at company dinners and share a bottle of Beefeater after dinner.

QA Tester Team 

Daniel Rodríguez, “The beast (QA) of Metal”

Works together with the Support and Development Departments. He is devoted to testing new features and finding possible bugs to help improve the product. He loves sci-fi movies and metal:

“My duties as department head are mainly to manage and supervise the work of the department, design and improve test plans, carry out manual tests and coordinate communication with the rest of the departments.”

Manuel Montes

is from Madrid and began as part of the Development department, although he later joined the QA team. He loves cycling when the weather allows it, watching movies, reading and going for a walk with his family

“In addition to manual tests, we carry out automatic tests with technologies such as Selenium Webdriver and Java to interact with the browser, Cucumber with Gherkin language so that the tests to be carried out are somewhat more understandable for less technical colleagues and, in turn, serve as documentation, and Allure to generate reports with the results of said tests.”

Diego Muñoz, “The Gamer Alchemist”

is a QA tester, although he also helps the Support team, solving different problems for customers. He is from Huelva and although he has lived there all his life, he has no accent, which he boasts about. His hobbies range from watching movies, to video games, listening to music and watching series:

“Every piece of code that is implemented in Pandora FMS goes through my hands or through those of one of my colleagues, who judge if the changes work correctly or have any errors. We also sometimes suggest alternative ways to present features to developers or to solve the bugs that we have been able to find. In addition, in the days prior to the product release, we review the whole console in all its Metaconsole, Node and Open variants, once again making sure that the code introduced in the new version works as well as possible.”

The importance of the QA department

From the QA department, an average of 180 tickets are generated per release and, as you know, we present 10 releases a year. This adds up to more than 1,800 tickets annually, how cool is that? Sometimes it is a heinous job, because it involves throwing back the work of a Development colleague, and also difficult because it is impossible to see everything, and when a problem explodes in a client’s environment, it attracts all eyes. Although QA work has little visibility and can be very thankless, it is fundamental to the success of everyone’s work and the final product.

If you want to find out more about our departments out of curiosity or for the simple fact that this way you can find out more about yours, you can request it in the comments box, one of our busy social networks or by post, which is a little bit outdated but should totally come back. Scented letters and vermilion sealing wax. There can’t be anything more romantic!