Elasticsearch is a NoSQL database, usually used as a search engine. However, with its powerful logic, it can be used for pretty much anything. In our case, we have used Elasticsearch to hold aggregated NetFlow data. As we are incorporating it step by step, currently Elasticsearch database is in the beta phase.
Elasticsearch holds data inside indices, which can be interpreted as a number of documents. A document can represent a collection of rows that can keep pretty much anything. In our case, PostgreSQL (currently the main database) stores data in weekly tables, and then filters data by the date chosen on the date picker. Elasticsearch now holds data in daily indices, that are more suitable for providing compliance with GDPR, for example. And many other cool stuff, but about that we will talk some other time.
The biggest advantage NoSQL databases provide, compared to the older RDBMS, is distribution. Meaning, you can scale Elasticsearch to pretty much unlimited number of nodes, on premise or in cloud. Naturally, bigger memory and disk requirements are necessities in this case.
In NetVizura, Elasticsearch is optimized for searching and grouping data based on NetFlow. There are two kinds of fields in Elasticsearch, text and terms. While text is more suitable searching (and therefore more applicable in our EventLog module for example), terms are proper for grouping, bucketing and mathematical calculations - that is why NetVizura has decided on these fields for NetFlow Analyzer. Not only that we have achieved better performances, but we got the chance to extend NetVizura with new features in the future (like minute aggregations,drill-down menus, to mention some).
At the moment, in NetVizura, the feature coming with Elasticsearch is the new Dashboard Beta. Here, we have replaced GWT with more modern React REST API's (read more about this in our other blog post: How to integrate React code with legacy Google Web Toolkit (GWT)). Meaning, the new release, NetVizura 4.7.1 is pretty much a bagful of wonders!
