Project

General

Profile

Wiki » History » Version 6

Shuvam Misra, 15/09/2023 12:16 PM

1 1 Shuvam Misra
# The Remiges product family
2
3
We currently have six products on the anvil, and this list will not grow till end of 2024. We give a short note about each here.
4
5 2 Shuvam Misra
Some common points about all the products:
6
* All of them are server-side frameworks or systems which help the application developer develop some key functions very fast, securely, reliably and at scale.
7
* All the products are scalable -- they can work with any application, irrespective of how fast the application processes transactions or how many transactions per second it handles
8
* They will integrate with application code written in any programming language. Wherever client libraries are required, they will be provided initially in Go (the mother language of all the products other than IDshield), then on Java, then on PHP, dotNet, NodeJS, *etc*.
9 1 Shuvam Misra
10 2 Shuvam Misra
## Remiges ServerSage: an observability framework
11 1 Shuvam Misra
12 2 Shuvam Misra
This product will allow an application to log performance metrics and health metrics about all aspects of its operation into an observability framework. This framework will include screens to see the data in historical time-series format, correlate different events on a common time axis, and generate automatic alerts wherever readings fall outside acceptable watermarks.
13 1 Shuvam Misra
14 5 Shuvam Misra
The framework is being built by extending **Prometheus**, which is a gold standard in observability solutions today. Prometheus provides infrastructure monitoring agents out of the box, so that you can track CPU load, RAM utilisation, etc, at the server level. ServerSage will extend this to track application health. Duration of each web service call, duration of each database command, durations of key parameters at the business operation level, will all be exported *via* custom-built agents to a central Prometheus data store. The data visualisation front-end for ServerSage is being built by extending **Grafana**, the industry-standard data dashboard tool.
15 1 Shuvam Misra
16 3 Shuvam Misra
Source code for agents will be made available in Go and Java to allow embedding inside the business logic of your application, so that you can decide what to log in what manner, and then just plug in the right agents inside your code to push out that data to Prometheus.
17
18
A two-tiered Prometheus setup will be available to allow data to be gathered into lower-level Prometheus servers, and then operate Tier 1 servers to pull in and hold the aggregate data in a cluster-wide data store for higher level reports and alerts.
19
20 2 Shuvam Misra
## Remiges LogHarbour: a logging framework
21 1 Shuvam Misra
22 4 Shuvam Misra
This product will allow application developers to log three types of data in a unified cluster-wide data store from all types of applications:
23
* application change log audit trail, giving the "before" and "after" values of all data items which are changing or getting deleted
24
* activity and event logging, for information and forensics
25
* debug logging, for developers to trace problems in the flow of code
26
27 6 Shuvam Misra
The data thus logged will be sent into Kafka streams and will finally reach Kafka consumers which will store the data in an ElasticSearch database. The product is built by utilising and integrating **Kafka**, the gold standard in distributed, very high performance logging frameworks, and **ElasticSearch**, the top full-text searchable document database. The proven performance and reliability of these systems will be available in LogHarbour.
28 4 Shuvam Misra
29
LogHarbour will also have connectors which can parse log files generated from legacy systems and ingest them into the central data store. ElasticSearch will operate in cluster mode with cross-replication, to allow large data stores of several billion entries and still serve queries at speed and scale.
30
31
Every application development team re-invents the wheel when it decides on a logging module, and most of the time, this logging is done into the main transaction database, thus impacting overall scalability and throughput of the system. Integrating LogHarbour will allow applications to scale at will without worrying about whether the logging system can keep up or where the logs need to be stored.
32
33 2 Shuvam Misra
## Remiges IDshield: an authentication and authorization framework
34
35
## Remiges Rigel: a global configuration management system
36
37
## Remiges Alya: a framework for writing web service calls in Go
38
39
## Remiges Crux: a business rules engine (BRE) and workflow engine