Project

General

Profile

Wiki » History » Version 7

Shuvam Misra, 15/09/2023 01:29 PM

1 1 Shuvam Misra
# The Remiges product family
2
3
We currently have six products on the anvil, and this list will not grow till end of 2024. We give a short note about each here.
4
5 2 Shuvam Misra
Some common points about all the products:
6
* All of them are server-side frameworks or systems which help the application developer develop some key functions very fast, securely, reliably and at scale.
7
* All the products are scalable -- they can work with any application, irrespective of how fast the application processes transactions or how many transactions per second it handles
8
* They will integrate with application code written in any programming language. Wherever client libraries are required, they will be provided initially in Go (the mother language of all the products other than IDshield), then on Java, then on PHP, dotNet, NodeJS, *etc*.
9 1 Shuvam Misra
10 2 Shuvam Misra
## Remiges ServerSage: an observability framework
11 1 Shuvam Misra
12 2 Shuvam Misra
This product will allow an application to log performance metrics and health metrics about all aspects of its operation into an observability framework. This framework will include screens to see the data in historical time-series format, correlate different events on a common time axis, and generate automatic alerts wherever readings fall outside acceptable watermarks.
13 1 Shuvam Misra
14 5 Shuvam Misra
The framework is being built by extending **Prometheus**, which is a gold standard in observability solutions today. Prometheus provides infrastructure monitoring agents out of the box, so that you can track CPU load, RAM utilisation, etc, at the server level. ServerSage will extend this to track application health. Duration of each web service call, duration of each database command, durations of key parameters at the business operation level, will all be exported *via* custom-built agents to a central Prometheus data store. The data visualisation front-end for ServerSage is being built by extending **Grafana**, the industry-standard data dashboard tool.
15 1 Shuvam Misra
16 3 Shuvam Misra
Source code for agents will be made available in Go and Java to allow embedding inside the business logic of your application, so that you can decide what to log in what manner, and then just plug in the right agents inside your code to push out that data to Prometheus.
17
18
A two-tiered Prometheus setup will be available to allow data to be gathered into lower-level Prometheus servers, and then operate Tier 1 servers to pull in and hold the aggregate data in a cluster-wide data store for higher level reports and alerts.
19
20 2 Shuvam Misra
## Remiges LogHarbour: a logging framework
21 1 Shuvam Misra
22 4 Shuvam Misra
This product will allow application developers to log three types of data in a unified cluster-wide data store from all types of applications:
23
* application change log audit trail, giving the "before" and "after" values of all data items which are changing or getting deleted
24
* activity and event logging, for information and forensics
25
* debug logging, for developers to trace problems in the flow of code
26
27 6 Shuvam Misra
The data thus logged will be sent into Kafka streams and will finally reach Kafka consumers which will store the data in an ElasticSearch database. The product is built by utilising and integrating **Kafka**, the gold standard in distributed, very high performance logging frameworks, and **ElasticSearch**, the top full-text searchable document database. The proven performance and reliability of these systems will be available in LogHarbour.
28 4 Shuvam Misra
29
LogHarbour will also have connectors which can parse log files generated from legacy systems and ingest them into the central data store. ElasticSearch will operate in cluster mode with cross-replication, to allow large data stores of several billion entries and still serve queries at speed and scale.
30
31
Every application development team re-invents the wheel when it decides on a logging module, and most of the time, this logging is done into the main transaction database, thus impacting overall scalability and throughput of the system. Integrating LogHarbour will allow applications to scale at will without worrying about whether the logging system can keep up or where the logs need to be stored.
32
33 2 Shuvam Misra
## Remiges IDshield: an authentication and authorization framework
34
35 7 Shuvam Misra
Every business application needs to build, or plug in, some module to handle the user table and implement authentication and authorization. Remiges IDshield provides application developers with a ready-made service, together with its own database, to take care of these features. It will provide a login authentication interface which can be used by browser based applications or mobile apps, and will support stateless JWT tokens for session management. JWT tokens allow session tracking without any reference to a central session table in a database which invariably becomes a hotspot when the application usage scales. IDshield will also provide the framework for defining authorization rules, so that the application can check at each point whether the current user has the right to perform each operation.
36
37
Remiges IDshield is built on top of the very well-known open source identity and authentication management system, **Keycloak**. Clients will also have the freedom to opt for **RedHat SSO**, which is a derivative product of Keycloak with commercial support from RedHat. IDshield extends Keycloak to add features which do not exist in the default distribution, like 2FA using SMS, geo-IP based access restrictions, *etc* It also provides a client library which allows application developers to make web service calls to IDshield to manage users, change access rights, *etc* The user table is stored in the relational database private to IDshield. Horizontal scalability is easily achieved by running multiple instances of the IDshield service connecting to a single data store. Accesses to this data store are infrequent, because IDshield uses a caching layer to optimise performance.
38
39
With Remiges IDshield, application developers can focus more resources on actually building the business logic and less on developing the surrounding framework. In addition, IDshield offers a full-featured identity and authentication management framework which has been reviewed by public scrutiny of the source code and security testing.
40
41 2 Shuvam Misra
## Remiges Rigel: a global configuration management system
42
43
## Remiges Alya: a framework for writing web service calls in Go
44
45
## Remiges Crux: a business rules engine (BRE) and workflow engine