The Remiges product family¶

We currently have six products on the anvil. This list will not grow till end of 2024. We give a short overview about each here.

Some common points about all the products:

• All of them are server-side frameworks or systems which help the application developer develop some key functions very fast, securely, reliably and at scale.
• All the products are scalable -- they can work with any application, irrespective of how fast the application processes transactions or how many transactions per second it handles
• They will integrate with application code written in any programming language. Wherever client libraries are required, they will be provided initially in Go (the mother language of all the products other than IDshield), then on Java, then on PHP, dotNet, NodeJS, etc. Remiges Alya is a web development framework, so there are no client libraries -- the entire framework is linked with the application code. This will be Go initially, then in Java.
• All of them are available as open source products from their respective GitHub repositories, and may be downloaded, modified, and re-used without restrictions. They are fully maintained and supported by dedicated teams within Remiges.
• All of them are for application designers and developers. Some of them have UI which may be used by non-technical users after they have been integrated into business applications, but they are not usable standalone from a UI till they are integrated into an application.

Remiges ServerSage: an observability framework¶

This product will allow an application to log performance metrics and health metrics about all aspects of its operation into an observability framework. This framework will include screens to see the data in historical time-series format, correlate different events on a common time axis, and generate automatic alerts wherever readings fall outside acceptable watermarks.

The framework is being built by extending Prometheus, which is a gold standard in observability solutions today. Prometheus provides infrastructure monitoring agents out of the box, so that you can track CPU load, RAM utilisation, etc, at the server level. ServerSage will extend this to track application health. The duration of each web service call, duration of each database command, durations of key parameters at the business operation level, will all be exported via custom-built agents to a central Prometheus data store. The data visualisation front-end for ServerSage is being built by extending Grafana, the industry-standard open observability platform.

Source code for agents will be made available in Go and Java to allow embedding inside the business logic of your application, so that developers can decide what to log in what manner, and then just plug in the right agents inside their code to push out that data to Prometheus.

A two-tiered Prometheus setup will be available to allow data to be gathered first in Tier 2 Prometheus servers, and then operate Tier 1 servers to pull in data from them and hold the aggregate data in a cluster-wide data store for consolidated reports and alerts.

The name ServerSage points to an entity which has wisdom and is aware about the servers and other components in its care.

Remiges LogHarbour: a logging framework¶

This product will allow application developers to log three types of data in a unified cluster-wide data store from all types of applications:

• application change log audit trail, giving the "before" and "after" values of all data items which are changing or getting deleted
• activity and event logging, for information and forensics
• debug logging, for developers to trace problems in the flow of code

The data thus logged will be sent into Kafka streams and will finally reach Kafka consumers which will store the data in an ElasticSearch database. The product is built by utilising and integrating Apache Kafka, the gold standard in open source distributed event steaming platforms, and ElasticSearch, the top full-text searchable document database. The proven performance and reliability of these systems will be available in LogHarbour.

LogHarbour will also have connectors which can parse log files generated from legacy systems and ingest them into the central data store. ElasticSearch will operate in cluster mode with cross-replication, to allow large data stores of several billion entries and still serve queries at speed and scale.

Every application development team re-invents the wheel when it decides on a logging module, and most of the time, this logging is done into the main transaction database, thus impacting overall scalability and throughput of the system. Integrating LogHarbour will allow applications to scale at will without worrying about whether the logging system can keep up or where the logs need to be stored.

Remiges LogHarbour can operate as a shared multi-tenant log repository on public cloud infrastructure, and can be used by multiple applications concurrently. Each tenant of such a LogHarbour installation gets its own ElasticSearch index and log data is completely isolated from other applications.

The name LogHarbour points to the vision of logs of all types and for all applications protected in a safe store.

Remiges IDshield: an IAM (identity and access management) product¶

Every business application needs to build, or plug in, some module to handle the user table and implement authentication and authorization. Remiges IDshield provides application developers with a ready-made service to implement these features. It provides a login authentication interface which can be used by browser based applications and mobile apps, and will support stateless JWT tokens for session management. JWT or JSON Web tokens allow session tracking without any reference to a central session table in a database; this table invariably becomes a hotspot when the application usage scales. IDshield will also provide the framework for defining authorization rules, so that the application can check at each point whether the current user has the right to perform each operation.

Remiges IDshield is built on top of the very well-known open source identity and authentication management system, Keycloak. Clients will also have the freedom to opt for RedHat SSO, which is a derivative product of Keycloak with commercial support from RedHat. IDshield extends Keycloak to add features which do not exist in the default distribution, like 2FA using SMS, geo-IP based access restrictions, etc It also provides a client library which allows application developers to make web service calls to IDshield to manage users, change access rights, etc The user table is stored in the relational database private to IDshield. Horizontal scalability is easily achieved by running multiple instances of the IDshield service connecting to a single data store. Accesses to this data store are infrequent, because IDshield uses a caching layer.

With Remiges IDshield, application developers can focus more resources on actually building the business logic and less on developing the surrounding framework. In addition, IDshield is standards-compliant (it complies with RFC 7519 and related standards) and has been reviewed by public scrutiny of the source code and security testing.

IDshield also adds industry-leading features for authorisation, where each user or group is given capabilities with constraints, and IDshield can inform the business application about which constraints apply when a user attempts an operation. It is not enough, for instance, to specify whether a user can enter a voucher in your application, it may be necessary to track whether the user is allowed to create vouchers less than a certain value limit, or whether she can create vouchers only of certain types. Therefore, a base-level capability may need to be qualified with what we call scope and limit constraints, to arrive at a qualified capability for each operation. When the application queries IDshield to verify authorisation, IDshield can report back with constraints which apply. This is not available in Keycloak or other leading IAM systems.

Remiges IDshield can operate as a multi-tenanted SaaS service on public cloud infrastructure, and provide services for multiple applications.

The name IDshield builds upon the idea of "ID" referring to identity, and "shield" referring to security.

Remiges Rigel: a global configuration management system¶

All well-designed applications need configuration parameters, which should be editable without releasing a new version of the application binaries. At the simplest end, a configuration management module is a thin wrapper over a JSON or YAML file; the file holds the configuration values in the form of key-value pairs. Modern configuration management requirements demand much more. The configuration must not get accidentally deleted due to file system corruption. The configuration must be available at each point of a distributed cluster of servers. The data must be version controlled for forensic purposes, and there must be access rules controlling who can make changes to the parameters.

Remiges Rigel is a cluster-aware multi-tenant configuration management system which replicates configuration information and offers an access-controlled GUI to allow administrators to make changes to individual entries. Application code may query individual entries in the configuration database by calling functions in a client library. Changes need to be made only once, and replicate automatically to all nodes in the cluster, where they are cached. The system maintains a version number to detect updates to the data. Rigel logs its activities into files or optionally into Remiges LogHarbour to keep an audit trail of changes. Access control for the configuration management interface may optionally be integrated with Remiges IDshield.

Rigel is a simple system which has all the necessary features needed for a robust distributed configuration management system. Designers of large business applications must not rely on simple local config files for the most critical of system parameters, and Rigel offers a simple and easy alternative. If Rigel is hosted on a public cloud as a service, it operates in multi-tenant mode where each application is a tenant of the Rigel installation and its data is isolated from that of other applications.

Rigel is a very bright super-giant star in the constellation Orion, and this imagery aligns well with the idea of global configuration being a bright component which other components refer to and are guided by.

Remiges Alya: a framework for writing web service calls¶

The Go programming language has already established itself as the worldwide and worthy successor to C for most systems programming projects, and a lot of the learning from forty years of C have been ploughed back into its design. For web service calls, Go has the Gin framework which offers a lot of flexibility, but is not easy to use for high volume web service calls development and testing by a large team. A lot of common actions need to be implemented in template form to enhance programmer productivity and reduce errors. This picture is repeated with Java, where Springboot is quite popular but additional features and templates can enhance productivity.

The Remiges Alya framework provides these reusable pieces.

• a validation framework and template to validate each parameter in the call request, so that basic syntactic checking is always done consistently
• an error response management package, which allows every web service call author to return an array of errors as per a prescribed structure. Each error gets an error code and an i18n-indexed error message in the language of choice as per the caller's preferences
• a framework for managing SQL statements, so that all the boilerplate code and error checking best practices are incorporated automatically through code generated by sqlc. This dramatically reduces errors and increases programmer productivity, and as a side-effect, results in the developer team maintaining a global list of SQL statements, which can be maintained independent of the business logic. (The Java Springboot implementation of Alya uses a different toolset.)
• a framework for processing batches asynchronously, distributing the load across all application instances in the cluster
• integration with Remiges LogHarbour for complete logging of audit trails, activity logs, and debug logs, with the feature to turn on or off debug logging for just one module of one application without restarting any services
• integration with Remiges IDshield for authentication, authorization and user management
• integration with Remiges Rigel for global configuration
• various other utility functions

When all these packages are aggregated, the designer can worry less about peripheral challenges and decisions and focus on the core business logic. The developer gets a trusted framework which takes care of all the housekeeping tasks unrelated to the business logic and can focus on adding functional features.

The name Alya means sky, loftiness and heaven in Arabic. It is also one star in a triple-star system in the constellation of Serpens.

Remiges Crux: a business rules engine (BRE) and workflow engine¶

Complex business applications have built-in rules to take decisions, e.g. decisions about how to ship an item, how to validate a new applicant, how to calculate interest payable. These rules are never constant and universal -- different rules apply in different situations. What is worse is that rules need to change over time. If code for the application needs to change to reflect each change in rules or each special case, then application maintenance becomes error-prone and expensive. It is better to have a repository of all such rules, treat the rules as data not code, and let the rest of the application query the rules engine with the full context of each case to get, as response, the rules to apply.

Such a business rules engine (BRE) must have an interface which users may use to define rules, and an application programming interface (API) to allow the business logic code to query the rules engine by submitting the full context of each "case" or "entity" being processed. Remiges Crux is such a rules engine. It has its own data store, is entirely managed by web service calls, runs as a separate service, can serve a cluster of applications, is independent of programming language, and is amenable to horizontal scale-out.

Remiges Crux will also act as a workflow engine, where the business logic will pass on the full context of an entity which is traversing a workflow path, and the workflow engine will respond by saying what is the next processing step to apply to that entity.

With Remiges Crux, application designers can tackle frequent changes in business rules, or define exceptions to business rules, without having to go through an expensive code change process, with time-consuming testing and release cycle. Application owners can respond to business context changes by changing rules without waiting for the development team to go through their elaborate song-and-dance routine to implement the change. Complex business applications simply become better, more intelligent, with Crux.

With any business rules engine, the rules are a crucial part of the business logic of the application which uses it. Errors in rule formulation will lead to misbehaviour in the business application. Therefore, testing of rules in a safe sandbox environment is critical to reliable maintenance of rulesets. Crux provides features to allow this. Querying of the rules engine is a deterministic process, but it may become difficult to understand why a query is emerging with specific unexpected result. Crux allows detailed trace data to be generated for each query, to allow operators and rule management teams to trace the traversal of a query through the forest of rulesets and understand exactly how a specific response is thrown up at the end. Large business applications need change management features to allow a smooth transition from an old rules regime to a new regime at one sharp point in time, with all testing and trace-analysis done beforehand. Crux supports multiple watertight ruleset spaces, called "realm slices", to allow this. In short, Crux has features which reflect the concerns of large application management teams and their business continuity processes.

Remiges Crux can operate as a multi-tenant SaaS service on public cloud infrastructure, with complete isolation between applications.

The name Crux refers to a constellation in the southern sky which is centred around four bright stars in a cross-shaped asterism often referred to as The Southern Cross.