Global

# How extensible do we want our software to be?

{{>toc}}

This is an informal, top-of-the-head note about some basic ideas about extensible software design. We'll discuss these and decide whether we need to incorporate some of these in the design of systems.

## Of themes and plug-ins

If you look at a mature open source product like Wordpress or Drupal, you see incredible levels of extensibility, which is never to be found in bespoke business applications. I don't use words like "incredible" lightly -- when I began to study such systems, I couldn't believe what I found.

This incredible level of extensibility comes from two broad areas of extensibility:

* **Themes**: design the front-end to be a separate sub-system, including both appearance and features which are visible to the human users

* **Plug-ins** or modules: design the core engine, or heart, of the software to call extensions at almost every point, if such extensions exist. These extensions are called plug-ins or modules.

This is seen in practically every mature open source product. Apache, the plain old web server, has [modules](https://httpd.apache.org/docs/2.4/developer/modguide.html). In MySQL, even the core storage management is built on an unchanging core which can call any one of a set of [pluggable storage managers](https://dev.mysql.com/doc/refman/5.7/en/pluggable-storage.html). Since MySQL and Apache do not have any significant UI component, there are no themes for them.

*Themes* are not to be confused with *skins*. Many products let you replace the default *skin* with new ones. Skins typically purely change the look, without changing functionality. Themes make much deeper changes. There are themes which change a blogging system (WordPress) into an [online e-commerce system](https://wordpress.org/themes/search/ecommerce/), for instance. This involves adding vast amounts of internal features, not just skinning.

## Do bespoke systems need such stuff?

Important systems need 4-5 times as much investment in their extensions and modifications, over their operational life, as in the original design and implementation of v1.0. If we can reduce the investment needed for maintenance, it'll save a lot of money and time in the long run.

Moreover, it reduces risk during the implementation of these extensions, because a lot of these changes can be done without getting into open-heart surgery of the application code. Separation of core from outer layers, or just designing the entire system as a set of interacting modules rather than a monolithic block, reduces the risk of things breaking during the maintenance phase.

It's hard to quantify the money saved or risk reduced, of course, because it's hard to set up control experiments.

## Maintainability for a software system

A lot of areas in any software system are essentially nitty-gritty detail, and do not need to be treated as core. I am afraid that a straitlaced design of any system will result in all these getting locked into the design, and there will be no separation between "core" and "detail".

I'll share some rough examples.

* **Lists of values**: there are lots of places where there are enumerated lists. For instance, the list of designations or positions of candidates, the list of sectors in which a recruitment agency may specialise in finding candidates, the list of sectors to which a specific company may belong, etc. Very few of these lists have any semantic significance to the actual business logic of the application. The list of states of an object has semantic significance, since actions are taken in the logic based on the state of the object. But most of the other enumerated lists are purely informational. These do not need to be hard-coded -- they can be listed in a table or a `lists.json` config file.

* **LoV with semantic significance**: many objects have states, and these states are used for controlling state transition based on some business logic. We all know that typical bespoke applications hard-code these enumerated lists, and hard-code the logic which will handle transitions from state to state. We also know that it doesn't need to be this way -- there are things like rules engines. Even if we don't go all out and use a full-blown powerful rules engine, we can use the *concept of a rules engine* to separate out the core from the actual LoV. We can design an engine which uses a lookup table to call the corresponding call-back functions to execute the business logic for the transition.

    My mental model here is a parser or a lexer, when generated by `lex` or `yacc`. These are table-driven engines. The `lex` or `yacc` tool generates the table, but the code which reads and "executes" the table, the "engine", is actually general-purpose -- it's not language syntax specific at all.

    If we separate this business logic into a core and a set of executable pieces which are plugged in, then we can make changes to the state transition logic (which in the end-user's eye is the "core functionality" of the application) in a much more modular way, with minimal peripheral impact on the rest of the application. We can also add/ remove transitions, or even add/ remove states, to the business logic.

* **Actions on objects**: What can you do with the object called "candidate", "accounting voucher", "agency" or "vendor"? Take the case of a "proposal" in a Proposal Management Module. You can

  * create a proposal

  * send the proposal to a reviewer

  * share it with someone over email

  * delete or deactivate it

  * convert it into a PDF document/ Word file

  * ask someone for comments

  * ... and so on and so forth

    What if we need to add more actions to the "proposal" object tomorrow? (In the language of object-oriented programming, if the "proposal" is an object class, I'm talking about adding new "methods" to it.)

    If the application is designed to allow a programmer to add new artefacts to the UI (*e.g.* a new item to a menu), add new entries to an authorisation lookup table (*e.g.* an entry in a table to indicate who can trigger this menu option), and add new back-end code which will be called when this menu item is triggered, *all without re-compiling existing code*, then we can add new features or actions to an object without destabilising existing code. The new code can be a drop-in addition, exactly like a new theme or extension to Wordpress.

* **Attributes of objects**: how do we make our code so modular or extensible that we can add two new fields to a "proposal" without re-building the application? (We can do this to incredible levels with objects in Wordpress code.)

## The downside

Designing software to be extensible needs more thoughtful designers who have studied the design of software beyond bespoke TCS-type applications. It also increases the design effort. It may not increase coding effort, and it definitely will reduce testing and debugging time.

It will also increase some documentation effort, because there will now need to be a Programmer's Guide to Extending XYZ System.