Global

# From spec to technical design

{{>toc}}

This page will describe

* what processes we can follow to arrive at detailed software design from layman specs, and

* what documents we need to create along the way

We will first start by taking a look at the spec and design documents conventionally used in the industry for a few decades, then look at the terminology used by the much more recent agile development chaps, and then arrive at the documentation we need for our use in Remiges. After that, we will describe the process we can follow to arrive at the detailed design document.

The objective of this entire process is to arrive at documentation which can be handed over to a bunch of good programmers who can then do the coding and testing without discussing spec with the client.

## Documentation in the traditional approach

The traditional approach, called the "waterfall" approach, believed in documenting everything in great detail, getting the client to sign off on each document, and then starting software development. In that approach, the steps were as follows:

* The **BA** (Business Analyst) studies the business operations of the client, suggests some changes to the process for suitable computerisation, and then captures all this in something called a **BRD** (Business Requirements Document). The BRD is also known as BRS (Business Requirements Specifications), FRS (Functional Requirements Specification), etc. They roughly mean the same thing.

* The tech team then sits with the **BRD** and creates the **SDD** (Software Design Document). This contains technical design and architecture which is useful for the programmers who will develop the software.

* One or more architects are sometimes brought in to design the overall architecture of the solution.

* Modern applications are expected to have very good UX and UI, since no user wants to go through a training programme or read a user manual to use it. Therefore, a third activity is the UX and UI design, which is done by studying the BRD. This UI design is used by developers to design the screens.

This is the four-part process for software design and specification in the conventional approach. The dev team finally gets 

* the SSD

* the UI spec and screens

to do their coding.

## Documentation in the agile approach

The agile approach does not believe in creating documentation first, and then starting their coding activity. They believe that documentation needs to be created in short chunks, enough to feed the dev team for 1-2 sprints. They have a very specific structure for their software spec. They talk of specifications in a 4-level hierarchy:

* **The theme**: a company-wide focus area, specified in broad terms

* **The initiative**: a large objective or target, assigned to a department or multiple departments, to achieve in a time-bound manner. Many initiatives may emerge from a theme.

* **The epic**: a target to be achieved by a single team. Many epics may emerge from a single initiative.

* **The user story**: a small, single deliverable, demo-able task or function or feature to be implemented in a software application. Many user stories together are used to deliver an epic.

### Examples of themes, initiatives and epics

* A **theme** could be: "we need to increase our profits by 20% in the next financial year."

* Some **initiatives** which may emerge from this theme:

    * **Initiative 1**: cut costs by 10% in the fixed overheads of the company

    * **Initiative 2**: improve programmer productivity by 5% through all teams

    * **Initiative 3**: acquire 15 new customers in the next financial year

* Some **epics** which may be triggered by the Initiative 3 (acquire 15 new customers):

    * **Epic 1**: recruit and train 3 new sales executives and deploy them on the field (Sales Dept)

    * **Epic 2**: revamp the website to reflect more attractive offerings we have (Marketing Dept)

    * **Epic 3**: set up a call centre for outbound sales calls for lead generation (Marketing Dept)

    * **Epic 4**: deploy the latest version of a salesforce automation system and train the full sales team on it (IT Dept)

* Some **epics** which may be triggered by Initiative 1 (cut fixed costs):

    * **Epic 5**: explore relocation of HO to less expensive real estate (Estate Dept)

    * **Epic 6**: install intelligent AC monitoring systems to reduce AC electricity bills (Admin and Facilities Mgmt)

    * **Epic 7**: downsize some of the clerical departments and outsource to a BPO (the relevant clerical depts)

As is evident, the agile approach to specifying a "project" is not limited to software development alone, and works very well both for software and for non-software projects.

The examples clearly show:

* A theme is a company-wide focus area, stretching over a longer term. It is implemented by kicking off various initiatives in the company.

* An initiative is a large activity with a single target or objective, but may involve multiple departments, functions or teams

* An epic is a smaller activity which involves exactly one team within one department, and it is expected to be completed in a shorter timeframe than a theme or initiative. The epic is what we, in the software world, call "a project".

We can almost characterise these things this way:

||Time scale|Size|

|---|---|---|

|Theme|a financial year|full organisation|

|Initiative|a financial year|one goal across multiple departments or teams|

|Epic|3-4 months|One team in one department|

|User story|a few days|One software dev team of 5-10 people|

### Examples of user stories 

Here we are assuming that we are building an FA system.

* As an accounts executive, I should be able to enter new vouchers into the system so that I can log all accounts transactions.

* As an accounts executive, I should be able to edit existing vouchers so that I can correct errors in them.

* As an accounts manager, I should be able to see a list of vouchers entered into the system in the last day or week, so that I can monitor progress of work and keep track of recent transactions.

* As a Financial Controller, I should be able to see the up-to-date P&L statement, so that I can keep track of overall company income and expenditure.

Every user story *always* has the following parts:

* **Persona**: *e.g.* an accounts executive

* **Desired feature or capability**: *e.g.* see a list of vouchers entered into the system

* **So that**: *e.g.* I can keep track of progress of work and see recent transactions

User stories, in the context of software development have the following features:

* They are the smallest unit of software spec or feature which can be delivered and demonstrated to the end-user

* They always specify something which the non-technical end-user can *use*, therefore they will never specify just a library to be developed or a database table to be created

* They are almost always something which can be implemented by a small dev team (a sprint team) in a few days at max.

As a result of all this, they are almost always full-stack. In other words, implementing a user story involves building the screens (HTML, CSS, *etc*), writing the front-end code (browser-based or app code), writing back-end server-side web service calls, *etc*, creating database tables, integrating with other systems as needed, and so on. Each user story may be a very narrow feature or function *horizontally*, but is full-stack *vertically*.

### Documentation in the agile world

Since documentation is built incrementally as software is developed, in sprints, therefore the documentation is quite different in this approach.

There are two broad approaches to agile software development: [scrum](https://en.wikipedia.org/wiki/Scrum_(software_development)) and [kanban](https://en.wikipedia.org/wiki/Kanban_(development)). These are two *implementations* of the agile *methodology* or *approach*. Scrum is more suited for the turnkey development of a software application, and kanban is better suited to the continuous improvement phase of a product. We will focus on the scrum approach henceforth.

In the scrum approach, the documentation which is developed is:

* the **epic**: just a few lines of text

* the list of **user stories**: each stated in 1-2 sentences

Needless to say, this is not enough for the developers to actually write code. But this is what is created and entered in a project tracking system like Redmine or Jira as a "burn-down chart". If all the user stories are implemented and accepted by the end-users, the project is deemed complete.

Actual development needs more work, which is done within the sprint by the multi-functional dev team:

* add details to the user story: write additional notes of the business logic, work flow, etc

* create the screens for the user story

* define the data model: add new tables, add columns to existing tables

* decide the code to be written: new code as well as re-factoring and re-structuring of existing code

* test automation scripts, test cases, *etc*

Therefore, a user story usually leads to a set of **sub-tasks** which are created by the team (maybe by the team lead or the PM) and then assigned to individuals who execute them one by one. All the sub-tasks of a user story must be completed within the same sprint, because the mandatory requirement is that a sprint must deliver a few *complete user stories*. Half-complete user stories are not allowed to be spread across sprints.

## Documentation for Remiges

In Remiges, we can follow the following documentation:

* The BA phase can create what we call the Level 2 specifications, which are similar to a traditional BRD

* The UX/UI phase can create high-fidelity screens from the Level 2 specifications

* The detailed software specs can be created from the Level 2 specs

These three can then be fed to the dev and  testing teams to implement the software, in the form of a sequence of sprints. In other words, we are doing some documentation in the conventional approach, and then implementing the code using the agile approach of sprints.

## What the documents contain

The Level 2 software specs will contain

* a half-page description of the overall project and its objectives (like a long epic)

* a list of personas: the various categories of users who will use the system

* a list of business operations or user stories: for each business operation there will need to be:

    * a wireframe or "rough" screen with fields

    * a list of input and output fields

    * brief notes about the calculations or logic, if needed (for more complex operations)

* NFR notes: non-functional requirements like

    * performance: must give millisecond response for XYZ screens

    * database size: how many vouchers total, how many vouchers per year?

    * number of users,

    * languages supported: do you need multi-lingual support?

    * platforms supported: Chrome on Linux, Chrome on Windows, iPads, Android phones, *etc*

    * integration with other systems if any

    * and any other such requirements

The detailed software spec will contain

* architecture of the final system: could be just one big diagram

* list of components: Apache, JBoss, NodeJS, Postgres database, Redis, Angular front-end, *etc*

* data model: which will include

    * ER diagram

    * database schema: for main database, Redis cache, secondary databases, if any

* list of screens: with notes about operation or business logic of each screen (half-page to one page per screen)

* list of web service calls: with input and output parameters: 1-2 pages per call

* other modules: with their code if any (*e.g.* code for Alfresco, code to run in an ESB, *etc*)

The UI spec will contain

* the list of user stories: and for each story:

    * the sequence of screen names used, in a bullet-list form

* the set of screens: their pixel-perfect images or HTML+CSS

* style guide: specifying

    * which theme to use (if ready-made theme is being used)

    * which colours to use where, in hash-hex notation

    * actual set of icons in PNG/JPEG format

    * other sizes of things like margins, borders, font sizes, etc

    * font spec, including font files if any

With these three documents, we can start doing software development.