Global

# Web service design standards

{{>toc}}

## Web services: basic rules

* Web service calls will expect requests using the `POST` method, not `GET`, typically. We will usually not design URLs to communicate request parameters. The only exception will be in "deep URL" cases where we need to supply a link or button somewhere which will take the user directly to a specific object instance within a module, for instance, the edit screen of a specific voucher within the FA module. In such cases, we can append the object ID to the URL, and use `GET`.

* All JSON data structures will have member names in full lowercase. No mixed-case will be used.

* We will use only HTTPS for all web services traffic. Calls must be set up to fail if HTTP is attempted; HTTP must not be redirected to HTTPS.

* HTTP response codes should not be examined by the caller to decide success or failure of the call -- any HTTP code in the range `200` to `299` should be accepted as success. HTTP protocol response codes are supposed to indicate protocol level and basic transport level issues -- not application level success or failure.

## Web service request format

This is one example:

``` json

{

    "data": {

        "goal_id": 23

    }

}

```

* **`data`** is the request data object. Each web service will define its own data object, with some mandatory and some optional fields. The exact contents of the `data` block, the exact member names, *etc*, will change from call to call. Here the example request has just one member: `goal_id`.

In addition, there will be three other parameters which will be supplied with (almost) all requests, (almost) always, irrespective of whether we use GET or POST. These are:

* the JWT: this token will be passed in the HTTP header. The server will receive the request, pull out the token, and validate it. Only if it is valid will the web service call code begin its work

* `ver`: an integer version number. This will go either in the HTTP header or in the URL (with a GET request). When a new call is released, it will expect `ver` to carry the value `1` always. As new and incompatible versions are released, the web service call will expect to receive `1`, or `2`, or `3`, *etc*, and switch to that version of its code. This allows multiple parallel versions of the code to serve multiple generations of front-end clients. If the `ver` is to be passed in the URL, it will be embedded in the following way: `https://app.datakatana.com/mis/v1/gettrialbalance?branch=402`. In this example, `mis` is the name of the application, and the request is hitting the `v1` release of the application, and in that, the access is to the `gettrialbalance` web service call, and it is being told to process the trial balance for branch `402`. This sort of URL can be used with a GET method. The `ver` value will need to be extracted from the URL and passed to the code as a header parameter.

* `trace ID`: an opaque string, hopefully unique in time and space. This will be set in the HTTP header by the client code, and will be passed to the web service call code as a parameter. It will be logged by the code on the server, and will be passed along for all inter-service calls, messages, and inter-module interaction, to track all the activities of one processing thread from another. A typical name for this header will be `X-CBSSOportal-Trace-ID`, where `CBSSOportal` is the name of the application being developed. So the full header line may be something like 

```

X-CBSSOportal-Trace-ID: cfb8ed3e-619f-401c-af6e-0e0a8e9a066d

```

where `CBSSOportal` is the name of the application, and the trace ID has been generated using UUID tools.

These are not shown in the request example, because they're not part of the *request body* but are there in the *request header*. The JWT may be omitted in cases where the call is unauthenticated.

## Web service response format

A consistent JSON data structure will be returned by the API for success or errors. For successes, the following JSON structure will be returned:

``` javascript

{

    "status": "ok",

    "data": {

    /* Application-specific data payload goes here. */

    },

    "messages": [] /* Or optional success message */

}

```

The contents of `data` for each web service call is provided in its detailed documentation. Note how the `message` block is an empty array here. The `status` attribute will always be returned in every response.

For errors, the JSON structure to be returned must be language independent, so that the client code can make sense of the error in an language-independent way, and still display a meaningful error message in the language of the end-user.

``` json

{

    "status": "error",

    "data": {},

    "messages": [{

        "msgid": 235,

        "errcode": "toobig",

        "field": "maxdelay",

        "vals": [ "7", "3" ]

    }, {

        "msgid": 45,

        "errcode": "missing",

        "field": "fullname"

    }]

}

```

The `data` block is empty in case of errors, and the error messages will be listed in `messages`.

The `data` block is an object, not an array. And the `messages` array is always an array, never an object.

In the `messages` array, the two mandatory fields are:

* `errcode`: a one-word value, giving the error code for the error. The list of error codes the system can send back is listed in the error dictionary. This code is useful for the client code programmer or front-end programmer -- she can see from the value of `errcode` what was the nature of the error, and take action. For instance, some types of error codes indicate internal system errors or statuses, like `auth` or `trylater`, whereas a lot of other error codes indicate errors in user-submitted data. The front-end code can distinguish these errors into these categories and act accordingly.

* `msgid`: an integer, giving the index for the error message in human readable form. This index, combined with the language code the client software will apply based on the language being used in the front-end, will be used to pull out a message template string from a message dictionary. If, for instance, six languages are being supported by the front-end, then there will be six versions of errid `235` in the six languages.

The other two fields in each `messages` element are optional.

* `field`: gives a field name, indicating a field in the request which triggered the error. If the error is not field-specific, then this attribute will be omitted.

* `vals`: will be an array of strings, giving one or more values which will be plugged into the message to be displayed to the end-user, as explained below in examples. If `field` is omitted, then `vals` will certainly be omitted.

And example of an error message object:

``` json

    {

        "msgid": 235,

        "errcode": "toobig",

        "field": "maxdelay",

        "vals": [ "7", "3" ]

    }

```

This may be an error to indicate that a value of `7` was supplied in the field `maxdelay` in the request, where the max value permitted for this attribute is `3`. The error code is `toobig`, which tells the programmer that the general nature of the error is that some attribute in the request had a value which was too big for the server to accept. If you look up `msgid` 235 for English, you may find that it will be the string

```

"@<field>@ has the value @<val_0>@, exceeds maximum value @<val_1>@"

```

This tells the client code that it can pull out message 235 from the text dictionary, patch the value of `field` in the first placeholder, `vals[0]` in the second, `vals[1]` in the third, and display it to the user, and be fairly certain that the resulting message will be meaningful to the human. If this was an English language user, she would see

```

maxdelay has the value 7, exceeds maximum value 3

```

If you look up `msgid` 235 for Bengali, you may find the string

```

@<field>@এর মান @<val_0>@, সর্বোচ্চ মান @<val_1>@ ছাড়িয়ে গেছে

```

After patching the three placeholders, the client code will get

```

maxdelay এর মান 7, সর্বোচ্চ মান 3 ছাড়িয়ে গেছে

```

which is meaningful for a Bengali user.

It is possible that the UI message designer may decide to omit some or all of the placeholders from a message template (in one language, some languages, or all languages), and the `message` object will still be usable. For instance, the text template may say

```

Maximum batch delay has the value @<val_0>@, cannot exceed maximum value @<val_1>@

```

If the message object is processed in the front-end code and patched into this template, it will result in

```

Maximum batch delay has the value 7, cannot exceed maximum value 3

```

The `@<field>@` tag was omitted, but the message block still works well and the resulting message is still meaningful. It's up to the message designer for the front-end.

A second example:

``` json

    {

        "msgid": 45,

        "errcode": "missing",

        "field": "fullname"

    }

```

This may be an error to indicate that a mandatory attribute was missing in the request. The error code says `missing`, which clearly tells the client code software that the nature of the error is a missing mandatory field. The `msgid` is 45, which will point to a string in the text dictionary. And the `field` field gives the name of the attribute which should have been present. The message string 45 may be

```

Mandatory field @<field>@ missing

```

## Reporting errors by web services

Each error reported has two parts:

* the error code

* the human readable error message

The errcode is always going to be one word, always in lower case, with letters, digits, and/or underscore characters.

The errcode is the real indicator of the type of error, and must be used by the client code. It may handle the error, it may retry the operation, it may branch into a different path in the code, it may select a message to show the end-user, _etc_, as needed. The error message supplied by the web service is meant for the programmer of the client software to understand and debug the web service call. It is _not_ meant to be used for end-user display.

In multi-lingual user interfaces, the error message must be indexed to the combination of `(language, errcode)` and the appropriate language-specific error message must be selected from a table and displayed to end-users. The responsibility of designing and selecting meaningful error messages in the correct language lies with the client software. The client software dev team must build error message tables in each supported language. The error message supplied by the web service will always be in English, and may not have appropriate wording to display directly to the end-user. Its audience is the programmer of the client software.

Common error codes are:

* `auth`: for login calls, this means that credentials are invalid. For all authentication calls, this error indicates that the user does not have access rights to perform the action she is attempting

* `data_invalid`: one of the mandatory parameters is missing, or some unexpected parameter has been added, or the value of some parameter is not valid

* `trylater`: the server is receiving too many requests from the client's IP address and is rate throttling the call (typically for unauthenticated calls)

## Authentication

Typically, the web services layer will need to authenticate clients which are invoking them. If it's a human-user oriented application, then the human will need to be authenticated sometime. If the web services are invoked by devices, then each device will need to be authenticated at least once, maybe periodically, to ensure that the API does not remain completely open to outside access.

Therefore, there will be at least one `login()` web service in every application, where the client will identify itself. 

**User-based authentication**: The `login()` call may accept a username and password, or there may be a back-end OTP loop where the user just submits his username and the system sends an OTP to his registered mobile number which is later submitted through a second web service.

Finally, on successful authentication, the web service will return a JWT token to the client. This JWT token will need to be submitted as part of the fixed header fields of all subsequent web services which expect authenticated access.

**Device-based authentication**: The `login()` call may be designed to require a unique ID. This ID may be generated from some unique aspects of the device hardware (_e.g._ 

```

dmidecode | grep relevant-lines | md5sum

```

on Linux, as a very rough mechanism, or some ROM-embedded unique serial number embedded there by the hardware manufacturer, or some other naturally occurring unique ID like Ethernet MAC address.

This ID will uniquely (we assume) identify one device. The web service will check to see if this device is authorised to access the system, and may do some back-end cross-checks using OTP to some authorised person's mobile phone or some other two-factor authentication. The device may then have to make a second call to submit the second factor for the authentication to complete. Finally, when the web service is satisfied that all is well, it will issue a JWT token to the client. This client will need to be included as part of the fixed header of all subsequent web service requests. All web services which expected authenticated access will first check the validity of the token before processing other parameters. If the token is invalid, the web service will respond with a `authtoken_invalid` error.

**JWT token replacement**: All JWT tokens expire, and it is a good idea to let them refresh, for security reasons. When a web service receives an expired token in its request, it will respond immediately with a special error code indicating `authtoken_expired` without looking at other parameters. What happens next will depend on the application design.

* **Token replacement**: One approach will be to issue a fresh token in lieu of the old one. There will be a special web service whose only job will be to issue a new token. It will take the old token, check all other fields for validity, check that the internal user tables or device masters still permit this client to continue using the services, and then it will create and send back a new token with a fresh expiry date. The client will receive this new token and will use it henceforth for all accesses. To use this approach, the client code will have to be designed to detect `authtoken_expired` errors and automatically make this call for generating a fresh token.

* **Fresh authentication**: For higher security, it may be necessary, specially in user-based applications, to do a full authentication afresh. In this approach, the user will simply be logged out and will be asked to "sign in again". Therefore, whatever was done the first time to authenticate the user or device will now have to be re-done, and manual intervention may be needed at the client end.

## API versions

Any web service (we will refer to it as "service" here) which is exposed to multiple clients (software systems which send web services requests to our service) will, over time, have to serve old clients and new ones.

Our service will grow in features and functionality, and the semantics of some web services may change. For instance, a particular web service may have been released initially with just three parameters in its input, but a fourth parameter was felt necessary after two years of service. Some older clients will still access the web service with three parameters and newer clients will pass four parameters to it. _C'est la vie_. One of the most obvious examples of this is when mobile apps access a web service. Some phones will have older releases of the mobile app and others will have newer releases. This makes it necessary for the web service to serve all generations of clients without breaking any.

One way to handle this is by adding an "API version number" parameter to each web service. This will be a mandatory parameter from Day 1. Initial versions of clients will all be designed to use **`"ver": 1`** in their requests. When the software on the server-side application is upgraded and the semantics of a web service change, then it will support the old semantics under **`"ver" = 1`** and newer semantics with **`"ver" = 2`**. Newer clients which are aware of the updated semantics will use **`ver=2`** in their requests, with the new semantics. Older clients will not be aware of any changes.

The version number applies to each web service in isolation. The service as a whole may have versions 1 and 2 supported for some calls, 1, 2 and 3 for other calls, and just 1 for calls which have not changed at all.

This will go on, and new generations of web services will keep getting released. This will allow a single service to service multiple generations of clients at the same time.