spiral construction

DevOps: REST API Execution Through Bash Shell Scripting II

A Simple Framework (Extension)

This is the second part of the paper proposing a framework that enables DevOps teams to issue REST API calls via bash shell scripts. This part discusses the extension beyond the core of the proposed framework. The third part shares some lessons learned from real world applications.

The Basics

The basic set of operations that address the REST API endpoints directly is only the beginning. To increase the readability and development efficiency, you may soon find yourself having the need to create other higher-level operations. These operations do not invoke the curl command, rather they either produce or consume the JSON data for the operations that actually do.

While creating these operations, we need to consider the usage of these operations and attempt to minimize the performance impacts they may bring to the overall framework. In addition, we need to account for other practical considerations include the uniqueness and other possible runtime conflicts. Some applications actually use flags to allow objects to be disabled to prevent runtime conflicts.

Common Higher-Level Operations

Let us first look at the exist.sh, cleanse.sh and maketemp.sh.

exist.sh detects whether an object exists and indicate the status via the exit code while outputs the exact content the read.sh does to save the need to redundantly invoke the read.sh again afterwards. The curl GET operation in the read.sh alone does not give us the semantical meaning about the existence of the intended objects.

cleanse.sh is meant to prepare a JSON formatted data for creation/update purposes. For your API, it may be a no-op. For applications that implement both internal and human-readable unique identifiers, it is usually meant to allow the human-readable identifiers to be modified while maintaining existing relationships among related/linked objects. To prevent consistency issues that may rise throughout the life cycle of these objects, the cleanse.sh is used to manipulate its JSON input to remove or translate the internal attributes into human readable attributes.

maketemp.sh takes an existing object to create a template bash shell script that only parameterized the naming attribute. Users could further tailor its result to include more parameters that suits their own needs.

It may look like:

This script uses exist.sh to retrieve the JSON data of an existing object. The series of echoes print out a shell script that accepts a parameter for the new object name. The retrieved JSON data is first cleansed and the naming attribute is added with a parameterized value in the generated script.

Multi-level Structure Operations

For applications that use a multi-level structure, a REST API endpoint could require multiple parameters to address a particular object. For example, Layer7 SiteMinder uses domain and realm to identify how users of a domain use resources. Reading a realm object that belongs to a particular domain could look like “bash realms/read.sh DemoDomain DemoRealm”.

The common operations mentioned earlier may apply to applications that use a multi-level structure as well. However, you are encouraged to take into account the aggregated nature of the objects that come with the relationships.

Supporting Operations

In addition to the aforementioned cleanse.sh that is application- and actually object type- specific, there may be other more generic supporting operations needed to support the overall framework. These supporting operations could be stored under a special subdirectory.

Beyond the Basics

While we were working on applications that adopted this framework, we thought the common operations we constructed could be shared along with the framework we started.

Other High Level Operations

We were creating applications that involved testing different settings of certain objects. To properly preserve different sets of settings and gain the ability to quickly switch the settings for demonstration purposes, we developed save.sh, copy.sh and rename.sh:

save.sh ExistingObj – saves an existing object to a new object with a new random unique identifier. The save.sh could look like:

 MYPATH=$(dirname "$0")
 APP='.'$(date +"%Y%m%d%N")
 EXIST=$(bash "${MYPATH}/exist.sh" "$NAME") || exit $?
 echo "$EXIST" | bash "${MYPATH}/cleanse.sh" | \
     jq --arg app "$APP" \
     '.Name = (.Name + $app)' >"$JSON"
 bash "${MYPATH}/create.sh" "$JSON"

In this case, the script detects the subdirectory where the save.sh resides and invokes the exist.sh, cleanse.sh and create.sh from the same directory. This allows the same script to be reused for  other objects that has a similar subdirectory structure.

copy.sh ToObj FromObj – takes the content of an existing object, cleanses it, changes both internal and human-readable identifiers to match another object, and then uses update.sh to modify the target object while retaining all existing incoming relationships.

rename.sh OldName NewName – change the naming identifier of an existing object to a new name.

Open Source Computing

Since there is really no proprietary stuff within a framework of this sort, the decision to set it up for Open Source should not be difficult. It would actually encourage the customers of the API to help explore the application side of the framework. It also addresses the kind of stylish or even cultural concerns people may raise. For example, a reasonable question could be “Why did you construct the save.sh the way you did? Would it not be better to have the save.sh to take the second parameter as the new name to be saved?”

Hence, we would suggest using a subdirectory likes “ext” under each object type subdirectory to house these non-basic high level operations, so you would have a objtype1/ext/save.sh for this version and allow other efforts to have something like objectype1/myeffort/save.sh for the particular version they would rather use.

Real World Lessons

In the next part of this paper, we want to share with you what we learned through adopting this framework on both the Layer7 SiteMinder and Layer7 Directory and provide other suggestions you could apply to your own REST API shell script framework.

Recent Posts