AWS Step Functions - Features & Usage - Part #01

 

So, Guys - For the last 2 months, I have been working on the AWS Ecosystem more like an AWS Solution Architect. It was a great experience to delve into the AWS world to a fair depth that includes API Gateway, Step Functions, Elastic Beanstalk and EC2. Even though I am the hard-core programmer at heart, who is always against configuration and votes for building things - AWS is a clear winner 

The series of articles will cover the Features and Usage of Step Functions. These are from the point of view of Integration with AWS API Gateway and REST Endpoints (Spring Boot) hosted on Elastic Beanstalk/EC2. Also, The discussion is from a purview of Data Management in Microservices, Transaction Management, Service Orchestration and Saga Design Pattern. 

[Important Information: The article is based on the experience of the author working on the PoC/Prototype/Pilot of AWS Step Functions, Elastic Beanstalk* (EC2), AWS API Gateway* for a Neo Bank Startup of Philippines/Singapore. It contains knowledge collated, referenced from various manuals, blogs and tutorials on the internet. The transaction shown below is a mock use-case or a mock business transaction. Any Images are copyright of their respective owners and only used for non-commercial purposes. I have verified and certify that it contains no copyright or confidential business information.]

 

The topics covered in this specific article are :

• Understanding and Exploration...
• Working of Step Functions (SF)
• [ Sample Workflow in AWS SF ]
• Amazon States Language in SF
• Configuration
• Retry, Back-off Rates
• Catcher in Amazon SF
• Parallel Flow in Amazon SF
• JSON Path Usage in Amazon SF

Working of Step Functions
After careful consideration of multiple workflow and data management pattern, we selected Step Functions for our purposes. We have the following important requirements from the selected framework:

• Service Orchestration
• Retry Mechanisms
• Catcher Mechanisms
• Saga Implementation*
• External REST API
• High Throughput
• Overall Sustainable Cost
• Existing Cloud Platform

 
AWS Step Function supports all the above requirements and is also, already existing on the AWS Cloud. AWS Step Functions supports creation of a use-case or workflow visually using the Workflow Studio. It provided multiple constructs like Start, End, Pass, Parallel, Invoke API Gateway and so on… We may invoke any AWS Service via configuration. Also, It provides a very intuitive way to configure each step in the step function. It also allows to manipulate input, output, and setup the retry and catcher mechanisms very easily. Also, each of the Step Function translated in code or definition to Amazon States Languages (ASL). ASL is similar to JSON.

Access Step Functions via [Login] > [AWS Console] > [Search for Services] > [Step Functions]

 

 
[Scope of the Article]
Note: The scope of this document is not to cover every step to create a simple workflow or step function but to understand the most important features that were explored during this Proof of Concept (PoC). We will be discussing Step Functions in terms of its integration with the following:

1. API Gateway
2. Elastic Beanstalk (EC2)
3. Spring Boot / REST

 
 
[Deployment Context]
We will discuss this in context of deploying Spring Boot/REST endpoints on Elastic Beanstalk (EC2) that is further exposed via API Gateway.


[Sample Workflow/ASL in AWS Step Function]
Click on [Workflow Studio] to launch the Step Functions visual editor. You may use the drag and drop features here to design your Step Function. It is straightforward, and I will not digress from the scope of the article.

 

 

 

Fig.1 : Sample Workflow (Mock Business Use-Case) / Amazon States Language

 
 
[Configuration]
Once you click on any step of the workflow, you will get any or all the following tabs: Configuration, Input, Output, Error Handling. In the above example, you can observe the configuration for the API Gateway REST Endpoint. Each of the specific integration endpoints have a specific way to integrate which can be referenced from: AWS Step Functions - Official Documentation - Use AWS with Other Services 

 

 

 

 

Fig.2 : Mock Transaction (Configuration of AWS API Gateway Invocation)

 

There is exhaustive documentation available for all facets of AWS Step Functions including the API. Hence, please use Google Search to locate all the required documentation as per your integration need.


[Retry, Backoff Rates]
For each of the steps, the fault tolerance and failure recovery in terms of retrying a service. From the below screen, you can see that we can specify the ‘Interval’, ‘Max Attempts’ and ‘Back off Rate’ for the purposes of a retrying a service. 

 

 

 

Fig.3 : Mock Transaction (Retry, Back-Off for AWS API Gateway Invocation)

 

 

Retrying a service maybe required due to temporary service downtime, specific business exception, runtime exception, temporary environment error or an error in a dependent service.

 
[Catcher]
Continuing the same topic – In case of a Total Failure or an Unrecoverable Error, we may need to take some form of a final action of a form notification or clean-up before we proceed out of the step function. Just as in normal programming, Step Functions allows us to perform the same. 

 

 

 

Fig.4 : Mock Transaction (Catcher for AWS API Gateway Invocation)

 

 

You may observe from the below diagram, you can catch the error and transfer control to the required fallback state. We can also filter/manipulate the result path as shown below.

[Parallel Flow]
Sometimes, there is a need to execute multiple flows in parallel and execution moves to next step only once all the parallel flows are complete. If any of the step in parallel step fails, it may be deemed as the failure of the parallel step. Rest of the options of Parallel Flow are like other actions – Configuration, Retry, Catcher, Input (Filter/Manipulation) and Output (Filter/Manipulation).

 

 

 

Fig.5 : Mock Transaction (Parallel Flow of Execution)

 

 

 
[JSON Path Usage]
JSON Path is used to manipulate/filter input and output in various actions or flows in Step Functions. To learn, understand and test our JSON Path for usage in AWS Step Functions – Developer/Architect may use the following handy link: https://jsonpath.com/

 

 

 

Fig.6 : JSON Path Usage (Show Above is a Mock Business Workflow)

 

Watch this Space for Part-02 of this Article!