AWS Step Functions Examples: Real Workflows for Order Processing, ETL, and ML Pipelines

The best way to understand Step Functions is to read working state machines, not descriptions of them. This article presents three complete, real-world workflow examples with Amazon States Language (ASL) definitions and explanations of the design decisions behind each one.

Example 1: E-Commerce Order Processing

An order arrives through an API and must be validated, checked against inventory, charged, and then fulfilled. Each step can fail independently, and the failure handling is different at each stage.

[Start]
   |
   v
[ValidateOrder]
   |
   +-- ValidationError --> [SendRejectionEmail] --> [Fail]
   |
   v
[CheckInventory]
   |
   +-- OutOfStock --> [AddToBackorderQueue] --> [Succeed]
   |
   v
[ChargePayment]
   |
   +-- PaymentDeclined --> [ReleaseInventoryHold] --> [SendDeclineNotification] --> [Fail]
   +-- PaymentTimeout  --> [Retry x3 with backoff]
   |
   v
[FulfillOrder] (Parallel)
   |
   +-- [UpdateWarehouseSystem]
   +-- [SendConfirmationEmail]
   +-- [UpdateLoyaltyPoints]
   |
   v (all three must complete)
[Succeed]

ASL Definition:

{
  "Comment": "E-commerce order processing workflow",
  "StartAt": "ValidateOrder",
  "States": {
    "ValidateOrder": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789:function:ValidateOrder",
      "Catch": [
        {
          "ErrorEquals": ["ValidationError"],
          "Next": "SendRejectionEmail",
          "ResultPath": "$.error"
        }
      ],
      "Next": "CheckInventory"
    },
    "CheckInventory": {
      "Type": "Task",
      "Resource": "arn:aws:states:::dynamodb:getItem",
      "Parameters": {
        "TableName": "inventory",
        "Key": {
          "sku": {"S.$": "$.sku"}
        }
      },
      "ResultSelector": {
        "stock_count.$": "$.Item.stock.N"
      },
      "ResultPath": "$.inventory",
      "Next": "IsInStock"
    },
    "IsInStock": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.inventory.stock_count",
          "NumericGreaterThan": 0,
          "Next": "ChargePayment"
        }
      ],
      "Default": "AddToBackorderQueue"
    },
    "ChargePayment": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789:function:ChargePayment",
      "Retry": [
        {
          "ErrorEquals": ["PaymentTimeout"],
          "IntervalSeconds": 3,
          "MaxAttempts": 3,
          "BackoffRate": 2
        }
      ],
      "Catch": [
        {
          "ErrorEquals": ["PaymentDeclined"],
          "Next": "ReleaseInventoryHold"
        }
      ],
      "Next": "FulfillOrder"
    },
    "FulfillOrder": {
      "Type": "Parallel",
      "Branches": [
        {
          "StartAt": "UpdateWarehouse",
          "States": {
            "UpdateWarehouse": {
              "Type": "Task",
              "Resource": "arn:aws:lambda:us-east-1:123456789:function:UpdateWarehouse",
              "End": true
            }
          }
        },
        {
          "StartAt": "SendConfirmationEmail",
          "States": {
            "SendConfirmationEmail": {
              "Type": "Task",
              "Resource": "arn:aws:states:::sns:publish",
              "Parameters": {
                "TopicArn": "arn:aws:sns:us-east-1:123456789:order-confirmations",
                "Message.$": "States.Format('Order {} confirmed', $.order_id)"
              },
              "End": true
            }
          }
        }
      ],
      "Next": "OrderComplete"
    },
    "OrderComplete": {
      "Type": "Succeed"
    },
    "SendRejectionEmail": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789:function:SendEmail",
      "Parameters": {
        "template": "order_rejected",
        "order_id.$": "$.order_id"
      },
      "Next": "OrderFailed"
    },
    "ReleaseInventoryHold": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789:function:ReleaseInventory",
      "Next": "OrderFailed"
    },
    "AddToBackorderQueue": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sqs:sendMessage",
      "Parameters": {
        "QueueUrl": "https://sqs.us-east-1.amazonaws.com/123456789/backorders",
        "MessageBody.$": "States.JsonToString($)"
      },
      "End": true
    },
    "OrderFailed": {
      "Type": "Fail",
      "Error": "OrderProcessingFailed",
      "Cause": "See execution history for details"
    }
  }
}

Design notes:

The CheckInventory state calls DynamoDB directly (no Lambda) using the dynamodb:getItem optimised integration
ResultPath: "$.inventory" merges the DynamoDB result into the state input as a new field rather than replacing it
The Parallel state in FulfillOrder runs warehouse update and email simultaneously and waits for both to complete before proceeding
States.Format is a built-in Step Functions function for string interpolation

Example 2: Data ETL Pipeline

A nightly pipeline extracts data from three source systems, transforms each one independently in parallel, and then loads the results into a data warehouse. If any transform fails, the load step is skipped and an alert is sent.

[Start: 02:00 UTC trigger]
         |
         v
[StartCrawlers] (Parallel)
    |
    +-- [RunCrawler: sales-raw]
    +-- [RunCrawler: inventory-raw]
    +-- [RunCrawler: customers-raw]
    |
    v (all three must complete)
[TransformData] (Parallel)
    |
    +-- [GlueJob: transform-sales]
    +-- [GlueJob: transform-inventory]
    +-- [GlueJob: transform-customers]
    |
    v (all three must complete)
[LoadToRedshift]  -- Glue job: load merged data
         |
         v
[RunPostLoadQualityCheck]
         |
         v
[Succeed]

{
  "Comment": "Nightly data ETL pipeline",
  "StartAt": "CrawlSources",
  "States": {
    "CrawlSources": {
      "Type": "Parallel",
      "Branches": [
        {
          "StartAt": "CrawlSales",
          "States": {
            "CrawlSales": {
              "Type": "Task",
              "Resource": "arn:aws:states:::glue:startCrawler.sync",
              "Parameters": {"Name": "sales-raw-crawler"},
              "End": true
            }
          }
        },
        {
          "StartAt": "CrawlInventory",
          "States": {
            "CrawlInventory": {
              "Type": "Task",
              "Resource": "arn:aws:states:::glue:startCrawler.sync",
              "Parameters": {"Name": "inventory-raw-crawler"},
              "End": true
            }
          }
        }
      ],
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "Next": "AlertOnFailure"
        }
      ],
      "Next": "RunTransformJobs"
    },
    "RunTransformJobs": {
      "Type": "Parallel",
      "Branches": [
        {
          "StartAt": "TransformSales",
          "States": {
            "TransformSales": {
              "Type": "Task",
              "Resource": "arn:aws:states:::glue:startJobRun.sync",
              "Parameters": {
                "JobName": "transform-sales",
                "Arguments": {
                  "--run_date.$": "$.run_date"
                }
              },
              "End": true
            }
          }
        },
        {
          "StartAt": "TransformInventory",
          "States": {
            "TransformInventory": {
              "Type": "Task",
              "Resource": "arn:aws:states:::glue:startJobRun.sync",
              "Parameters": {
                "JobName": "transform-inventory",
                "Arguments": {
                  "--run_date.$": "$.run_date"
                }
              },
              "End": true
            }
          }
        }
      ],
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "Next": "AlertOnFailure"
        }
      ],
      "Next": "LoadToRedshift"
    },
    "LoadToRedshift": {
      "Type": "Task",
      "Resource": "arn:aws:states:::glue:startJobRun.sync",
      "Parameters": {
        "JobName": "load-to-redshift",
        "Arguments": {
          "--run_date.$": "$.run_date"
        }
      },
      "Next": "PipelineComplete"
    },
    "AlertOnFailure": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sns:publish",
      "Parameters": {
        "TopicArn": "arn:aws:sns:us-east-1:123456789:etl-alerts",
        "Message": "ETL pipeline failed. Check Step Functions execution for details."
      },
      "Next": "PipelineFailed"
    },
    "PipelineComplete": {"Type": "Succeed"},
    "PipelineFailed": {
      "Type": "Fail",
      "Error": "ETLPipelineFailed"
    }
  }
}

Design notes:

glue:startJobRun.sync waits for the Glue job to complete before proceeding — without .sync, Step Functions would fire the job and immediately move to the next state
The Catch on the Parallel state catches failures from any branch — if even one transform job fails, the pipeline goes to AlertOnFailure rather than attempting the load step

Example 3: ML Model Training and Deployment

A weekly workflow retrains a fraud detection model on the latest transaction data, evaluates it against the current production model, and deploys it only if it is better.

[Start: Monday 03:00 UTC]
         |
         v
[PrepareTrainingData]  -- Glue job
         |
         v
[TrainModel]  -- SageMaker training job (may take 2-3 hours)
         |
         v
[EvaluateModel]  -- Lambda: compare new vs current AUC score
         |
         |-- New model better --> [DeployToStaging]
         |-- Not better       --> [LogResult: no deployment] --> [Succeed]
         |
         v
[DeployToStaging]  -- SageMaker endpoint update
         |
         v
[WaitForSmokeTest]  -- Wait 30 minutes
         |
         v
[RunSmokeTests]  -- Lambda: check endpoint health
         |
         |-- Tests pass --> [PromoteToProduction]
         |-- Tests fail --> [RollbackStaging] --> [AlertTeam]
         |
         v
[Succeed]

{
  "Comment": "ML model weekly retraining pipeline",
  "StartAt": "PrepareTrainingData",
  "States": {
    "PrepareTrainingData": {
      "Type": "Task",
      "Resource": "arn:aws:states:::glue:startJobRun.sync",
      "Parameters": {"JobName": "prepare-fraud-training-data"},
      "Next": "TrainModel"
    },
    "TrainModel": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sagemaker:createTrainingJob.sync",
      "Parameters": {
        "TrainingJobName.$": "States.Format('fraud-model-{}', $.run_date)",
        "AlgorithmSpecification": {
          "TrainingImage": "382416733822.dkr.ecr.us-east-1.amazonaws.com/xgboost:1",
          "TrainingInputMode": "File"
        },
        "RoleArn": "arn:aws:iam::123456789:role/SageMakerRole",
        "OutputDataConfig": {
          "S3OutputPath": "s3://ml-artifacts/fraud-model/"
        },
        "ResourceConfig": {
          "InstanceType": "ml.m5.2xlarge",
          "InstanceCount": 1,
          "VolumeSizeInGB": 50
        },
        "StoppingCondition": {"MaxRuntimeInSeconds": 14400}
      },
      "Next": "EvaluateModel"
    },
    "EvaluateModel": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789:function:EvaluateModel",
      "Next": "IsNewModelBetter"
    },
    "IsNewModelBetter": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.new_model_better",
          "BooleanEquals": true,
          "Next": "DeployToStaging"
        }
      ],
      "Default": "LogNoDeployment"
    },
    "DeployToStaging": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sagemaker:updateEndpoint.sync",
      "Parameters": {
        "EndpointName": "fraud-detection-staging",
        "EndpointConfigName.$": "$.new_endpoint_config"
      },
      "Next": "WaitForSmokeTests"
    },
    "WaitForSmokeTests": {
      "Type": "Wait",
      "Seconds": 1800,
      "Next": "RunSmokeTests"
    },
    "RunSmokeTests": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789:function:SmokeTestEndpoint",
      "Next": "DidTestsPass"
    },
    "DidTestsPass": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.tests_passed",
          "BooleanEquals": true,
          "Next": "PromoteToProduction"
        }
      ],
      "Default": "RollbackStaging"
    },
    "PromoteToProduction": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sagemaker:updateEndpoint.sync",
      "Parameters": {
        "EndpointName": "fraud-detection-production",
        "EndpointConfigName.$": "$.new_endpoint_config"
      },
      "Next": "TrainingComplete"
    },
    "LogNoDeployment": {
      "Type": "Task",
      "Resource": "arn:aws:states:::dynamodb:putItem",
      "Parameters": {
        "TableName": "model-registry",
        "Item": {
          "run_date": {"S.$": "$.run_date"},
          "action": {"S": "no_deployment"},
          "reason": {"S": "new model did not outperform current"}
        }
      },
      "End": true
    },
    "RollbackStaging": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789:function:RollbackEndpoint",
      "Next": "AlertOnTestFailure"
    },
    "AlertOnTestFailure": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sns:publish",
      "Parameters": {
        "TopicArn": "arn:aws:sns:us-east-1:123456789:ml-alerts",
        "Message": "Fraud model smoke tests failed. Staging rolled back."
      },
      "End": true
    },
    "TrainingComplete": {"Type": "Succeed"}
  }
}

Design notes:

The Wait state pauses for 30 minutes without consuming Lambda execution time or Step Functions per-second charges during the wait period
The Choice state for model comparison uses a boolean output from the evaluation Lambda — clean and testable
sagemaker:createTrainingJob.sync waits for the training job to complete before moving on, even if training takes hours — this is a Standard Workflow use case

Common Patterns Illustrated

Pattern	State Type	Example Usage
Sequential steps	Task + Next	Order validation chain
Branching logic	Choice	Route based on stock level or model quality
Parallel work	Parallel	Run multiple Glue jobs simultaneously
Retry with backoff	Retry block	Payment API timeout
Error diversion	Catch block	Route to alert state on failure
Timed pause	Wait	Wait after deployment before smoke test
Loop over array	Map	Process each item in an order
External approval	Task + waitForTaskToken	Human review in document workflow