Step Functions vs Chained Lambdas: When a Business Flow Needs an Explicit State Machine

A common pattern when a business flow starts out simple is chaining Lambda functions through events: one function finishes, publishes a message to SNS or SQS, and that message triggers the next function. It works fine while the flow has two or three steps and the happy path is nearly always the one that runs. The trouble starts as the flow grows: more steps, more conditional branches, and a real need to know exactly where a given case is stuck without reconstructing that by reading CloudWatch Logs across four different functions.

We ran into this in an onboarding flow: validate data, check against an external service, create resources in two separate systems, and notify at the end. It started as four Lambdas chained by SQS events. It ended as a Step Functions state machine. The path in between is worth walking through, because not every chained flow needs this change, and forcing it where it isn't needed just adds a layer of indirection that buys nothing.

What the chain looks like before it starts to hurt

The first version of the onboarding flow looked like this: the validation Lambda receives the request, validates the payload, and if everything checks out, publishes a message to an SQS queue that triggers the external-verification Lambda. That function calls a third-party service, and if the response is positive, publishes to another queue that triggers the provisioning Lambda. And so on to the end.

Each function is simple, deploys and tests in isolation, and the code for each step knows nothing about the others. That independence is the real advantage of this pattern, and it's a genuine one: for short flows, with little branching, where a simple retry (letting SQS retry message delivery) is enough, chaining by events is still the simplest thing to operate. There's no Amazon States Language to learn, no extra component in the architecture, and debugging a happy-path case just means reading one log after another in order.

The signals that the chain no longer holds up

Three signals showed up at roughly the same time in our case.

The first was per-step retry. SQS retries the full message delivery when the consumer fails, which is fine when the step is idempotent end to end. But the provisioning Lambda created a resource in one system and then another in a second system; if the second call failed, an SQS retry re-ran the entire function, including creating the first resource, which already existed. The in-code fix was writing manual idempotency checks before each side effect -checking whether the resource already exists before creating it-, which is a reasonable practice in any distributed system, but it didn't solve the underlying problem: there was no way to retry only the second side effect without repeating the first, because from SQS and Lambda's point of view the whole handler is a single atomic unit of retry.

The second was state visibility. When a support agent asked which step a specific user's onboarding was stuck on, the answer meant correlating a request ID across the logs of four different functions in CloudWatch Logs Insights, with the added discomfort that a lost message or a Lambda that failed before logging anything left a case with no visible trace of where it had stopped.

The third was conditional branching. The real flow wasn't linear: if external verification failed for a transient reason (a provider timeout), the case needed to be retried later; if it failed for a definitive reason (an invalid document), the user needed to be notified without touching the provisioning steps. Coding that branching logic inside the verification Lambda, deciding which queue to publish to based on the error type, worked, but it moved the definition of the business flow -what happens after each possible outcome- inside the code of one individual step, instead of keeping it in one place that could be read at a glance.

What an explicit state machine solves

Step Functions doesn't change what each Lambda does; it changes who orchestrates the sequence and who holds the state between steps. The flow definition, in Amazon States Language, lives in a single document:

{
  "StartAt": "ValidateRequest",
  "States": {
    "ValidateRequest": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:validate-request",
      "Next": "ExternalVerification"
    },
    "ExternalVerification": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:external-verification",
      "Retry": [
        {
          "ErrorEquals": ["TransientError"],
          "IntervalSeconds": 30,
          "MaxAttempts": 3,
          "BackoffRate": 2.0
        }
      ],
      "Catch": [
        {
          "ErrorEquals": ["InvalidDocument"],
          "Next": "NotifyRejection"
        }
      ],
      "Next": "ProvisionSystemA"
    },
    "ProvisionSystemA": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:provision-system-a",
      "Next": "ProvisionSystemB"
    },
    "ProvisionSystemB": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:provision-system-b",
      "Next": "NotifySuccess"
    },
    "NotifySuccess": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:notify-success",
      "End": true
    },
    "NotifyRejection": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:notify-rejection",
      "End": true
    }
  }
}

The two changes that actually fixed the problems we'd been carrying live in the Retry and Catch blocks on the verification step. Exponential-backoff retry is now declarative and applies to that step alone: if ProvisionSystemA fails, Step Functions retries that specific step according to its own Retry configuration, without re-running ValidateRequest or ExternalVerification. The transient error and the definitive error stop being resolved inside the verification Lambda's code and become part of the flow definition instead: anyone reading the JSON understands, without opening the function's code, what happens for each kind of verification failure.

State visibility comes for free: the Step Functions console shows, for each execution, which state it's in or where it stopped, along with the input and output of every step. The support question about where a given case is stuck gets answered by looking up the execution by its ID, not by correlating logs across four functions.

What didn't work: migrating everything for consistency's sake

After this migration we tried applying the same pattern to a second flow, an image-processing pipeline with two steps: resize and upload to storage. The reason to migrate it wasn't any of the three signals above -there was no conditional branching, SQS's retry was enough because each step was idempotent, and nobody was asking which step a given case was stuck on because the whole flow took seconds, not hours. We migrated it only because the pattern was fresh and it seemed more consistent to have everything under the same orchestration mechanism.

The result was worse than the original version: a two-step state machine that added one more component to deploy and version, without solving any real problem we had, and with extra latency from Step Functions orchestrating between steps that simply didn't exist in the original event chain. We reverted that flow back to chaining by events.

The real cost of each approach

The two pricing models differ in shape, not just in size: Step Functions Standard charges per state transition executed, while Lambda and SQS charge per invocation, execution time, and messages processed. For a very high-volume flow with few steps, adding up state transitions introduces a cost that the event chain doesn't have. For a flow with retry logic and conditional branching that would otherwise have to be written and maintained by hand inside each Lambda's code, that cost competes against the engineering cost of writing and sustaining that logic -not against zero. It isn't a calculation that gets settled by looking only at the price per transition: it has to be weighed against how much the state logic costs, in development time and in incidents, when it's left scattered across the code of each step instead.

The rule we ended up using

If a flow is linear or close to it, with retries the queue itself handles and no operational need to see which step a given case is on, we leave it as Lambdas chained by events. The moment one of the three signals shows up -a retry that needs to apply to one specific step without repeating the ones before it, conditional branching that defines the business flow, or a real need for someone outside the development team to see what state a case is in- we migrate that specific flow to Step Functions, without dragging along every other flow that still doesn't show those signals.