'How to handle missing values in JSON when processing AWS Step Functions

I have a Step Function that takes a JSON payload. The handler accepts a Map<String, Object> and returns the same thing.

The issue I have run into is keys are lost when serializing the map back to JSON which causes me exceptions at the next step.

I have the following which transforms the input, into a common structure for the lambda:

"SearchForPersonCustomer": {
     "Type": "Task",
     "Resource": "arn:aws:states:::lambda:invoke",
     "Parameters": {
     "FunctionName": "arn:aws:lambda:us-west-2:xxxxx:function:searchForCustomer:$LATEST",
     "Payload": {
           "businessSearchInd": false,
           "taxIdentifier.$": "$.sSN",
           "firstName.$": "$.firstName",
           "middleInitial.$": "$.middleInitial",
           "lastName.$": "$.lastName",
           "birthDate.$": "$.birthDate"
         }
     }

The issue is that while the field middleInitial ("middleInitial":null) is populated in the JSON, the prior steps output no longer serializes it, so when it attempts to access it, the step function fails as follows:

The JSONPath '$.middleInitial' specified for the field 'middleInitial.$' could not be found in the input

Is there a way to handle this edge case, so that if the missing field, in this case middleInitial is just assumed to be null as it translates to the payload?

Thanks

aws-step-functions

Solution 1:^[1]

You can do one of two things: If this is coming from an API then you can set up a Method or Integration Request Transformation - this will enable you to add a default value if its missing. See this documentation.

Alternatively, if you're not dealing with an API then your best bet is to add a translation layer lambda as your first step - which accepts any event and checks for the required fields - it can refuse it if you can't default or it can default those values for you, depending on your requirements.

Step Functions itself is a "dumb" system - it can do some very basic choice and repeating logic but it cant make assumptions or parse or set things if they are missing. Thats what Lambdas are for - to handle that sort of stuff for your step function. the Step Function should just be a framework for transfering data to the right lambda/resource/service

Solution 2:^[2]

I think your issue boils down to detecting the presence of a variable. Stepfunction "Choice" was updated in 2020 to allow detection of presence of a variable (as well as detecting various types). See https://aws.amazon.com/blogs/aws/aws-step-functions-adds-updates-to-choice-state-global-access-to-context-object-dynamic-timeouts-result-selection-and-intrinsic-functions-to-amazon-states-languages/

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	lynkfox
Solution 2	Shamus Husheer

'How to handle missing values in JSON when processing AWS Step Functions

Solution 1:[1]

Solution 2:[2]

Sources

Related Questions

Solution 1:^[1]

Solution 2:^[2]