'DynamoDB QueryRequest not returning all the results

    val queryReq: com.amazonaws.services.dynamodbv2.model.QueryRequest = new com.amazonaws.services.dynamodbv2.model.QueryRequest(tableName)
                    .withIndexName(...)
                    .addKeyConditionsEntry(..., new com.amazonaws.services.dynamodbv2.model.Condition()
                                        .withAttributeValueList(Seq(new com.amazonaws.services.dynamodbv2.model.AttributeValue(...)):_*)
                                        .withComparisonOperator(...))
                    .addKeyConditionsEntry(..., new com.amazonaws.services.dynamodbv2.model.Condition()
                                        .withAttributeValueList(Seq(new com.amazonaws.services.dynamodbv2.model.AttributeValue().withN(...)):_*)
                                        .withComparisonOperator(...))
                    .withScanIndexForward(scanDirection)
                    .withLimit(5000000)

This query is having withLimit = 5000000, but only returned about 3000 results. Is there a limit in how much data being returned by dynamodb in each query? If so, is there a clean way to overcome this limit?



Solution 1:[1]

I just found the solution. So I will answer my own question: I can just create a while loop that appends to the result set when getLastEvaluatedKey() is not null nor empty.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 RhinoH