KStream disable local state strore

Question

I am using Kafka Stream with Spring cloud Stream. Our application is stateful as it does some aggregation. When I run the app, I see the below ERROR message on the console. I am running this app in a Remote Desktop Windows machine.

Failed to change permissions for the directory C:\Users\andy\project\tmp
Failed to change permissions for the directory C:\Users\andy\project\tmp\my-local-local

But when the same code is deployed in a Linux box, I don't see the error. So I assume it an access issue.

As per our company policy, we do not have access to the change a folder's permission and hence chmod 777 did not work as well.

My question is, is there a way to disable creating the state store locally and instead use the Kafka change log topic to maintain the state. I understand this is not ideal, but it only for my local development. TIA.

Answer 1

You could try to use in-memory state stores instead of the default persistent state stores.

You can do that by providing a state store supplier for in-memory state stores to your stateful operations:

StateStoreSupplier storeSupplier = Stores.inMemoryKeyValueStore("in-mem");
StreamsBuilder builder = stream("input-topic")
    .groupByKey()
    .count(Materialized.as(storeSupplier))

From Apache Kafka 3.2 onwards, you can set the store type in the stateful operation without the need for a state store supplier:

StreamsBuilder builder = stream("input-topic")
    .groupByKey()
    .count(Materialized.withStoreType(StoreType.IN_MEMORY))

Or you can set the state store type globally with:

props.put(StreamsConfig.DEFAULT_DSL_STORE_CONFIG, StreamsConfig.IN_MEMORY);