'How does a global state store differ from a normal state store in kafka?

How does a global State Store differ from a normal state store ?

Has a global state store a copy of the data in all the instances running on different machines ? How does it behave in case of a restart ? Because a global state store doesn't use any change-log topic for restoration, in my scenario, the source topic from the global store has no key.

javaapache-kafkaapache-kafka-streams


Solution 1:[1]

  1. Is Global state store has copy of data in all the instance running on different machine?

    Yes.

  2. How it behave in case of restart because global state store doesn't use any change-log topic for restore in my scenario the source topic from the global store has no key:
    • GlobalKTable disable logging by default so it'll not push changelog to the changelog topic for GlobalKTable (it still creates the changelog topic though). You have to re-populate data to GlobalKTable from an input topic which enables log compaction (cleanup.policy=compact) which message's key is the key you want to lookup in your GlobalKTable. Kafka Stream will just re-populate data from the input topic to GlobalKTable when you restart application.
    • in my scenario the source topic from the global store has no key: you have to map your source topic to the new topic which I mentioned above using a KeyValueMapper, and enable log compaction on the output topic.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1

Related Questions