Adding Zookeeper Nodes for Clickhouse From 3 to 5

Question

I was testing out adding 2 zookeeper nodes to a clickhouse cluster. However it seems whenever I had 2 new nodes all the tables that were created with 3 zookeeper nodes turns into read only mode and there's no way to recover them. Has any one have any experience adding 2 new zookeeper nodes?

I should mention that it seems to be when I take down the leader is when the table goes into read only mode.

Answer 1

You need to check that

echo stats|nc zk_ip_new_server 2181

shows

Mode: follower

or

Mode: leader

before adding these 2 ZK servers into CH configs.