Migrate Service Fabric Reliable Collections to Kubernetes

Question

We are in the process of migrating our Service Fabric services to Kubernetes. Most of them were "stateless" services and were easy to migrate. However, we have one "stateful" service that uses SF's Reliable Collections pretty heavily.

K8s has Statefulsets, but that's not really comparable to SF's reliable collections.

Is there a .NET library or other solution to implement something similar to SF's Reliable Collections in K8s?

Answer 1

However, they don't solve the coordination problem (saving data on the primary instance will automatically replicate to the others for recovery when the primary instance dies). That's what SF reliable collections does out of the box.

StatefulSets are valuable for applications that require one or more of the following.

• Stable (persistent), unique network identifiers.
• Stable, persistent storage.
• Ordered, graceful deployment and scaling.
• Ordered, automated rolling updates.

If your application doesn't require any of these, you should deploy your application using a Deployment.

There is a good Kubernetes guide on how to Run a Replicated Stateful Application here.

This page shows how to run a replicated stateful application using a StatefulSet controller. This application is a replicated MySQL database. The example topology has a single primary server and multiple replicas, using asynchronous row-based replication.

The StatefulSet controller starts Pods one at a time, in order by their ordinal index. It waits until each Pod reports being Ready before starting the next one. And it is used to perform orderly startup of MySQL replication by running Init Containers.

Operators can help

While operators are not necessary, they can help run stateful apps on Kubernetes with features like application-level HA management, backups, and restore.

You can use existing Operators or develop your own. The operator package includes all the configurations needed to deploy and manage the application from a Kubernetes point of view – from a StatefulSet to be used to any required storage, rollout strategies, persistence and affinity configuration, and more. Kubernetes will then rely on the operator to validate instances of the application against the specification to ensure it runs in the same way across instances in all clusters it is deployed in.

Some DB operators:

• You can deploy MySQL database using Kubernetes operator developed by Oracle (currently is in a preview state): https://github.com/mysql/mysql-operator
• There’s also a PostgreSQL operator by Crunchydata to deploy PostgreSQL to Kubernetes: https://github.com/CrunchyData/postgres-operator
• MongoDB owns an operator to deploy MongoDB Enterprise to a Kubernetes cluster: https://github.com/mongodb/mongodb-enterprise-kubernetes

You can find ready-made operators on OperatorHub.io to suit your use case.