5 Crucial Facts About Kubernetes Server-Side Sharded List and Watch

As Kubernetes clusters balloon to tens of thousands of nodes, traditional watch mechanisms become a bottleneck. Controllers drowning in irrelevant events waste resources. Enter server-side sharded list and watch, an alpha feature in v1.36 that promises to revolutionize data streaming. Here's what you need to know.

1. The Scaling Challenge: Why Controllers Struggle

In large clusters, controllers such as those managing Pods must process every event from the API server. With high-cardinality resources, each replica of a horizontally scaled controller receives the full event stream. This means every replica deserializes all objects, even those it doesn't manage. The CPU, memory, and network costs scale linearly with the number of replicas, not with the actual workload. For instance, a 10-replica controller processing 100,000 Pod events wastes 90% of resources on irrelevant data. This scaling wall limits cluster growth and increases operational costs. Server-side sharding directly addresses this by filtering events at the source.

2. Client-Side Sharding: A Partial Fix

Some controllers, like kube-state-metrics, already implement client-side sharding. Each replica is assigned a portion of the keyspace and discards objects outside its range. While this distributes the workload logically, it does not reduce the data volume from the API server. Network bandwidth grows with the number of replicas because each replica still downloads the full event stream. Similarly, CPU cycles spent on deserialization are wasted for discarded events. This approach is functionally correct but inefficient at scale. Server-side sharding solves this by moving the filtering logic upstream into the API server itself, eliminating redundant data transmission.

3. Server-Side Sharding: The Game Changer

Kubernetes v1.36 introduces a new shardSelector field in ListOptions. Clients specify a hash range using the shardRange() function, which computes a deterministic 64-bit FNV-1a hash of a chosen field (e.g., object.metadata.uid). The API server then returns only objects whose hash falls within the provided range [start, end). This applies to both initial list responses and subsequent watch event streams. The hash function is consistent across all API server instances, ensuring correctness in multi-replica setups. Currently supported fields are object.metadata.uid and object.metadata.namespace. This approach significantly reduces network traffic and deserialization overhead.

4. Implementing Sharded Watches in Controllers

To leverage this feature, controllers modify their informer setup. Use WithTweakListOptions to inject a shardSelector. For example, in a two-replica deployment, replica 0 handles the lower half of the hash space with shardRange(object.metadata.uid, '0x0000000000000000', '0x8000000000000000'). The code snippet below illustrates the integration:

import (
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/client-go/informers"
)

shardSelector := "shardRange(object.metadata.uid, '0x0000000000000000', '0x8000000000000000')"
factory := informers.NewSharedInformerFactoryWithOptions(client, resyncPeriod,
    informers.WithTweakListOptions(func(opts *metav1.ListOptions) {
        opts.ShardSelector = shardSelector
    }),
)

This simple change ensures each replica receives only the events it owns, drastically cutting unnecessary processing.

5. What This Means for Cluster Operators

Server-side sharding promises major benefits: reduced network bandwidth, lower CPU usage on controllers, and better scaling characteristics. For operators managing clusters with thousands of nodes, this alpha feature could unlock new levels of efficiency. However, as an alpha feature, it requires enabling a feature gate and may have limitations. Future enhancements may support additional hash fields and dynamic shard rebalancing. By adopting this early, operators can optimize resource usage and prepare for upcoming Kubernetes improvements. Start experimenting with ShardSelector in test environments to evaluate the impact on your workloads.

Server-side sharded list and watch represents a paradigm shift in how Kubernetes controllers consume events. By filtering at the API server level, it eliminates redundant data flow and enables true horizontal scaling without multiplicative resource waste. As the feature matures, it will become an essential tool for large-scale deployments. Start understanding the scaling challenge, and consider how this innovation can benefit your cluster operations.