Posts Tagged ‘CNCF’
[VoxxedDaysTicino2026] May the Control Plane Be with You: Kamaji and the Rise of Kubernetes at Scale
Lecturer
Dario Tranchitella serves as the Chief Technology Officer at Clastix, a startup he co-founded in 2020 during the global pandemic. With a background as a site reliability engineer and software developer, Dario specializes in Kubernetes engineering and multi-tenancy solutions. He has extensive experience managing large-scale Kubernetes fleets and contributes to open-source projects, drawing from his prior roles in the tech industry. Relevant links include his LinkedIn profile (https://it.linkedin.com/in/dariotranchitella) and Clastix’s website (https://clastix.io/).
Abstract
This article explores Dario Tranchitella’s insights into scaling Kubernetes through Kamaji, an open-source initiative transforming Kubernetes into a control-plane-as-a-service platform. Originating from real operational challenges, the discussion dissects Kubernetes architecture, the hosted control plane model, community-driven evolution, and adoption by major entities. It analyzes methodologies for multi-tenancy, resource optimization, and resilience, while considering implications for large-scale deployments in cloud-native environments.
Origins and Challenges in Kubernetes Management
Dario’s journey with Kamaji began amid personal and professional turmoil, exemplified by an outage during his father’s wedding that required restoring a Kubernetes cluster. This incident underscored the operational and financial hurdles of scaling Kubernetes beyond a few clusters. As a former site reliability engineer managing a fleet for a U.S. company, Dario encountered the complexities of multi-tenancy, where infrastructure or applications are shared among tenants—be they customers or internal teams—while ensuring fair resource allocation and preventing privilege escalation.
Kubernetes, donated to the Cloud Native Computing Foundation (CNCF), orchestrates containers in a distributed system comprising a control plane and worker nodes. The control plane acts as the “brain,” maintaining application states, while worker nodes provide computational power. Dario likens this to a reconciliation loop: users specify desired states, and Kubernetes aligns current states accordingly, handling tasks like load balancing without manual intervention. It runs ubiquitously—on laptops, clouds, bare metal, or edge devices—abstracting deployment details.
However, scaling introduces bottlenecks. The control plane includes the API server for information handling, the controller manager for reconciliation loops, the scheduler for pod placement to avoid single points of failure, and etcd for state storage using the Raft consensus algorithm. Etcd requires at least three instances for fault tolerance (n/2 + 1), making it resource-intensive and a primary challenge in multi-tenant setups.
In multi-tenancy, Dario emphasizes dividing resources imperatively, akin to apartments in a building: tenants occupy their spaces without infringing on others. Kubernetes excels here, but traditional setups demand separate clusters per tenant to isolate workloads, leading to overhead. Dario’s prior experience revealed inefficiencies, prompting Kamaji’s creation to address these pain points.
The Kamaji Architecture and Hosted Control Plane Model
Kamaji redefines Kubernetes by running control planes as regular pods within a management cluster, adopting a hosted control plane architecture. This separates control planes from worker nodes, allowing a single management cluster to host multiple tenant control planes efficiently. Worker nodes join via the management cluster’s API endpoint, optimizing resources and reducing costs.
Dario contrasts this with traditional setups: instead of dedicating machines per control plane, Kamaji leverages Kubernetes’ scheduling for etcd and other components as pods. This “Kubernetes-in-Kubernetes” approach, inspired by Google’s 2017 Kubernetes Engine, avoids vendor lock-in by supporting tools like kubeadm for certificate management and cluster bootstrapping.
Key innovations include multi-tenant datastores: Kamaji supports etcd, PostgreSQL, or MySQL, allowing collision of databases into single instances for optimization, though Dario advises multiple clusters to minimize blast radius. Scalability tests show a single management cluster handling up to a thousand control planes, but he recommends diversification for resilience.
Methodologically, Kamaji integrates with community projects like Cluster API for node provisioning across providers (Azure, AWS, Google). It avoids reinventing orchestration, focusing solely on control planes while enabling seamless worker node integration. Code samples illustrate simplicity:
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: example
spec:
kubernetes:
version: v1.25.0
dataStore:
name: default
This YAML defines a tenant control plane, specifying Kubernetes version and datastore, demonstrating declarative management.
Implications include cost savings—reducing dedicated machines—and operational ease, as upgrades affect only the management cluster without tenant disruption.
Community Collaboration and Evolution of Kamaji
Kamaji’s growth stems from open-source collaboration since its 2022 launch at KubeCon Valencia. Dario highlights cross-pollination with organizations like NVIDIA, Rackspace, OVH, Ionos, and the CNCF community. Early adopters provided feedback, debunking scalability myths and proving PostgreSQL viability as an etcd alternative.
Dario’s philosophy: “Do what you love,” drove pursuits like running Kubernetes on PostgreSQL, challenging skeptics. Community tools like Kine (etcd shim) enabled alternative datastores, enhancing flexibility.
Evangelism involved panels at conferences, demystifying hosted control planes alongside Red Hat’s Hypershift and Mirantis’ K0s. Despite similarities, Kamaji’s vanilla Kubernetes focus and multi-datastore support differentiate it.
Code integration with kubeadm ensures portability:
kamaji create --kubeadm-config /path/to/config.yaml
This command bootstraps clusters, allowing imports from existing setups without lock-in.
Consequences: Kamaji fosters a collaborative ecosystem, reducing proprietary dependencies and promoting standards. Adoption by giants validates its scalability, though Dario cautions against over-reliance on single clusters.
Implications for Cloud-Native Scalability and Future Directions
Kamaji addresses Kubernetes’ scaling pains by commoditizing control planes, lowering barriers for multi-tenant platforms. It optimizes resources, crucial in cloud environments where costs accumulate. By hosting control planes as pods, it leverages Kubernetes’ strengths for self-management, a meta-approach enhancing resilience.
Broader implications include democratizing large-scale deployments: smaller teams manage vast fleets without proportional infrastructure. However, Dario stresses evaluating trade-offs—colliding datastores risks contention, necessitating careful architecture.
Future directions involve deeper community integration, potentially expanding to more datastores or advanced scheduling. Kamaji’s open-source ethos ensures evolution through contributions, avoiding silos.
In conclusion, Dario’s work with Kamaji exemplifies pragmatic innovation in cloud-native computing, balancing efficiency, resilience, and community-driven progress.
Links:
[DevoxxGR2024] Devoxx Greece 2024 Sustainability Chronicles: Innovate Through Green Technology With Kepler and KEDA
At Devoxx Greece 2024, Katie Gamanji, a senior field engineer at Apple and a technical oversight committee member for the Cloud Native Computing Foundation (CNCF), delivered a compelling presentation on advancing environmental sustainability within the cloud-native ecosystem. With Kubernetes celebrating its tenth anniversary, Katie emphasized the urgent need for technologists to integrate green practices into their infrastructure strategies. Her talk explored how tools like Kepler and KEDA’s carbon-aware operator enable practitioners to measure and mitigate carbon emissions, while fostering a vibrant, inclusive community to drive these efforts forward. Drawing from her extensive experience and leadership in the CNCF, Katie provided a roadmap for aligning technological innovation with climate responsibility.
The Imperative of Cloud Sustainability
Katie began by underscoring the critical role of sustainability in the tech sector, particularly given the industry’s contribution to global greenhouse gas emissions. She highlighted that the tech sector accounts for 1.4% of global emissions, a figure that could soar to 10% within a decade without intervention. However, by leveraging renewable energy, emissions could be reduced by up to 80%. International agreements like COP21 and the United Nations’ Sustainable Development Goals (SDGs) have spurred national regulations, compelling organizations to assess and report their carbon footprints. Major cloud providers, such as Google Cloud Platform (GCP), have set ambitious net-zero targets, with GCP already operating on renewable energy since 2022. Yet, Katie stressed that sustainability cannot be outsourced solely to cloud providers; organizations must embed these principles internally.
The emergence of “GreenOps,” inspired by FinOps, encapsulates the processes, tools, and cultural shifts needed to achieve digital sustainability. By optimizing infrastructure—through strategies like using spot instances or serverless architectures—organizations can reduce both costs and emissions. Katie introduced a four-phase strategy proposed by the FinOps Foundation’s Environmental Sustainability Working Group: awareness, discovery, roadmap, and execution. This framework encourages organizations to educate stakeholders, benchmark emissions, implement automated tools, and iteratively pursue ambitious sustainability goals.
Measuring Emissions with Kepler
To address emissions within Kubernetes clusters, Katie introduced Kepler, a CNCF sandbox project developed by Red Hat and IBM. Kepler, a Kubernetes Efficient Power Level Exporter, utilizes eBPF to probe system statistics and export power consumption metrics to Prometheus for visualization in tools like Grafana. Deployed as a daemon set, Kepler collects node- and container-level metrics, focusing on power usage and resource utilization. By tracing CPU performance counters and Linux kernel trace points, it calculates energy consumption in joules, converting this to kilowatt-hours and multiplying by region-specific emission factors for gases like coal, petroleum, and natural gas.
Katie demonstrated Kepler’s practical application using a Grafana dashboard, which displayed emissions per gas and allowed granular analysis by container, day, or namespace. This visibility enables organizations to identify high-emission components, such as during traffic spikes, and optimize accordingly. As a sandbox project, Kepler is gaining momentum, and Katie encouraged attendees to explore it, provide feedback, or contribute to its development, reinforcing its potential to establish a baseline for carbon accounting in cloud-native environments.
Scaling Sustainably with KEDA’s Carbon-Aware Operator
Complementing Kepler’s observational capabilities, Katie introduced KEDA (Kubernetes Event-Driven Autoscaler), a graduated CNCF project, and its carbon-aware operator. KEDA, created by Microsoft and Red Hat, scales applications based on external events, offering a rich catalog of triggers. The carbon-aware operator optimizes emissions by scaling applications according to carbon intensity—grams of CO2 equivalent emitted per kilowatt-hour consumed. In scenarios where infrastructure is powered by renewable sources like solar or wind, carbon intensity approaches zero, allowing for maximum application replicas. Conversely, high carbon intensity, such as from coal-based energy, prompts scaling down to minimize emissions.
Katie illustrated this with a custom resource definition (CRD) that configures scaling behavior based on carbon intensity forecasts from providers like WattTime or Electricity Maps. In her demo, a Grafana dashboard showed an application scaling from 15 replicas at a carbon intensity of 530 to a single replica at 580, dynamically responding to grid data. This proactive approach ensures sustainability is embedded in scheduling decisions, aligning resource usage with environmental impact.
Nurturing a Sustainable Community
Beyond technology, Katie emphasized the pivotal role of the Kubernetes community in driving sustainability. Operating on principles of inclusivity, open governance, and transparency, the community fosters innovation through technical advisory groups (TAGs) focused on domains like observability, security, and environmental sustainability. The TAG Environmental Sustainability, established just over a year ago, aims to benchmark emissions across graduated CNCF projects, raising awareness and encouraging greener practices.
To sustain this momentum, Katie highlighted the need for education and upskilling. Resources like the Kubernetes and Cloud Native Associate (KCNA) certification and her own Cloud Native Fundamentals course on Udacity lower entry barriers for newcomers. By diversifying technical and governing boards, the community can continue to evolve, ensuring it scales alongside technological advancements. Katie’s vision is a cloud-native ecosystem where innovation and sustainability coexist, supported by a nurturing, inclusive community.
Conclusion
Katie Gamanji’s presentation at Devoxx Greece 2024 was a clarion call for technologists to prioritize environmental sustainability. By leveraging tools like Kepler and KEDA’s carbon-aware operator, practitioners can measure and mitigate emissions within Kubernetes clusters, aligning infrastructure with climate goals. Equally important is the community’s role in fostering education, inclusivity, and collaboration to sustain these efforts. Katie’s insights, grounded in her leadership at Apple and the CNCF, offer a blueprint for innovating through green technology while building a resilient, forward-thinking ecosystem.