Posts Tagged ‘CNCF’
[VoxxedDaysTicino2026] May the Control Plane Be with You: Kamaji and the Rise of Kubernetes at Scale
Lecturer
Dario Tranchitella serves as the Chief Technology Officer at Clastix, a startup he co-founded in 2020 during the global pandemic. With a background as a site reliability engineer and software developer, Dario specializes in Kubernetes engineering and multi-tenancy solutions. He has extensive experience managing large-scale Kubernetes fleets and contributes to open-source projects, drawing from his prior roles in the tech industry. Relevant links include his LinkedIn profile (https://it.linkedin.com/in/dariotranchitella) and Clastix’s website (https://clastix.io/).
Abstract
This article explores Dario Tranchitella’s insights into scaling Kubernetes through Kamaji, an open-source initiative transforming Kubernetes into a control-plane-as-a-service platform. Originating from real operational challenges, the discussion dissects Kubernetes architecture, the hosted control plane model, community-driven evolution, and adoption by major entities. It analyzes methodologies for multi-tenancy, resource optimization, and resilience, while considering implications for large-scale deployments in cloud-native environments.
Origins and Challenges in Kubernetes Management
Dario’s journey with Kamaji began amid personal and professional turmoil, exemplified by an outage during his father’s wedding that required restoring a Kubernetes cluster. This incident underscored the operational and financial hurdles of scaling Kubernetes beyond a few clusters. As a former site reliability engineer managing a fleet for a U.S. company, Dario encountered the complexities of multi-tenancy, where infrastructure or applications are shared among tenants—be they customers or internal teams—while ensuring fair resource allocation and preventing privilege escalation.
Kubernetes, donated to the Cloud Native Computing Foundation (CNCF), orchestrates containers in a distributed system comprising a control plane and worker nodes. The control plane acts as the “brain,” maintaining application states, while worker nodes provide computational power. Dario likens this to a reconciliation loop: users specify desired states, and Kubernetes aligns current states accordingly, handling tasks like load balancing without manual intervention. It runs ubiquitously—on laptops, clouds, bare metal, or edge devices—abstracting deployment details.
However, scaling introduces bottlenecks. The control plane includes the API server for information handling, the controller manager for reconciliation loops, the scheduler for pod placement to avoid single points of failure, and etcd for state storage using the Raft consensus algorithm. Etcd requires at least three instances for fault tolerance (n/2 + 1), making it resource-intensive and a primary challenge in multi-tenant setups.
In multi-tenancy, Dario emphasizes dividing resources imperatively, akin to apartments in a building: tenants occupy their spaces without infringing on others. Kubernetes excels here, but traditional setups demand separate clusters per tenant to isolate workloads, leading to overhead. Dario’s prior experience revealed inefficiencies, prompting Kamaji’s creation to address these pain points.
The Kamaji Architecture and Hosted Control Plane Model
Kamaji redefines Kubernetes by running control planes as regular pods within a management cluster, adopting a hosted control plane architecture. This separates control planes from worker nodes, allowing a single management cluster to host multiple tenant control planes efficiently. Worker nodes join via the management cluster’s API endpoint, optimizing resources and reducing costs.
Dario contrasts this with traditional setups: instead of dedicating machines per control plane, Kamaji leverages Kubernetes’ scheduling for etcd and other components as pods. This “Kubernetes-in-Kubernetes” approach, inspired by Google’s 2017 Kubernetes Engine, avoids vendor lock-in by supporting tools like kubeadm for certificate management and cluster bootstrapping.
Key innovations include multi-tenant datastores: Kamaji supports etcd, PostgreSQL, or MySQL, allowing collision of databases into single instances for optimization, though Dario advises multiple clusters to minimize blast radius. Scalability tests show a single management cluster handling up to a thousand control planes, but he recommends diversification for resilience.
Methodologically, Kamaji integrates with community projects like Cluster API for node provisioning across providers (Azure, AWS, Google). It avoids reinventing orchestration, focusing solely on control planes while enabling seamless worker node integration. Code samples illustrate simplicity:
apiVersion: kamaji.clastix.io/v1alpha1
kind: TenantControlPlane
metadata:
name: example
spec:
kubernetes:
version: v1.25.0
dataStore:
name: default
This YAML defines a tenant control plane, specifying Kubernetes version and datastore, demonstrating declarative management.
Implications include cost savings—reducing dedicated machines—and operational ease, as upgrades affect only the management cluster without tenant disruption.
Community Collaboration and Evolution of Kamaji
Kamaji’s growth stems from open-source collaboration since its 2022 launch at KubeCon Valencia. Dario highlights cross-pollination with organizations like NVIDIA, Rackspace, OVH, Ionos, and the CNCF community. Early adopters provided feedback, debunking scalability myths and proving PostgreSQL viability as an etcd alternative.
Dario’s philosophy: “Do what you love,” drove pursuits like running Kubernetes on PostgreSQL, challenging skeptics. Community tools like Kine (etcd shim) enabled alternative datastores, enhancing flexibility.
Evangelism involved panels at conferences, demystifying hosted control planes alongside Red Hat’s Hypershift and Mirantis’ K0s. Despite similarities, Kamaji’s vanilla Kubernetes focus and multi-datastore support differentiate it.
Code integration with kubeadm ensures portability:
kamaji create --kubeadm-config /path/to/config.yaml
This command bootstraps clusters, allowing imports from existing setups without lock-in.
Consequences: Kamaji fosters a collaborative ecosystem, reducing proprietary dependencies and promoting standards. Adoption by giants validates its scalability, though Dario cautions against over-reliance on single clusters.
Implications for Cloud-Native Scalability and Future Directions
Kamaji addresses Kubernetes’ scaling pains by commoditizing control planes, lowering barriers for multi-tenant platforms. It optimizes resources, crucial in cloud environments where costs accumulate. By hosting control planes as pods, it leverages Kubernetes’ strengths for self-management, a meta-approach enhancing resilience.
Broader implications include democratizing large-scale deployments: smaller teams manage vast fleets without proportional infrastructure. However, Dario stresses evaluating trade-offs—colliding datastores risks contention, necessitating careful architecture.
Future directions involve deeper community integration, potentially expanding to more datastores or advanced scheduling. Kamaji’s open-source ethos ensures evolution through contributions, avoiding silos.
In conclusion, Dario’s work with Kamaji exemplifies pragmatic innovation in cloud-native computing, balancing efficiency, resilience, and community-driven progress.
Links:
[KCDUK2024] Building the Future, Together | KCDUK2024
From Neopets to Google: A Tech Journey
Cheryl Hung, a Senior Director at Arm, delivered an inspiring keynote at KCDUK2024 titled “Building the Future, Together.” Reflecting on her journey as a cloud native pioneer, Cheryl shared personal anecdotes and community insights, weaving a narrative of resilience and collaboration. Born to a non-technical family of Hong Kong immigrants, Cheryl’s entry into tech was sparked by fascination with Google’s innovative culture and early experiments with Neopets’ HTML coding.
At Cambridge, Cheryl pursued a computer science degree, driven by her ambition to join Google. Despite a failed interview at 19, she persevered, securing a role as a software engineer working on Google Maps’ C++ backend. However, burnout and a lack of fulfillment led her to quit after five years, questioning her place in tech. Cheryl’s rediscovery of passion through community meetups, particularly around Docker and containers, marked a turning point, aligning with her prior exposure to Google’s Borg infrastructure.
Embracing Developer Advocacy
Defying warnings about career risks, Cheryl embraced developer advocacy, joining a startup as employee number 12. There, she honed her public speaking skills and founded the Cloud Native London Meetup, building strong community ties. Her role as a CNCF ambassador, secured through a simple photo submission, amplified her influence, leading to her appointment as Director of Ecosystem at CNCF. Cheryl’s tenure involved extensive travel, delivering keynotes and engaging with cloud native “celebrities” like Kelsey Hightower and Liz Rice, though the 2020 pandemic disrupted this dynamic lifestyle.
Cheryl’s interactions revealed the human side of tech luminaries, from Dan Kohn’s supportive mentorship to Kelsey’s off-stage humor. These experiences underscored the importance of authentic connections in driving innovation. Her narrative highlighted the CNCF’s growth from a nascent organization to a powerhouse managing 190 projects, reflecting the community’s role in shaping cloud native technologies.
Navigating Career Transitions
Pregnancy prompted Cheryl to reassess her role at CNCF, leading to a move to Apple, followed by a swift transition to Arm as Senior Director of Ecosystem. Despite personal challenges, including back-to-back maternity leaves, Cheryl’s commitment to community remained steadfast. Her narrative emphasized the power of relationships in career navigation, as opportunities arose through trusted connections rather than formal applications.
Cheryl’s candid reflection on overcoming burnout and imposter syndrome resonated deeply, illustrating the resilience required in tech. Her move to Arm, driven by an irresistible role, underscored her belief in seizing opportunities that align with personal and professional growth. This journey of adaptation and reinvention highlighted the importance of flexibility in a rapidly evolving industry.
Building a Collaborative Future
Cheryl concluded with a call to action, urging attendees to foster community through connection. Acknowledging her struggle with facial blindness, she emphasized the courage required to engage with strangers, advocating for inclusive spaces where diverse ideas thrive. Her vision for the future centers on collective effort, encouraging practitioners to build together, leveraging events like KCDUK2024 to forge lasting relationships.
Cheryl’s narrative serves as a beacon for aspiring technologists, demonstrating that the future of cloud native lies in community-driven innovation. By sharing her vulnerabilities and triumphs, she inspired attendees to embrace collaboration, ensuring the ecosystem continues to evolve through shared knowledge and diverse perspectives.
Links:
[KCDUK2024] Sustainability Chronicles: Innovate Through Green Technology With Kepler and KEDA | KCDUK2024
The Imperative of Cloud Sustainability
At KCDUK2024, Katie Gamanji, a Senior Field Engineer at Apple and a prominent figure in the Cloud Native Computing Foundation (CNCF), delivered a compelling session titled “Sustainability Chronicles: Innovate Through Green Technology With Kepler and KEDA.” Katie’s presentation addressed the critical need for environmental consciousness in the tech sector, particularly within the cloud native ecosystem. With a decade of Kubernetes driving industry transformation, Katie emphasized the urgency of integrating sustainability into technological decision-making, given the tech sector’s 1.4% contribution to global greenhouse gas emissions.
Katie outlined the global context, referencing the Paris Agreement (COP21) and the United Nations’ Sustainable Development Goals (SDG13), which call for proactive climate action. She highlighted that adopting renewable energy could reduce tech emissions by 80%, a potential that major cloud providers like GCP, AWS, and Azure are pursuing through net-zero targets. Katie’s narrative framed sustainability as an integral part of operational efficiency, introducing the concept of GreenOps, which aligns resource optimization with reduced environmental impact.
Measuring Emissions with Kepler
Katie introduced Kepler, a CNCF sandbox project developed by Red Hat and IBM, as a pivotal tool for measuring application emissions in Kubernetes. Kepler leverages eBPF to collect energy consumption data, converting it into CO2 emissions using hardcoded emission factors for coal, petroleum, and natural gas. Deployed on a local Kind cluster, Kepler’s Grafana dashboards provide granular insights into emissions per container, day, and namespace. This capability enables organizations to identify high-emission workloads and implement targeted optimizations.
Katie encouraged community engagement with Kepler, noting its sandbox status and potential for growth. By providing feedback or contributing features, practitioners can enhance Kepler’s utility, fostering a collaborative approach to sustainability. Her demonstration underscored the importance of observability in addressing emissions, setting the stage for proactive workload management.
Scaling with KEDA and Carbon-Aware Operator
To move beyond measurement, Katie introduced KEDA (Kubernetes Event-Driven Autoscaler) and its Carbon-Aware Operator, which optimize application scaling based on carbon intensity. KEDA, a graduated CNCF project, scales workloads in response to external events, such as carbon intensity data from grid providers like WattTime or Electricity Map. The Carbon-Aware Operator adjusts replica counts inversely to carbon intensity, maximizing replicas when renewable energy is abundant and minimizing them during high-emission periods.
Katie’s local deployment showcased this dynamic scaling, visualized through Grafana, where low carbon intensity correlated with higher replica counts. This approach exemplifies proactive sustainability, aligning computational demands with environmental impact. Katie’s advocacy for KEDA highlighted its evolution from a niche solution to an industry standard, encouraging practitioners to explore its potential for green innovation.
Community-Driven Sustainability
Beyond technology, Katie emphasized the role of community in driving sustainability. The Kubernetes community’s inclusive ethos fosters innovation through open governance and diverse contributions. She highlighted the CNCF’s Technical Advisory Group (TAG) for Environmental Sustainability, which promotes green practices across projects. Initiatives like the Green Reviews Working Group and sustainability white papers offer avenues for engagement, encouraging practitioners to contribute beyond code.
Katie’s call to action urged attendees to diversify technical boards and create welcoming spaces for new ideas. Her narrative wove together technological innovation and community collaboration, positioning sustainability as a shared responsibility. By integrating tools like Kepler and KEDA with community efforts, the cloud native ecosystem can lead in green technology, shaping a sustainable future.
Links:
[DevoxxGR2024] Devoxx Greece 2024 Sustainability Chronicles: Innovate Through Green Technology With Kepler and KEDA
At Devoxx Greece 2024, Katie Gamanji, a senior field engineer at Apple and a technical oversight committee member for the Cloud Native Computing Foundation (CNCF), delivered a compelling presentation on advancing environmental sustainability within the cloud-native ecosystem. With Kubernetes celebrating its tenth anniversary, Katie emphasized the urgent need for technologists to integrate green practices into their infrastructure strategies. Her talk explored how tools like Kepler and KEDA’s carbon-aware operator enable practitioners to measure and mitigate carbon emissions, while fostering a vibrant, inclusive community to drive these efforts forward. Drawing from her extensive experience and leadership in the CNCF, Katie provided a roadmap for aligning technological innovation with climate responsibility.
The Imperative of Cloud Sustainability
Katie began by underscoring the critical role of sustainability in the tech sector, particularly given the industry’s contribution to global greenhouse gas emissions. She highlighted that the tech sector accounts for 1.4% of global emissions, a figure that could soar to 10% within a decade without intervention. However, by leveraging renewable energy, emissions could be reduced by up to 80%. International agreements like COP21 and the United Nations’ Sustainable Development Goals (SDGs) have spurred national regulations, compelling organizations to assess and report their carbon footprints. Major cloud providers, such as Google Cloud Platform (GCP), have set ambitious net-zero targets, with GCP already operating on renewable energy since 2022. Yet, Katie stressed that sustainability cannot be outsourced solely to cloud providers; organizations must embed these principles internally.
The emergence of “GreenOps,” inspired by FinOps, encapsulates the processes, tools, and cultural shifts needed to achieve digital sustainability. By optimizing infrastructure—through strategies like using spot instances or serverless architectures—organizations can reduce both costs and emissions. Katie introduced a four-phase strategy proposed by the FinOps Foundation’s Environmental Sustainability Working Group: awareness, discovery, roadmap, and execution. This framework encourages organizations to educate stakeholders, benchmark emissions, implement automated tools, and iteratively pursue ambitious sustainability goals.
Measuring Emissions with Kepler
To address emissions within Kubernetes clusters, Katie introduced Kepler, a CNCF sandbox project developed by Red Hat and IBM. Kepler, a Kubernetes Efficient Power Level Exporter, utilizes eBPF to probe system statistics and export power consumption metrics to Prometheus for visualization in tools like Grafana. Deployed as a daemon set, Kepler collects node- and container-level metrics, focusing on power usage and resource utilization. By tracing CPU performance counters and Linux kernel trace points, it calculates energy consumption in joules, converting this to kilowatt-hours and multiplying by region-specific emission factors for gases like coal, petroleum, and natural gas.
Katie demonstrated Kepler’s practical application using a Grafana dashboard, which displayed emissions per gas and allowed granular analysis by container, day, or namespace. This visibility enables organizations to identify high-emission components, such as during traffic spikes, and optimize accordingly. As a sandbox project, Kepler is gaining momentum, and Katie encouraged attendees to explore it, provide feedback, or contribute to its development, reinforcing its potential to establish a baseline for carbon accounting in cloud-native environments.
Scaling Sustainably with KEDA’s Carbon-Aware Operator
Complementing Kepler’s observational capabilities, Katie introduced KEDA (Kubernetes Event-Driven Autoscaler), a graduated CNCF project, and its carbon-aware operator. KEDA, created by Microsoft and Red Hat, scales applications based on external events, offering a rich catalog of triggers. The carbon-aware operator optimizes emissions by scaling applications according to carbon intensity—grams of CO2 equivalent emitted per kilowatt-hour consumed. In scenarios where infrastructure is powered by renewable sources like solar or wind, carbon intensity approaches zero, allowing for maximum application replicas. Conversely, high carbon intensity, such as from coal-based energy, prompts scaling down to minimize emissions.
Katie illustrated this with a custom resource definition (CRD) that configures scaling behavior based on carbon intensity forecasts from providers like WattTime or Electricity Maps. In her demo, a Grafana dashboard showed an application scaling from 15 replicas at a carbon intensity of 530 to a single replica at 580, dynamically responding to grid data. This proactive approach ensures sustainability is embedded in scheduling decisions, aligning resource usage with environmental impact.
Nurturing a Sustainable Community
Beyond technology, Katie emphasized the pivotal role of the Kubernetes community in driving sustainability. Operating on principles of inclusivity, open governance, and transparency, the community fosters innovation through technical advisory groups (TAGs) focused on domains like observability, security, and environmental sustainability. The TAG Environmental Sustainability, established just over a year ago, aims to benchmark emissions across graduated CNCF projects, raising awareness and encouraging greener practices.
To sustain this momentum, Katie highlighted the need for education and upskilling. Resources like the Kubernetes and Cloud Native Associate (KCNA) certification and her own Cloud Native Fundamentals course on Udacity lower entry barriers for newcomers. By diversifying technical and governing boards, the community can continue to evolve, ensuring it scales alongside technological advancements. Katie’s vision is a cloud-native ecosystem where innovation and sustainability coexist, supported by a nurturing, inclusive community.
Conclusion
Katie Gamanji’s presentation at Devoxx Greece 2024 was a clarion call for technologists to prioritize environmental sustainability. By leveraging tools like Kepler and KEDA’s carbon-aware operator, practitioners can measure and mitigate emissions within Kubernetes clusters, aligning infrastructure with climate goals. Equally important is the community’s role in fostering education, inclusivity, and collaboration to sustain these efforts. Katie’s insights, grounded in her leadership at Apple and the CNCF, offer a blueprint for innovating through green technology while building a resilient, forward-thinking ecosystem.