Posts Tagged ‘KCDUK2024’
[KCDUK2024] Sustainability Chronicles: Innovate Through Green Technology With Kepler and KEDA | KCDUK2024
The Imperative of Cloud Sustainability
At KCDUK2024, Katie Gamanji, a Senior Field Engineer at Apple and a prominent figure in the Cloud Native Computing Foundation (CNCF), delivered a compelling session titled “Sustainability Chronicles: Innovate Through Green Technology With Kepler and KEDA.” Katie’s presentation addressed the critical need for environmental consciousness in the tech sector, particularly within the cloud native ecosystem. With a decade of Kubernetes driving industry transformation, Katie emphasized the urgency of integrating sustainability into technological decision-making, given the tech sector’s 1.4% contribution to global greenhouse gas emissions.
Katie outlined the global context, referencing the Paris Agreement (COP21) and the United Nations’ Sustainable Development Goals (SDG13), which call for proactive climate action. She highlighted that adopting renewable energy could reduce tech emissions by 80%, a potential that major cloud providers like GCP, AWS, and Azure are pursuing through net-zero targets. Katie’s narrative framed sustainability as an integral part of operational efficiency, introducing the concept of GreenOps, which aligns resource optimization with reduced environmental impact.
Measuring Emissions with Kepler
Katie introduced Kepler, a CNCF sandbox project developed by Red Hat and IBM, as a pivotal tool for measuring application emissions in Kubernetes. Kepler leverages eBPF to collect energy consumption data, converting it into CO2 emissions using hardcoded emission factors for coal, petroleum, and natural gas. Deployed on a local Kind cluster, Kepler’s Grafana dashboards provide granular insights into emissions per container, day, and namespace. This capability enables organizations to identify high-emission workloads and implement targeted optimizations.
Katie encouraged community engagement with Kepler, noting its sandbox status and potential for growth. By providing feedback or contributing features, practitioners can enhance Kepler’s utility, fostering a collaborative approach to sustainability. Her demonstration underscored the importance of observability in addressing emissions, setting the stage for proactive workload management.
Scaling with KEDA and Carbon-Aware Operator
To move beyond measurement, Katie introduced KEDA (Kubernetes Event-Driven Autoscaler) and its Carbon-Aware Operator, which optimize application scaling based on carbon intensity. KEDA, a graduated CNCF project, scales workloads in response to external events, such as carbon intensity data from grid providers like WattTime or Electricity Map. The Carbon-Aware Operator adjusts replica counts inversely to carbon intensity, maximizing replicas when renewable energy is abundant and minimizing them during high-emission periods.
Katie’s local deployment showcased this dynamic scaling, visualized through Grafana, where low carbon intensity correlated with higher replica counts. This approach exemplifies proactive sustainability, aligning computational demands with environmental impact. Katie’s advocacy for KEDA highlighted its evolution from a niche solution to an industry standard, encouraging practitioners to explore its potential for green innovation.
Community-Driven Sustainability
Beyond technology, Katie emphasized the role of community in driving sustainability. The Kubernetes community’s inclusive ethos fosters innovation through open governance and diverse contributions. She highlighted the CNCF’s Technical Advisory Group (TAG) for Environmental Sustainability, which promotes green practices across projects. Initiatives like the Green Reviews Working Group and sustainability white papers offer avenues for engagement, encouraging practitioners to contribute beyond code.
Katie’s call to action urged attendees to diversify technical boards and create welcoming spaces for new ideas. Her narrative wove together technological innovation and community collaboration, positioning sustainability as a shared responsibility. By integrating tools like Kepler and KEDA with community efforts, the cloud native ecosystem can lead in green technology, shaping a sustainable future.
Links:
[KCDUK2024] An Odyssey with ArgoCD: From Git to Helm | KCDUK2024
Introduction to GitOps and ArgoCD
In the ever-evolving landscape of Kubernetes management, GitOps has emerged as a cornerstone for streamlined application deployment. At KCDUK2024, Farah Adbib and Antonio Alferez, both esteemed professionals from The Workshop, delivered an insightful session titled “An Odyssey with ArgoCD: From Git to Helm.” Their talk elucidated the journey of implementing GitOps using ArgoCD within their organization, navigating through initial challenges and innovative solutions. Farah, a DevOps Solution Architect, and Antonio, a Platform Engineer, shared their expertise on leveraging Git repositories and Helm charts to manage Kubernetes clusters efficiently, offering a narrative rich with practical insights.
The session began with an overview of their ecosystem, managing 55 Kubernetes clusters across public cloud and on-premises infrastructure, supporting 1,600 nodes. These clusters cater to 350 software engineers across various business units, each responsible for deploying their applications. The need for simplicity and security in deployment processes was paramount, given the isolation requirements for some clusters. Farah and Antonio’s narrative underscored the importance of aligning technological solutions with organizational needs, setting the stage for their exploration of ArgoCD’s capabilities.
Initial Approach: Git as the Source of Truth
Initially, Farah and Antonio adopted Git repositories as the primary source for ArgoCD, a logical choice given GitOps’ emphasis on declarative configuration. They opted for a decentralized approach, deploying an ArgoCD instance per cluster to meet stringent security and isolation requirements. Each application’s Helm chart was separated from the application code, stored in distinct Git repositories to avoid replication across data centers. This separation was driven by the differing lifecycles of application code and Helm charts, aiming to streamline management.
However, this approach revealed several pain points. The necessity for Git mirrors introduced additional infrastructure complexity, requiring maintenance and coordination between development and operations teams. The use of Git branches as target revisions led to confusion, as development and operational branches coexisted, complicating version control. Moreover, rendering Helm charts within ArgoCD itself delayed feedback loops, causing failures late in the pipeline. Farah and Antonio’s candid reflection on these challenges highlighted the need for a more robust solution, prompting a strategic pivot.
Evolving to Helm Rendered Manifests
Recognizing the limitations of their initial setup, Farah and Antonio transitioned to a Helm-rendered manifest pattern. This innovative approach involved pre-rendering Helm manifests in the CI/CD pipeline and storing them in an artifact registry, such as Nexus, rather than Git mirrors. By pointing ArgoCD to these pre-rendered manifests, they eliminated the need for ArgoCD to handle rendering, significantly reducing complexity and accelerating feedback loops. This shift also unified application code and Helm charts into a single repository, simplifying versioning and pipeline management.
A key enhancement was the introduction of an external Helm values repository per cluster, allowing operational changes like scaling without triggering a full pipeline rebuild. This decoupling of configuration from code enhanced flexibility and developer experience. Antonio emphasized the adoption of semantic versioning (SemVer) for both Docker images and Helm charts, incorporating commit hashes for traceability. This meticulous versioning strategy ensured clarity on deployed versions across 55 clusters, leveraging Git notes for additional auditing information.
Lessons Learned and Developer Experience
The transition to Helm-rendered manifests yielded significant improvements. By moving rendering logic to the CI/CD pipeline, Farah and Antonio achieved a “shift-left” approach, enabling earlier failure detection and faster iterations. The elimination of Git mirrors reduced infrastructure overhead, while unified repositories streamlined development workflows. The external Helm values repository facilitated rapid operational adjustments, enhancing agility.
Farah and Antonio underscored the importance of tailoring solutions to the organizational ecosystem. Their journey highlighted the pitfalls of adhering to industry defaults without considering specific requirements. They emphasized that developer experience is paramount, advocating for solutions that empower engineers while minimizing management overhead. Their narrative serves as a testament to the value of iterative improvement, encouraging practitioners to reassess and redesign when initial solutions fall short.