Posts Tagged ‘CICD’
[MunchenJUG] Advanced Automated Testing: Navigating the Integration Frontier (13/May/2024)
Lecturer
Daniel Istvan Buza is an accomplished Senior Software Engineer and Technical Lead with extensive experience in architecting Java-based ecosystems. His expertise spans a broad spectrum of technologies, including Spring, Angular, Kafka, MongoDB, and Microservices. As a leader of multiple development teams, Daniel focuses on enhancing code quality through initiatives like coding dojos and rigorous peer reviews. He is a dedicated mentor within the software community, constantly exploring innovative methodologies to bridge the gap between development and quality assurance.
Abstract
This article analyzes the transition from traditional unit testing to comprehensive acceptance and end-to-end (E2E) testing frameworks. It utilizes a real-world case study of a “silent” frontend-backend failure to illustrate why high test coverage often fails to detect integration defects. The discussion centers on the implementation of Playwright as a primary tool for stateful and stateless testing within a Java environment. By evaluating strategies for mocking external dependencies like Kafka and S3, and addressing common pitfalls such as internationalization and time zone sensitivities, this analysis provides a technical roadmap for building resilient CI/CD pipelines that go beyond the limitations of isolated component tests.
The Vulnerability of Isolated Testing
A critical challenge in modern web development is the “integration gap”—a scenario where backend and frontend components pass their respective unit tests but fail when operating in tandem. A common example involves dynamic attribute renaming: if a backend developer renames a field in a DTO (Data Transfer Object) and updates the corresponding tests, the backend remains “green.” However, if the frontend is not simultaneously updated to expect the new attribute name, the UI may fail to display data correctly, resulting in empty columns or broken features that are highly visible to users but invisible to isolated test suites.
This discrepancy highlights a fundamental truth: test coverage does not equate to test quality. Even 100% coverage cannot guarantee system correctness if the interactions between disparate services are not explicitly verified. To address this, teams must move toward “Acceptance Tests” (AC tests) that simulate actual user interactions across the entire stack.
Leveraging Playwright for Java-Centric Environments
For teams primarily composed of backend developers, selecting a testing tool that integrates seamlessly with the existing Java ecosystem is paramount. Playwright, a framework developed by Microsoft, has emerged as a robust solution due to its native Java support and its ability to automate browsers like Chrome, Firefox, and Safari.
Core Interactions in E2E Testing
The majority of web application functionality can be verified through four fundamental interaction types:
- Clicking: Interacting with buttons, links, and navigation elements.
- Input: Filling text and search fields.
- Assertion: Verifying visual properties, such as the presence, color, or size of elements.
- File Operations: Managing the upload and download of documents.
By focusing on these interactions, developers can create scripts that mirror user behavior, ensuring that the “happy path” of the application remains functional regardless of internal refactoring.
Architectural Strategies: Stateful vs. Stateless Testing
When implementing E2E tests, developers must choose between two primary architectural approaches: stateful and stateless testing.
Stateful (Environment-Targeted) Testing
In this model, tests are executed against a persistent environment with a shared database and live external APIs. This approach is highly realistic but introduces the risk of “pollution,” where data left by one test affects the outcome of subsequent tests. It requires rigorous cleanup procedures to maintain environment stability.
Stateless (Containerized) Testing
Stateless testing involves spinning up a fresh, full-featured frontend-backend pair within a CI/CD pipeline for every test run. This often utilizes embedded databases (e.g., MongoDB) and mocks for external dependencies like S3 buckets or Kafka topics. While more complex to set up, this method provides total isolation and reproducibility. However, it requires careful management of operating system dependencies within the test containers to ensure Playwright can execute the browsers correctly.
Technical Pitfalls and Best Practices
The transition to advanced automated testing reveals several subtle challenges that can undermine test reliability.
- Internationalization (i18n): Relying on UI text for selectors can lead to massive test failures when translation files are updated. Using unique element IDs is a safer alternative, though it may limit the ability to verify that the correct error messages are being displayed to the user.
- Time Zone Sensitivity: UI elements displaying timestamps will vary based on the local environment. Playwright allows developers to explicitly specify a locale to ensure consistent assertions across geographically distributed teams.
- Synchronicity: A major source of test flakiness is the misuse of
Thread.sleep(). Developers should instead utilize Playwright’s built-in “wait for condition” methods to handle asynchronous backend tasks, which are more resilient to varying network or processing speeds.
Conclusion
Modern software delivery requires a testing strategy that transcends the unit level. By integrating Playwright into the Java development lifecycle, teams can automate complex user journeys and bridge the gap between frontend and backend. While no single testing setup is superior, a combination of stateful environment checks and isolated CI/CD pipelines provides the most comprehensive defense against integration defects. Developers are encouraged to treat their test code with the same rigor as production code, striving for cleanliness and maintainability to ensure long-term system reliability.
Links:
[DevoxxFR2026] GitHub Actions as a Supply Chain Security Time Bomb: Real-World Attacks and Defensive Strategies
Lecturer
Thierry Abalea is the co-founder and CEO of Shipfox, a French AI Factory platform specializing in coding agent workflows. With a background in software development and security, he focuses on practical approaches to securing modern CI/CD pipelines in cloud-native environments.
Abstract
GitHub Actions has become a cornerstone of modern CI/CD practices, yet it remains insecure by default. Recent supply chain attacks such as those targeting tj-actions, s1ngularity, GhostAction, and Shai-Hulud have demonstrated how adversaries systematically exploit workflows to exfiltrate secrets and compromise downstream projects. This presentation dissects concrete attack vectors observed in 2025, explains why traditional mitigations fall short, and outlines actionable defenses including scoped secrets with approval workflows, minimal GITHUB_TOKEN permissions, egress controls, runner hardening, and runtime protection tools. Attendees gain a clear understanding of the current threat landscape and practical steps to secure their pipelines.
The Critical Role and Inherent Risks of CI/CD in Modern Development
Continuous integration and continuous deployment pipelines manage extraordinarily sensitive assets: source code manipulation, secret handling for production access, and package publishing. Any compromise here can lead to widespread downstream damage. Thierry Abalea emphasizes that while GitHub Actions provides powerful automation, its permissive default configuration creates a vast attack surface. Workflows often run with broad permissions, handle long-lived secrets, and interact with external networks without sufficient restrictions.
The 2025 attack wave—including Singularity targeting Nx builds, Shai-Hulud as the first npm-propagating worm, and multiple incidents against Trivy—highlighted how GitHub Actions serves as both an entry point for initial compromise and a vector for secret exfiltration and lateral movement. These incidents affected hundreds of organizations, underscoring that CI/CD security can no longer be treated as secondary to application security.
Dissecting Major Supply Chain Attacks via GitHub Actions
Several high-profile incidents illustrate common exploitation patterns. The Singularity attack combined batch injection vulnerabilities in pull request workflows with the pull_request_target trigger. This allowed attackers to exfiltrate secrets from forked repositories, including NPM tokens used to publish malicious packages. Downstream consumers of the compromised Nx tool were subsequently affected.
Shai-Hulud represented a novel worm-like propagation through npm. Attackers gained secrets via similar workflow vulnerabilities, published malicious packages, and leveraged maintainer permissions across multiple projects. This created cascading compromises as infected packages spread through dependency trees.
Trivy, a widely used open-source security scanner, suffered repeated attacks. Initial exploitation via pull_request_target and unsafe interpolation led to remote code execution, secret exfiltration (including high-privilege GITHUB_TOKENs), repository privatization, and deletion of releases. A follow-up attack succeeded due to incomplete secret rotation, enabling further malicious package publications.
These cases reveal recurring themes: overly permissive triggers, secret exposure in workflows, and insufficient isolation between CI environments and production assets.
Why Default GitHub Actions Security Falls Short
GitHub Actions operates with broad defaults that favor convenience over security. Workflows can trigger on untrusted events like pull_request_target, granting access to repository secrets. The GITHUB_TOKEN often possesses excessive permissions, especially in repositories created before 2023. Actions referenced by mutable tags (e.g., v3) can be hijacked by attackers controlling upstream repositories. Self-hosted runners, if not properly isolated, allow one compromised job to affect the entire machine.
Network egress remains largely unrestricted, enabling easy data exfiltration to attacker-controlled servers or even private repositories. Traditional advice—pinning actions and limiting secrets—proves insufficient against sophisticated chained exploits.
Practical Defenses: Hardening GitHub Actions Workflows
Effective protection requires a defense-in-depth approach. Begin by scoping secrets with approval rules, ensuring only necessary workflows can access them. Minimize GITHUB_TOKEN permissions on a per-workflow basis, adhering to the principle of least privilege. Implement egress controls to restrict outbound connections from runners.
For self-hosted runners, enforce ephemeral instances and strong isolation. Tools like Step Security provide runtime hardening and firewall-like controls around runners, while GitHub’s upcoming Level 7 runner protections promise kernel-level isolation outside the runner’s reach.
Static analysis tools such as CodeQL (free for open source) and Zizmor detect workflow vulnerabilities. Dependency review bots help manage pinned versions. Organizations should treat CI/CD as production-equivalent, applying the same scrutiny to workflows as to application code.
Runtime protections and regular secret rotation further reduce the blast radius of potential breaches. Automation of security scanning within pull requests ensures issues are caught early.
Conclusion: Treating CI/CD as Production Infrastructure
GitHub Actions represents both a productivity powerhouse and a significant supply chain risk. By understanding real attack patterns and implementing layered defenses—from minimal permissions and scoped secrets to runtime controls and automated analysis—teams can substantially reduce their exposure. Security must be integrated from the outset of workflow design rather than bolted on afterward. As attackers increasingly target the CI layer, proactive hardening becomes essential for maintaining trust in modern software delivery pipelines.
Links:
[NDCMelbourne2025] Front End Testing with GitHub Actions – Amy Kapernick
In a dynamic session at NDC Melbourne 2025, Amy Kapernick, a seasoned front-end developer and advocate for automation, unveils a streamlined approach to front-end testing using GitHub Actions. With a focus on practicality, Amy guides developers through constructing a robust continuous integration and continuous deployment (CI/CD) pipeline, ensuring that front-end tests run seamlessly against live websites. Her presentation underscores the necessity of automation to maintain quality in web development, offering actionable insights for teams seeking to integrate testing into their workflows without manual intervention.
The Imperative of Front-End Testing
Amy begins by highlighting the unique challenges of front-end testing, emphasizing that unlike unit tests, which can operate with dummy data, front-end tests require a live, functioning website to evaluate real-world performance. For instance, assessing accessibility for visually impaired users or determining page load speeds demands an environment that mirrors production. Amy illustrates this with a CSS code snippet, questioning whether it can reveal unintended style bleed or performance bottlenecks without a live interface. By advocating for environments as close to production as possible, she ensures that tests yield accurate, actionable results, setting the stage for automation to eliminate manual testing inconsistencies.
Automating with GitHub Actions
The core of Amy’s approach lies in leveraging GitHub Actions to automate front-end testing within a CI/CD pipeline. She explains that GitHub Actions’ workflows, defined in YAML files, enable developers to trigger tests on specific events, such as pull requests to a production branch. Amy walks through creating a workflow with jobs like “build” and “test,” detailing steps such as checking out repository code, setting up Node.js, and installing dependencies. By using existing GitHub Actions packages, like those for checking out code and configuring Node, she simplifies the process, ensuring tests run consistently without manual effort. This automation, Amy notes, prevents code merges that fail tests, safeguarding application quality.
Deploying and Testing Live Websites
A pivotal aspect of Amy’s workflow involves deploying a live website for testing, using Netlify for its ease and deploy preview capabilities. She demonstrates a custom bash script to deploy to Netlify, addressing challenges like handling sensitive data, such as site IDs, which GitHub Actions may flag as secrets. Amy ingeniously encodes the deployment URL to bypass security restrictions, decoding it for testing with tools like Lighthouse and Playwright. These tools provide comprehensive reports on performance and UI functionality, respectively, which Amy configures to upload as artifacts, ensuring developers can review results and address issues before merging code.
Enhancing Workflows with Additional Automation
Beyond testing, Amy showcases GitHub Actions’ versatility by integrating a package that converts code comments into GitHub issues, ensuring tasks like “fix later” are tracked. This automation assigns issues to the code’s author and auto-closes them when resolved, streamlining project management. Amy also touches on other uses, such as linting, checking broken links, and generating assets like static tweet images for blog posts. These examples highlight how GitHub Actions can extend beyond testing to enhance overall development efficiency, making it a powerful tool for modern workflows.
Links:
[GopherConUK2025] How Just Eat Uses Tooling to Deploy Go Micro-services in Minutes
Lecturer
Ainsley Clark is a Senior Software Engineer at Just Eat, working within the Jet Connect team. He has been with the organisation for just over two years and maintains a strong professional focus on Go. His work centres on the design and operation of the internal microservice development toolkit that underpins large-scale order and menu processing across hundreds of partner integrations.
Abstract
This article describes the evolution and capabilities of GoKit, the internal microservice development toolkit developed by Just Eat’s Jet Connect team. Confronted with the maintenance burden of hundreds of independently forked services, the team replaced a template-based approach with a centralised code-generation and infrastructure-as-code system. The resulting tool scaffolds services, generates event consumers and producers, provisions cloud resources, produces continuous-integration workflows and emits operational metrics with minimal engineer effort. A concrete pizza-order example illustrates how a fully instrumented, event-driven service can be created and extended in minutes rather than days, allowing engineers to concentrate on business logic rather than infrastructure boilerplate.
From Monolith and Templates to a Centralised Toolkit
Jet Connect began life as Flight, an integration platform founded in 2013. Its purpose is to unify order and menu processing across restaurants, groceries and electronics partners so that a single point-of-sale interaction replaces multiple device-specific payloads. Today the platform serves approximately 731 000 partners across seventeen countries and processes hundreds of millions of orders annually. The Jet Connect engineering group itself comprises roughly fifty people and owns more than one hundred Go microservices.
The journey to that estate began with a PHP monolith that became increasingly difficult to change. Integrations were subsequently extracted into TypeScript services that spoke gRPC to the monolith. A GitHub template repository accelerated the creation of new integrations: an engineer could fork the template and obtain environment files, Helm charts and build workflows in minutes. The approach delivered speed in the early days yet exacted a heavy maintenance cost. A bug fix or standardisation change had to be applied manually to every fork. Inconsistent folder structures and coding patterns made context-switching expensive. End-to-end testing was absent, reducing confidence in releases.
The decision was therefore taken to move the entire integration layer to Go and to replace the template model with a purpose-built toolkit. The requirements were exacting: generate or update a repository in seconds; treat OpenAPI documentation as a first-class artefact; allow an engineer to subscribe to events by writing only a handler; hide infrastructure details such as databases, event buses and object storage; auto-generate continuous-integration and continuous-delivery pipelines; guarantee safe production deployment on merge to main; support capability testing across the full service graph; and provide a consistent local development experience with minimal setup. The resulting system is known internally as GoKit.
Scaffolding, Event Handling and Infrastructure as Code
GoKit meets those requirements through a combination of code generation and a declarative service description. The command gokit new <service-name> presents a short interactive questionnaire: does the service consume events, produce events, expose an HTTP server, require a database or object store? On the basis of the answers it scaffolds a consistent folder structure whose most important directories are cmd/app (containing a generated main) and internal (the sole location for custom business logic). A file named service.json acts as the single source of truth for infrastructure; it is consumed by Terraform templates that provision the necessary cloud resources.
A typical event handler receives an HTTP client, an event-bus producer and the incoming event (typed via reflection performed by GoKit). After performing domain work—calling a partner API, writing analytics, storing a payload—the handler emits a successor event. Registration consists of a single method call that associates the handler with a concrete event type. A subsequent gokit update rewrites service.json, updates generated code and refreshes documentation. The resulting README lists every consumed and produced topic, giving any engineer an immediate, accurate overview of the service’s responsibilities without requiring manual documentation effort.
Because the same service.json also drives resource provisioning, adding a DynamoDB table or an S3 bucket is a matter of inserting a few lines of JSON and re-running the update command. Generated interfaces for read/write operations and object-store upload/download encourage dependency injection and make unit testing straightforward. Should the underlying technology later change (for example from DynamoDB to PostgreSQL), only the implementation behind the interface needs to be altered; every consuming service continues to compile and run unchanged. The same mechanism supports vertical and horizontal scaling declarations—minimum replica counts, CPU and memory class—again expressed as simple keys in service.json.
Continuous Integration, Capability Testing and Observability
All continuous-integration workflows are themselves generated by GoKit. A change to a workflow template is made once in the central repository; every service inherits the update the next time gokit update is run. Capability tests spin up the entire relevant service graph under Docker, inject a correlation identifier into every request and event, and assert final outcomes. The approach scales to capability chains that involve ten or twenty intermediate services and provides high confidence before production deployment. Drift detection prevents accidental divergence: if an engineer edits a generated file without running the update command, continuous integration fails the pull request. Version pinning of GoKit itself ensures that services cannot merge while they lag behind a required toolkit version, producing a natural, incremental migration path across the estate.
Operational metrics appear automatically. Grafana dashboards display event rates, HTTP status codes, latency distributions, DynamoDB read/write performance and S3 operation counts without any additional instrumentation code. The same consistency that simplifies development also simplifies observation. When a product request arrives for analytics storage or an order-ready notification endpoint, the engineer adds a handful of lines to service.json or an OpenAPI specification, runs the update command, implements a short handler, and obtains a fully instrumented, tested and documented service ready for review.
Outcomes and Lessons
In the six years of its existence GoKit has allowed the Jet Connect team to update the entire service estate with a single pull request, to maintain uniform folder structures that eliminate costly context switches, and to keep documentation accurate without manual effort. Engineers spend their time on business problems—partner-specific payloads, analytics requirements, notification flows—rather than on Terraform, Helm or continuous-integration boilerplate. The toolkit has been instrumental in scaling the platform to hundreds of millions of orders while keeping the cognitive load on individual developers manageable.
The most transferable lessons are structural rather than technological. A single source of truth for service shape, aggressive code generation of repetitive artefacts, enforcement of consistency through continuous integration, and the deliberate abstraction of infrastructure behind stable interfaces together convert the chronic maintenance burden of a large microservice estate into a manageable, largely automated background process. The result is that complex event-driven workflows can be designed, implemented, tested and deployed in minutes rather than days, freeing engineering capacity for the differentiated work that actually delivers value to partners and customers.
The pizza-service walkthrough, though simplified for presentation, captures the essential rhythm of day-to-day work. An engineer begins with a short questionnaire, receives a fully structured repository, writes a handful of domain-specific lines, runs an update command, and obtains a service that already possesses continuous-integration pipelines, infrastructure declarations, metrics dashboards and capability-test scaffolding. Subsequent product requests—analytics storage, partner callbacks, new event types—are accommodated by small, localised edits rather than by the recreation of entire deployment artefacts. The cognitive load remains focused on the business problem; the mechanical work of packaging, provisioning and observing has been systematically removed from the critical path.
The same pattern scales. Whether the service handles a single partner’s CSV feed or participates in a multi-stage order-reconciliation flow involving a dozen intermediate services, the surrounding machinery remains identical. Consistency of structure, generation of repetitive artefacts and enforcement of hygiene through continuous integration together produce an estate that can grow without a proportional increase in operational friction. That is the practical payoff of the investment in a centralised toolkit.
Looking ahead, the same principles that make GoKit effective inside Just Eat are portable to any organisation that maintains a large collection of similarly structured services. The precise implementation will differ—different cloud providers, different event buses, different continuous-integration systems—but the underlying ideas remain constant: centralise the definition of service shape, generate everything that can be generated, enforce consistency automatically, and keep the engineer’s attention on the business problem. When those ideas are applied with discipline, the cost of creating and operating microservices falls dramatically, and the organisation’s capacity to respond to new partner or product requirements rises correspondingly.
In the end the value of a toolkit such as GoKit is measured less by the number of lines it generates than by the number of decisions it removes from the critical path of feature delivery. Every database, every topic subscription, every continuous-integration step that an engineer no longer has to configure by hand is a unit of attention that can be redirected toward the unique requirements of a partner or a product. When that redirection is systematic and reliable, the organisation as a whole becomes more responsive, more consistent and more capable of sustaining growth without a proportional increase in operational complexity.
The pizza-service narrative also illustrates a secondary benefit that is easy to overlook: the progressive enrichment of operational visibility. Because metrics, dashboards and capability tests are generated rather than hand-crafted, every new service automatically participates in the organisation’s observability and quality regimes. There is no opportunity for a service to be “forgotten” or to ship without the standard instrumentation. Consistency of tooling produces consistency of operational posture, which in turn reduces the cognitive load on both developers and on-call engineers.
Taken together, the practices embodied in GoKit demonstrate that the apparent tension between rapid delivery and long-term maintainability can be resolved by investing in the right abstractions at the platform level. When the platform absorbs the repetitive work of scaffolding, provisioning, testing and observing, individual service teams are free to move quickly without accumulating the structural debt that eventually slows every organisation that scales through pure copy-and-paste. The result is an engineering culture that can sustain high throughput while preserving the coherence and reliability required for a production system that processes hundreds of millions of orders.
The same principles that enabled Jet Connect to move from a maintenance-heavy template model to a coherent, generated estate are available to any organisation facing a similar proliferation of services. The precise technology choices—Terraform versus another infrastructure-as-code system, Kafka versus another event bus, OpenAPI versus another interface description—matter less than the architectural decision to centralise the definition of service shape and to generate the repetitive surrounding machinery. Once that decision is made and enforced, the cost of each additional service falls and the organisation’s ability to respond to new requirements rises. In an environment that processes hundreds of millions of orders, that difference is decisive.
In practical terms the toolkit converts what would otherwise be a multi-day or multi-week effort—creating a repository, wiring continuous integration, declaring infrastructure, adding metrics, writing documentation—into a sequence of minutes. The engineer’s attention remains on the unique aspects of the partner integration or the product feature. Everything else is supplied by generation and enforced by pipeline. That separation of concerns is the essential achievement, and it is the reason the platform can continue to scale without a corresponding explosion in operational overhead.
The experience of Jet Connect demonstrates that the investment in a carefully designed internal platform pays continuing dividends. Each new service inherits the accumulated learning of the entire estate; each improvement to the toolkit propagates automatically; each engineer inherits a consistent, well-instrumented starting point. The result is not merely faster delivery of individual features but a durable increase in the organisation’s capacity to absorb complexity without sacrificing reliability or developer effectiveness.
In short, GoKit is less a code generator than a deliberate architectural intervention that realigns incentives and removes friction from the path of delivery.
Links:
[DevoxxFR2025] Dagger Modules: A Swiss Army Knife for Modern CI/CD Pipelines
Continuous Integration and Continuous Delivery (CI/CD) pipelines are the backbone of modern software development, automating the process of building, testing, and deploying applications. However, as these pipelines grow in complexity, they often become difficult to maintain, debug, and port across different execution platforms, frequently relying on verbose and platform-specific YAML configurations. Jean-Christophe Sirot, in his presentation, introduced Dagger as a revolutionary approach to CI/CD, allowing pipelines to be written as code, executable locally, testable, and portable. He explored Dagger Functions and Dagger Modules as key concepts for creating and sharing reusable, language-agnostic components for CI/CD workflows, positioning Dagger as a versatile “Swiss Army knife” for modernizing these critical pipelines.
The Pain Points of Traditional CI/CD
Jean-Christophe began by outlining the common frustrations associated with traditional CI/CD pipelines. Relying heavily on YAML or other declarative formats for defining pipelines can lead to complex, repetitive, and hard-to-read configurations, especially for intricate workflows. Debugging failures within these pipelines is often challenging, requiring pushing changes to a remote CI server and waiting for the pipeline to run. Furthermore, pipelines written for one CI platform (like GitHub Actions or GitLab CI) are often not easily transferable to another, creating vendor lock-in and hindering flexibility. This dependency on specific platforms and the difficulty in managing complex workflows manually are significant pain points for development and DevOps teams.
Dagger: CI/CD as Code
Dagger offers a fundamentally different approach by treating CI/CD pipelines as code. It allows developers to write their pipeline logic using familiar programming languages (like Go, Python, Java, or TypeScript) instead of platform-specific configuration languages. This brings the benefits of software development practices – such as code reusability, modularity, testing, and versioning – to CI/CD. Jean-Christophe explained that Dagger executes these pipelines using containers, ensuring consistency and portability across different environments. The Dagger engine runs the pipeline logic, orchestrates the necessary container operations, and manages dependencies. This allows developers to run and debug their CI/CD pipelines locally using the same code that will execute on the remote CI platform, significantly accelerating the debugging cycle.
Dagger Functions and Modules
Key to Dagger’s power are Dagger Functions and Dagger Modules. Jean-Christophe described Dagger Functions as the basic building blocks of a pipeline – functions written in a programming language that perform specific CI/CD tasks (e.g., building a Docker image, running tests, deploying an application). These functions interact with the Dagger engine to perform container operations. Dagger Modules are collections of related Dagger Functions that can be packaged and shared. Modules allow teams to create reusable components for common CI/CD patterns or specific technologies, effectively creating a library of CI/CD capabilities. For example, a team could create a “Java Build Module” containing functions for compiling Java code, running Maven or Gradle tasks, and building JAR or WAR files. These modules can be easily imported and used in different projects, promoting standardization and reducing duplication across an organization’s CI/CD workflows. Jean-Christophe demonstrated how to create and use Dagger Modules, illustrating their potential for building composable and maintainable pipelines. He highlighted that Dagger’s language independence means that modules can be written in one language (e.g., Python) and used in a pipeline defined in another (e.g., Java), fostering collaboration between teams with different language preferences.
The Benefits: Composable, Maintainable, Portable
By adopting Dagger, teams can create CI/CD pipelines that are:
– Composable: Pipelines can be built by combining smaller, reusable Dagger Modules and Functions.
– Maintainable: Pipelines written as code are easier to read, understand, and refactor using standard development tools and practices.
– Portable: Pipelines can run on any platform that supports Dagger and containers, eliminating vendor lock-in.
– Testable: Individual Dagger Functions and modules can be unit tested, and the entire pipeline can be run and debugged locally.
Jean-Christophe’s presentation positioned Dagger as a versatile tool that modernizes CI/CD by bringing the best practices of software development to pipeline automation. The ability to write pipelines in code, leverage reusable modules, and execute locally makes Dagger a powerful “Swiss Army knife” for developers and DevOps engineers seeking more efficient, reliable, and maintainable CI/CD workflows.
Links:
- Jean-Christophe Sirot: https://www.linkedin.com/in/jcsirot/
- Decathlon: https://www.decathlon.com/
- Dagger: https://dagger.io/
- Devoxx France LinkedIn: https://www.linkedin.com/company/devoxx-france/
- Devoxx France Bluesky: https://bsky.app/profile/devoxx.fr
- Devoxx France Website: https://www.devoxx.fr/
[DefCon32] OH MY DC: Abusing OIDC All the Way to Your Cloud
As organizations migrate from static credentials to dynamic authentication protocols, overlooked intricacies in implementations create fertile ground for exploitation. Aviad Hahami, a security researcher at Palo Alto Networks, demystifies OpenID Connect (OIDC) in the context of continuous integration and deployment (CI/CD) workflows. His examination reveals vulnerabilities stemming from under-configurations and misconfigurations, enabling unauthorized access to cloud environments. By alternating perspectives among users, identity providers, and CI vendors, Aviad illustrates attack vectors that compromise sensitive resources.
Aviad begins with foundational concepts, clarifying OIDC’s role in secure, short-lived token exchanges. In CI/CD scenarios, tools like GitHub Actions request tokens from identity providers (IdPs) such as GitHub’s OIDC provider. These tokens, containing claims like repository names and commit SHAs, are validated by workload identity federations (WIFs) in clouds like AWS or Azure. Proper configuration ensures tokens originate from trusted sources, but lapses invite abuse.
Common pitfalls include wildcard allowances in policies, permitting access from unintended repositories. Aviad demonstrates how fork pull requests (PRs) exploit these, granting cloud roles without maintainer approval. Such “no configs” scenarios, where minimal effort yields high rewards, underscore the need for precise claim validations.
Advanced Configurations and Misconfigurations
Delving deeper, Aviad explores “advanced configs” that inadvertently become misconfigurations. Features like GitHub’s ID token requests for forks introduce risks if not explicitly enabled. He recounts discovering a vulnerability in CircleCI, where reusable configurations allowed token issuance to forks, bypassing protections.
Shifting to the IdP viewpoint, Aviad discloses a real-world flaw in a popular CI vendor, permitting token claims from any repository within an organization. This enabled cross-project escalations, compromising clouds via simple PRs. Reported responsibly, the issue prompted fixes, emphasizing the cascading effects of IdP errors.
He references Tinder’s research on similar WIF misconfigurations, reinforcing that even sophisticated setups falter without rigorous claim scrutiny.
Exploitation Through CI Vendors
Aviad pivots to CI vendor responsibilities, highlighting how their token issuance logic influences downstream security. In CircleCI’s case, a bug allowed organization-wide token claims, exposing multiple projects. By requesting tokens in fork contexts, attackers could satisfy broad WIF conditions, accessing clouds undetected.
Remediation involved opt-in mechanisms for fork tokens, mirroring GitHub’s approach. Aviad stresses learning claim origins per IdP, avoiding wildcards, and hardening pipelines to prevent trivial breaches.
His tool for auditing Azure CLI configurations exemplifies proactive defense, aiding in identifying exposed resources.
Broader Implications for Secure Authentication
Aviad’s insights extend beyond CI/CD, advocating holistic OIDC understanding to thwart supply chain attacks. By dissecting entity interactions—users, IdPs, and clouds—he equips practitioners to craft resilient policies.
Encouraging bounty hunters to probe these vectors, he underscores OIDC’s maturity yet persistent gaps. Ultimately, robust configurations transform OIDC from vulnerability to asset, safeguarding digital infrastructures.
Links:
[DevoxxFR2014] Tips and Tricks for Releasing with Maven, Hudson, Artifactory, and Git: Streamlining Software Delivery
Lecturer
Michael Hüttermann, a freelance DevOps consultant from Germany, specializes in optimizing software delivery pipelines. With a background in Java development and continuous integration, he has authored books on Agile ALM and contributes to open-source projects. His expertise lies in integrating tools like Maven, Jenkins (formerly Hudson), Artifactory, andទ
System: ## Git, and Maven to create efficient release processes. His talk at Devoxx France 2014 shares practical strategies for streamlining software releases, drawing on his extensive experience in DevOps consulting.
Abstract
Releasing software with Maven can be a cumbersome process, often fraught with manual steps and configuration challenges, despite Maven’s strengths as a build tool. In this lecture from Devoxx France 2014, Michael Hüttermann presents a comprehensive guide to optimizing the release process by integrating Maven with Hudson (now Jenkins), Artifactory, and Git. He explores the limitations of Maven’s release plugin and offers lightweight alternatives that enhance automation, traceability, and efficiency. Through detailed examples and best practices, Hüttermann demonstrates how to create a robust CI/CD pipeline that leverages version control, binary management, and continuous integration to deliver software reliably. The talk emphasizes practical configurations, common pitfalls, and strategies for achieving seamless releases in modern development workflows.
The Challenges of Maven Releases
Maven is a powerful build tool that simplifies dependency management and build automation, but its release plugin can be rigid and complex. Hüttermann explains that the plugin often requires manual version updates, tagging, and deployment steps, which can disrupt workflows and introduce errors. For example, the mvn release:prepare and mvn release:perform commands automate versioning and tagging, but they lack flexibility for custom workflows and can fail if network issues or repository misconfigurations occur.
Hüttermann advocates for a more integrated approach, combining Maven with Hudson, Artifactory, and Git to create a streamlined pipeline. This integration addresses key challenges: ensuring reproducible builds, managing binary artifacts, and maintaining version control integrity.
Building a CI/CD Pipeline with Hudson
Hudson, now known as Jenkins, serves as the orchestration hub for the release process. Hüttermann describes a multi-stage pipeline that automates building, testing, and deploying Maven projects. A typical Jenkins pipeline might look like this:
pipeline {
agent any
stages {
stage('Checkout') {
steps {
git url: 'https://github.com/example/repo.git', branch: 'main'
}
}
stage('Build') {
steps {
sh 'mvn clean package'
}
}
stage('Deploy') {
steps {
sh 'mvn deploy -DskipTests'
}
}
}
}
The pipeline connects to a Git repository, builds the project with Maven, and deploys artifacts to Artifactory. Hüttermann emphasizes the importance of parameterized builds, allowing developers to specify release versions or snapshot flags dynamically.
Leveraging Artifactory for Binary Management
Artifactory, a binary repository manager, plays a critical role in storing and distributing Maven artifacts. Hüttermann highlights its ability to manage snapshots and releases, ensuring traceability and reproducibility. Artifacts are deployed to Artifactory using Maven’s deploy goal:
mvn deploy -DaltDeploymentRepository=artifactory::default::http://artifactory.example.com/releases
This command uploads artifacts to a specified repository, with Artifactory providing metadata for dependency resolution. Hüttermann notes that Artifactory’s cloud-based hosting simplifies access for distributed teams, and its integration with Jenkins via plugins enables automated deployment.
Git Integration for Version Control
Git serves as the version control system, managing source code and enabling release tagging. Hüttermann recommends using Git commit hashes to track builds, ensuring traceability. A typical release process involves creating a tag:
git tag -a v1.0.0 -m "Release 1.0.0"
git push origin v1.0.0
Jenkins’ Git plugin automates checkout and tagging, reducing manual effort. Hüttermann advises using a release branch for stable versions, with snapshots developed on main to maintain a clear workflow.
Streamlining the Release Process
To overcome the limitations of Maven’s release plugin, Hüttermann suggests custom scripts and Jenkins pipelines to automate versioning and deployment. For example, a script to increment version numbers in the pom.xml file can be integrated into the pipeline:
mvn versions:set -DnewVersion=1.0.1
This approach allows fine-grained control over versioning, avoiding the plugin’s rigid conventions. Hüttermann also recommends using Artifactory’s snapshot repositories for development builds, with stable releases moved to release repositories after validation.
Common Pitfalls and Best Practices
Network connectivity issues can disrupt deployments, as Hüttermann experienced during a demo when a Jenkins job failed due to a network outage. He advises configuring retry mechanisms in Jenkins and using Artifactory’s caching to mitigate such issues. Another pitfall is version conflicts in multi-module projects; Hüttermann suggests enforcing consistent versioning across modules with Maven’s versions plugin.
Best practices include maintaining a clean workspace, using Git commit hashes for traceability, and integrating unit tests into the pipeline to ensure quality. Hüttermann also emphasizes the importance of separating source code (stored in Git) from binaries (stored in Artifactory) to maintain a clear distinction between development and deployment artifacts.
Practical Demonstration and Insights
During the lecture, Hüttermann demonstrates a Jenkins pipeline that checks out code from Git, builds a Maven project, and deploys artifacts to Artifactory. The pipeline includes parameters for release candidates and stable versions, showcasing flexibility. He highlights the use of Artifactory’s generic integration, which supports any file type, making it versatile for non-Maven artifacts.
The demo illustrates a three-stage process: building a binary, copying it to a workspace, and deploying it to Artifactory. Despite a network-related failure, Hüttermann uses the opportunity to discuss resilience, recommending offline capabilities and robust error handling.
Broader Implications for DevOps
The integration of Maven, Hudson, Artifactory, and Git aligns with DevOps principles of automation and collaboration. By automating releases, teams reduce manual errors and accelerate delivery, critical for agile development. Hüttermann’s approach supports both small startups and large enterprises, offering scalability through cloud-based Artifactory and Jenkins.
For developers, the talk provides actionable strategies to simplify releases, while organizations benefit from standardized pipelines that ensure compliance and traceability. The emphasis on lightweight processes challenges traditional heavy release cycles, promoting continuous delivery.
Conclusion: A Blueprint for Efficient Releases
Michael Hüttermann’s lecture offers a practical roadmap for streamlining software releases using Maven, Hudson, Artifactory, and Git. By addressing the shortcomings of Maven’s release plugin and leveraging integrated tools, developers can achieve automated, reliable, and efficient release processes. The talk underscores the importance of CI/CD pipelines in modern software engineering, providing a foundation for DevOps success.
Links
[DevoxxFR2014] Runtime stage
FROM nginx:alpine
COPY –from=builder /app/dist /usr/share/nginx/html
EXPOSE 80
This pattern reduces final image size from hundreds of megabytes to tens of megabytes. **Layer caching** optimization requires careful instruction ordering:
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
Copying dependency manifests first maximizes cache reuse during development.
## Networking Models and Service Discovery
Docker’s default bridge network isolates containers on a single host. Production environments demand multi-host communication. **Overlay networks** create virtual networks across swarm nodes:
docker network create –driver overlay –attachable prod-net
docker service create –network prod-net –name api myapp:latest
Docker’s built-in DNS enables service discovery by name. For external traffic, **ingress routing meshes** like Traefik or NGINX provide load balancing, TLS termination, and canary deployments.
## Persistent Storage for Stateful Applications
Stateless microservices dominate container use cases, but databases and queues require durable storage. **Docker volumes** offer the most flexible solution:
docker volume create postgres-data
docker run -d \
–name postgres \
-v postgres-data:/var/lib/postgresql/data \
-e POSTGRES_PASSWORD=secret \
postgres:13
For distributed environments, **CSI (Container Storage Interface)** plugins integrate with Ceph, GlusterFS, or cloud-native storage like AWS EBS.
## Orchestration and Automated Operations
Docker Swarm provides native clustering with zero external dependencies:
docker swarm init
docker stack deploy -c docker-compose.yml myapp
“`
For advanced workloads, Kubernetes offers:
– Deployments for rolling updates and self-healing.
– Horizontal Pod Autoscaling based on CPU/memory or custom metrics.
– ConfigMaps and Secrets for configuration management.
Migration paths typically begin with stateless services in Swarm, then progress to Kubernetes for stateful and machine-learning workloads.
Security Hardening and Compliance
Production containers must follow security best practices:
– Run as non-root users: USER appuser in Dockerfile.
– Scan images with Trivy or Clair in CI/CD pipelines.
– Apply seccomp and AppArmor profiles to restrict system calls.
– Use RBAC and Network Policies in Kubernetes to enforce least privilege.
Production Case Studies and Operational Wisdom
Spotify manages thousands of microservices using Helm charts and custom operators. Airbnb leverages Kubernetes for dynamic scaling during peak booking periods. The New York Times uses Docker for CI/CD acceleration, reducing deployment time from hours to minutes.
Common lessons include:
– Monitor with Prometheus and Grafana.
– Centralize logs with ELK or Loki.
– Implement distributed tracing with Jaeger or Zipkin.
– Use chaos engineering to validate resilience.
Strategic Impact on DevOps Culture
Docker fundamentally accelerates the CI/CD pipeline and enables immutable infrastructure. Success requires cultural alignment: developers embrace infrastructure-as-code, operations teams adopt GitOps workflows, and security integrates into every stage. Orchestration platforms bridge the gap between development velocity and operational stability.