Jonathan Lalou's Blog

Posts Tagged ‘Kubernetes’

TL;DR

Kubernetes may look like an overly complicated version of Docker Compose, but it operates on a different level entirely. Where Compose excels at quick, local orchestration of containers, Kubernetes is a robust, distributed platform designed for automated scaling, fault-tolerance, and production-grade deployments across multi-node clusters. This article provides a comprehensive comparison and shows how ArgoCD enhances GitOps-based Kubernetes workflows.

Docker Compose vs Kubernetes – Similarities and First Impressions

At a high level, Docker Compose and Kubernetes share similar concepts: containers, services, configuration, and volumes. This often leads to the assumption that Kubernetes is just a verbose, harder-to-write Compose replacement. However, Kubernetes is more than a runtime. It’s a control plane, a state manager, and a policy enforcer.

Real-Life Use Cases – Docker Compose vs Kubernetes Examples

Tomcat + Oracle + MongoDB + NGINX Stack

Docker Compose

version: '3'
services:
  nginx:
    image: nginx:latest
    ports:
      - "80:80"
    depends_on:
      - tomcat

  tomcat:
    image: tomcat:9
    ports:
      - "8080:8080"
    environment:
      DB_URL: jdbc:oracle:thin:@oracle:1521:orcl

  oracle:
    image: oracle/database:19.3.0-ee
    environment:
      ORACLE_PWD: secretpass
    volumes:
      - oracle-data:/opt/oracle/oradata

  mongo:
    image: mongo:5
    volumes:
      - mongo-data:/data/db

volumes:
  oracle-data:
  mongo-data:

Kubernetes Equivalent

• Each service becomes a Deployment and a Service.
• Environment variables and passwords are stored in Secrets.
• Volumes are defined with PVC and StorageClass.

apiVersion: v1
kind: Secret
metadata:
  name: oracle-secret
type: Opaque
data:
  ORACLE_PWD: c2VjcmV0cGFzcw==

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat
spec:
  replicas: 2
  selector:
    matchLabels:
      app: tomcat
  template:
    metadata:
      labels:
        app: tomcat
    spec:
      containers:
      - name: tomcat
        image: tomcat:9
        ports:
        - containerPort: 8080
        env:
        - name: DB_URL
          value: jdbc:oracle:thin:@oracle:1521:orcl

NodeJS + Express + MySQL + NGINX

Docker Compose

services:
  mysql:
    image: mysql:8
    environment:
      MYSQL_ROOT_PASSWORD: rootpass
    volumes:
      - mysql-data:/var/lib/mysql

  api:
    build: ./api
    environment:
      DB_USER: root
      DB_PASS: rootpass
      DB_HOST: mysql

  nginx:
    image: nginx:latest
    ports:
      - "80:80"

Kubernetes Equivalent

apiVersion: v1
kind: Secret
metadata:
  name: mysql-secret
type: Opaque
data:
  MYSQL_ROOT_PASSWORD: cm9vdHBhc3M=
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 2
  template:
    spec:
      containers:
      - name: api
        image: node-app:latest
        env:
        - name: DB_PASS
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: MYSQL_ROOT_PASSWORD

⚙️ Docker Compose vs kubectl – Command Mapping

ArgoCD – GitOps Made Practical

ArgoCD is a Kubernetes-native continuous deployment tool. It uses Git as the single source of truth, enabling declarative infrastructure and GitOps workflows.

✨ Key Features

• Declarative sync of Git and cluster state
• Drift detection and automatic repair
• Multi-environment and multi-namespace support
• CLI and Web UI available

Example ArgoCD Commands

argocd login argocd.myorg.com
argocd app create my-app \
  --repo https://github.com/org/app.git \
  --path k8s \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace production

argocd app sync my-app
argocd app get my-app
argocd app diff my-app

Sample ArgoCD Application Manifest

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-api
spec:
  destination:
    namespace: default
    server: https://kubernetes.default.svc
  project: default
  source:
    path: k8s/app
    repoURL: https://github.com/org/api.git
    targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

✅ Conclusion

Docker Compose is perfect for prototyping and local dev. Kubernetes is built for cloud-native workloads, distributed systems, and high availability. ArgoCD makes declarative, Git-based continuous deployment simple, scalable, and observable.

Saeid Bostandoust, founder of CubeDemi.io, delivered an in-depth presentation at KCDUK2024 on Kubernetes runtime security, focusing on tools and techniques to secure containers during execution. As a CNCF project contributor, Saeid explored Linux security features like Linux Capabilities, SELinux, AppArmor, Seccomp-BPF, and KubeArmor, providing a comprehensive overview of how these can be applied in Kubernetes to mitigate zero-day attacks and enforce policies. His talk emphasized practical implementation, observability, and policy enforcement, aligning with KCDUK2024’s focus on securing cloud-native environments.

Understanding Runtime Security

Saeid defined runtime security as protecting applications during execution, contrasting it with pre-runtime measures like static code analysis. Runtime security focuses on mitigating zero-day attacks and malicious behavior through real-time intrusion detection, process isolation, policy enforcement, and monitoring. Linux offers over 30 security mechanisms, including SELinux, AppArmor, Linux Capabilities, Seccomp-BPF, and namespaces, alongside kernel drivers to counter threats like Meltdown and Spectre. Saeid focused on well-known features, explaining their roles and Kubernetes integration.

Linux Capabilities: Historically, processes were either privileged (with full root permissions) or unprivileged, leading to vulnerabilities like privilege escalation via commands like ping. Linux Capabilities, introduced to granularly assign permissions, allow processes to perform specific actions (e.g., opening raw sockets for ping) without full root privileges. In Kubernetes, capabilities can be configured in pod manifests to drop unnecessary permissions, enhancing security even for root-run containers.

Seccomp-BPF: Seccomp (Secure Computing) restricts system calls a process can make. Originally limited to basic calls (read, write, exit), Seccomp-BPF extends this with customizable profiles. In Kubernetes, a Seccomp-BPF profile can be defined in a JSON file and applied via a pod’s security context, terminating processes that attempt unauthorized system calls. Saeid demonstrated a restrictive profile that limits a container to basic operations, preventing it from running if additional system calls are needed.

LSM Modules (AppArmor, SELinux, BPF-LSM): Linux Security Modules (LSM) provide hooks to intercept operations, such as file access or network communication. AppArmor uses path-based profiles, while SELinux employs label-based policies. BPF-LSM, a newer option, allows dynamic policy injection via eBPF code, offering flexibility without requiring application restarts. Saeid noted that BPF-LSM, available in kernel versions 5.7 and later, supports stacking with other LSMs, enhancing Kubernetes security.

KubeArmor: Simplifying Policy Enforcement

KubeArmor, a CNCF project, simplifies runtime security by allowing users to define policies via Kubernetes Custom Resource Definitions (CRDs). These policies are translated into AppArmor, SELinux, or BPF-LSM profiles, depending on the worker node’s LSM. KubeArmor addresses the challenge of syncing profiles across cluster nodes, automating deployment and updates. It uses eBPF for observability, monitoring system calls and generating telemetry for tools like Prometheus and Elasticsearch. Saeid showcased KubeArmor’s architecture, including a daemon set with an init container for compiling eBPF code and a relay server for aggregating logs and alerts.

An example policy demonstrated KubeArmor denying access to sensitive files (e.g., /etc/passwd, /etc/shadow) and commands (e.g., apt, apt-get), with logs showing enforcement details. Unlike manual AppArmor or SELinux profiles, which are complex and hard to scale, KubeArmor’s declarative approach and default deny policies simplify securing containers, preventing access to dangerous assets like /proc mounts.

Practical Implementation and Community Engagement

Saeid provided practical examples, such as configuring a pod to drop all capabilities except those needed, applying a Seccomp-BPF profile to restrict system calls, and using KubeArmor to enforce file access policies. He highlighted KubeArmor’s integration with tools like OPA Gatekeeper to block unauthorized commands (e.g., kubectl exec) when BPF-LSM is unavailable. For further learning, Saeid offered a 50% discount on CubeDemi.io’s container security workshop, encouraging KCDUK2024 attendees to deepen their Kubernetes security expertise.

Links:

• CubeDemi.io website
• KubeArmor CNCF project
• Video: Deep Dive into Kubernetes Runtime Security

Hashtags: #Kubernetes #RuntimeSecurity #KubeArmor #CNCF #LinuxSecurity #SaeidBostandoust #KCDUK2024

At Devoxx France 2019, Charles Sabourdin and Jean-Christophe Sirot, seasoned professionals in cloud-native technologies, delivered an extensive exploration of managing application lifecycles within Kubernetes. Charles, an architect with over 15 years in Linux and Java, and Jean-Christophe, a Docker expert since 2002, combined their expertise to demystify Docker’s underpinnings, Kubernetes’ orchestration, and the practicalities of continuous integration and delivery (CI/CD). Through demos and real-world insights, they addressed security challenges across development and business-as-usual (BAU) phases, proposing organizational strategies to streamline containerized workflows. This post captures their comprehensive session, offering a roadmap for developers and operations teams navigating Kubernetes ecosystems.

Docker’s Foundations: Isolation and Layered Efficiency

Charles opened the session by revisiting Docker’s core principles, emphasizing its reliance on Linux kernel features like namespaces and control groups (cgroups). Unlike virtual machines (VMs), which bundle entire operating systems, Docker containers share the host kernel, isolating processes within lightweight environments. This design achieves hyper-density, allowing more containers to run on a single machine compared to VMs. Charles demonstrated launching a container, highlighting its process isolation using commands like ps within a containerized bash session, contrasting it with the host’s process list. He introduced Docker’s layer system, where images are built as immutable, stacked deltas, optimizing storage through shared base layers. Tools like Dive, he noted, help inspect these layers, revealing command histories and suggesting size optimizations. This foundation sets the stage for Kubernetes, enabling efficient, portable application delivery across environments.

Kubernetes: Orchestrating Scalable Deployments

Jean-Christophe transitioned to Kubernetes, describing it as a resource orchestrator that manages containerized applications across node pools. Kubernetes abstracts infrastructure complexities, using declarative configurations to maintain desired application states. Key components include pods—the smallest deployable units housing containers—replica sets for scaling, and deployments for managing updates. Charles demonstrated creating a namespace and deploying a sample application using kubectl run, which scaffolds deployments, replica sets, and pods. He showcased rolling updates, where Kubernetes progressively replaces pods to ensure zero downtime, configurable via parameters like maxSurge and maxUnavailable. The duo emphasized Kubernetes’ auto-scaling capabilities, which adjust pod counts based on load, and the importance of defining resource limits to prevent performance bottlenecks. Their demo underscored Kubernetes’ role in achieving resilient, scalable deployments, aligning with hyper-density goals.

CI/CD Pipelines: Propagating Versions Seamlessly

The session delved into CI/CD pipelines, illustrating how Docker tags facilitate version propagation across development, pre-production, and production environments. Charles outlined a standard process: developers build Docker images tagged with version numbers (e.g., 1.1, 1.2) or environment labels (e.g., prod, staging). These images, stored in registries like Docker Hub or private repositories, are pulled by Kubernetes clusters for deployment. Jean-Christophe highlighted debates around tagging strategies, noting that version-based tags ensure traceability, while environment tags simplify environment-specific deployments. Their demo integrated tools like Jenkins and JFrog Artifactory, automating builds, tests, and deployments. They stressed the need for robust pipeline configurations to avoid resource overuse, citing Jenkins’ default manual build triggers for tagged releases as a safeguard. This pipeline approach ensures consistent, automated delivery, bridging development and production.

Security Across the Lifecycle: Development vs. BAU

Security emerged as a central theme, with Charles contrasting development and BAU phases. During development, teams rapidly address Common Vulnerabilities and Exposures (CVEs) with frequent releases, leveraging tools like JFrog Xray and Clair to scan images for vulnerabilities. Xray integrates with Artifactory, while Clair, an open-source solution, scans registry images for known CVEs. However, in BAU, where releases are less frequent, unpatched vulnerabilities pose greater risks. Charles shared an anecdote about a PHP project where a dependency switch broke builds after two years, underscoring the need for ongoing maintenance. They advocated for practices like running containers in read-only mode and using non-root users to minimize attack surfaces. Tools like OWASP Dependency-Track, they suggested, could enhance visibility into library vulnerabilities, though current scanners often miss non-package dependencies. This dichotomy highlights the need for automated, proactive security measures throughout the lifecycle.

Organizational Strategies: Balancing Complexity and Responsibility

Drawing from their experiences, Charles and Jean-Christophe proposed organizational solutions to manage Kubernetes complexity. They introduced a “1-2-3 model” for image management: Level 1 uses vendor-provided images (e.g., official MySQL images) managed by operations; Level 2 involves base images built by dedicated teams, incorporating standardized tooling; and Level 3 allows project-specific images, with teams assuming maintenance responsibilities. This model clarifies ownership, reducing risks like disappearing maintainers when projects transition to BAU. They emphasized cross-team collaboration, encouraging developers and operations to share knowledge and align on practices like Dockerfile authorship and resource allocation in YAML configurations. Charles reflected on historical DevOps silos, advocating for shared vocabularies and traceable decisions to navigate evolving best practices. Their return-of-experience underscored the importance of balancing automation with human oversight to maintain robust, secure Kubernetes environments.

Links:

• Devoxx France 2019 Video
• Kubernetes Documentation
• Docker Documentation
• JFrog Xray Documentation
• Clair GitHub Repository
• OWASP Dependency-Track
• Dive GitHub Repository

Hashtags: #Kubernetes #Docker #DevOps #CICD #Security #DevoxxFR #CharlesSabourdin #JeanChristopheSirot #JFrog #Clair