Archive for the ‘en-US’ Category
[DevoxxGR2024] Meet Your New AI Best Friend: LangChain at Devoxx Greece 2024 by Henry Lagarde
At Devoxx Greece 2024, Henry Lagarde, a senior software engineer at Criteo, introduced audiences to LangChain, a versatile framework for building AI-powered applications. With infectious enthusiasm and live demonstrations, Henry showcased how LangChain simplifies interactions with large language models (LLMs), enabling developers to create context-aware, reasoning-driven tools. His talk, rooted in his experience at Criteo, a leader in retargeting and retail media, highlighted LangChain’s composability and community-driven evolution, offering a practical guide for AI integration.
LangChain’s Ecosystem and Composability
Henry began by defining LangChain as a framework for building context-aware reasoning applications. Unlike traditional LLM integrations, LangChain provides modular components—prompt templates, LLM abstractions, vector stores, text splitters, and document loaders—that integrate with external services rather than hosting them. This composability allows developers to switch LLMs seamlessly, adapting to changes in cost or performance without rewriting code. Henry emphasized LangChain’s open-source roots, launched in late 2022, and its rapid growth, with versions in Python, TypeScript, Java, and more, earning it the 2023 New Tool of the Year award.
The ecosystem extends beyond core modules to include LangServe for REST API deployment, LangSmith for monitoring, and a community hub for sharing prompts and agents. This holistic approach supports developers from prototyping to production, making LangChain a cornerstone for AI engineering.
Building a Chat Application
In a live demo, Henry showcased LangChain’s simplicity by recreating a ChatGPT-like application in under 10 lines of Python code. He instantiated an OpenAI client using GPT-3.5 Turbo, implemented chat history for context awareness, and used prompt templates to define system and human messages. By combining these components, he enabled streaming responses, mimicking ChatGPT’s real-time output without the $20 monthly subscription. This demonstration highlighted LangChain’s ability to handle memory, input/output formatting, and LLM interactions with minimal effort, empowering developers to build cost-effective alternatives.
Henry noted that LangChain’s abstractions, such as strong typing and output parsing, eliminate manual prompt engineering, ensuring robust integrations even when APIs change. The demo underscored the framework’s accessibility, inviting developers to experiment with its capabilities.
Creating an AI Agent for PowerPoint Generation
Henry’s second demo illustrated LangChain’s advanced features by building an AI agent to generate PowerPoint presentations. Using TypeScript, he configured a system prompt from LangSmith’s community hub, defining the agent’s tasks: researching a topic via the Serper API and generating a structured PowerPoint. He defined tools with Zod for runtime type checking, ensuring consistent outputs, and integrated callbacks for UI tracing and monitoring.
The agent, powered by Anthropic’s Claude model, performed internet research on Google Cloud, compiled findings, and generated a presentation with sourced information. Despite minor delays, the demo showcased LangChain’s ability to orchestrate complex workflows, combining research, data processing, and content creation. Henry’s use of LangSmith for prompt optimization and monitoring highlighted the framework’s production-ready capabilities.
Community and Cautions
Henry emphasized LangChain’s vibrant community, which drives its multi-language support and rapid evolution. He encouraged attendees to contribute, noting the framework’s open-source ethos and resources like GitHub for further exploration. However, he cautioned against over-reliance on LLMs, citing their occasional laziness or errors, as seen in ChatGPT’s simplistic responses. LangChain, he argued, augments developer workflows but requires careful integration to ensure reliability in production environments.
His vision for LangChain is one of empowerment, enabling developers to enhance applications incrementally while maintaining control over AI-driven processes. By sharing his demo code on GitHub, Henry invited attendees to experiment and contribute to LangChain’s growth.
Conclusion
Henry’s presentation at Devoxx Greece 2024 was a compelling introduction to LangChain’s potential. Through practical demos and insightful commentary, he demonstrated how the framework simplifies AI development, from basic chat applications to sophisticated agents. His emphasis on composability, community, and cautious integration resonated with developers eager to explore AI. As LangChain continues to evolve, Henry’s talk serves as a blueprint for harnessing its capabilities in real-world applications.
Links:
[KCD UK 2024] Deep Dive into Kubernetes Runtime Security
Saeid Bostandoust, founder of CubeDemi.io, delivered an in-depth presentation at KCDUK2024 on Kubernetes runtime security, focusing on tools and techniques to secure containers during execution. As a CNCF project contributor, Saeid explored Linux security features like Linux Capabilities, SELinux, AppArmor, Seccomp-BPF, and KubeArmor, providing a comprehensive overview of how these can be applied in Kubernetes to mitigate zero-day attacks and enforce policies. His talk emphasized practical implementation, observability, and policy enforcement, aligning with KCDUK2024’s focus on securing cloud-native environments.
Understanding Runtime Security
Saeid defined runtime security as protecting applications during execution, contrasting it with pre-runtime measures like static code analysis. Runtime security focuses on mitigating zero-day attacks and malicious behavior through real-time intrusion detection, process isolation, policy enforcement, and monitoring. Linux offers over 30 security mechanisms, including SELinux, AppArmor, Linux Capabilities, Seccomp-BPF, and namespaces, alongside kernel drivers to counter threats like Meltdown and Spectre. Saeid focused on well-known features, explaining their roles and Kubernetes integration.
Linux Capabilities: Historically, processes were either privileged (with full root permissions) or unprivileged, leading to vulnerabilities like privilege escalation via commands like ping. Linux Capabilities, introduced to granularly assign permissions, allow processes to perform specific actions (e.g., opening raw sockets for ping) without full root privileges. In Kubernetes, capabilities can be configured in pod manifests to drop unnecessary permissions, enhancing security even for root-run containers.
Seccomp-BPF: Seccomp (Secure Computing) restricts system calls a process can make. Originally limited to basic calls (read, write, exit), Seccomp-BPF extends this with customizable profiles. In Kubernetes, a Seccomp-BPF profile can be defined in a JSON file and applied via a pod’s security context, terminating processes that attempt unauthorized system calls. Saeid demonstrated a restrictive profile that limits a container to basic operations, preventing it from running if additional system calls are needed.
LSM Modules (AppArmor, SELinux, BPF-LSM): Linux Security Modules (LSM) provide hooks to intercept operations, such as file access or network communication. AppArmor uses path-based profiles, while SELinux employs label-based policies. BPF-LSM, a newer option, allows dynamic policy injection via eBPF code, offering flexibility without requiring application restarts. Saeid noted that BPF-LSM, available in kernel versions 5.7 and later, supports stacking with other LSMs, enhancing Kubernetes security.
KubeArmor: Simplifying Policy Enforcement
KubeArmor, a CNCF project, simplifies runtime security by allowing users to define policies via Kubernetes Custom Resource Definitions (CRDs). These policies are translated into AppArmor, SELinux, or BPF-LSM profiles, depending on the worker node’s LSM. KubeArmor addresses the challenge of syncing profiles across cluster nodes, automating deployment and updates. It uses eBPF for observability, monitoring system calls and generating telemetry for tools like Prometheus and Elasticsearch. Saeid showcased KubeArmor’s architecture, including a daemon set with an init container for compiling eBPF code and a relay server for aggregating logs and alerts.
An example policy demonstrated KubeArmor denying access to sensitive files (e.g., /etc/passwd, /etc/shadow) and commands (e.g., apt, apt-get), with logs showing enforcement details. Unlike manual AppArmor or SELinux profiles, which are complex and hard to scale, KubeArmor’s declarative approach and default deny policies simplify securing containers, preventing access to dangerous assets like /proc mounts.
Practical Implementation and Community Engagement
Saeid provided practical examples, such as configuring a pod to drop all capabilities except those needed, applying a Seccomp-BPF profile to restrict system calls, and using KubeArmor to enforce file access policies. He highlighted KubeArmor’s integration with tools like OPA Gatekeeper to block unauthorized commands (e.g., kubectl exec) when BPF-LSM is unavailable. For further learning, Saeid offered a 50% discount on CubeDemi.io’s container security workshop, encouraging KCDUK2024 attendees to deepen their Kubernetes security expertise.
Links:
Hashtags: #Kubernetes #RuntimeSecurity #KubeArmor #CNCF #LinuxSecurity #SaeidBostandoust #KCDUK2024
[DefCon32] On Your Ocean’s 11 Team, I’m the AI Guy (Technically Girl)
Blending the allure of high-stakes gambles with cutting-edge threats, Harriet Farlow, an AI security specialist, navigates the intersection of adversarial machine learning and casino operations. Targeting Canberra Casino, she exposes frailties in emerging AI integrations for surveillance and monitoring. Her exploits disrupt facial recognition, evade detection, and manipulate gameplay, illustrating broader perils in sectors reliant on such systems.
Harriet’s background spans physics, data science, and government intelligence, culminating in founding ML Security Labs. Her focus: deceiving AI to reveal weaknesses, akin to cyber intrusions but tailored to models’ statistical natures.
Casinos, epitomizing surveillance-heavy environments, increasingly adopt AI for identifying threats and optimizing play. Canberra, though modest, mirrors global trends where a few providers dominate, ripe for widespread impacts.
Adversarial attacks perturb inputs subtly, fooling models without human notice. Harriet employs techniques like fast gradient sign methods, crafting perturbations that reduce classification confidence.
Targeting Facial Recognition
Facial systems, crucial for barring excluded patrons, succumb to perturbations. Harriet generates adversarial examples via libraries like Foolbox, adding noise that misclassifies faces.
Tests show 40.4% success in evading matches, but practical adaptations ensure consistent bypasses. This equates to denial-of-service equivalents in AI, disrupting reliability.
Broader implications span medical diagnostics to autonomous navigation, where minor alterations yield catastrophic errors.
Evading Surveillance and Gameplay Monitoring
Surveillance AI detects anomalies; Harriet’s perturbations obscure actions, mimicking wild exploits.
Gameplay AI monitors for advantages; adversarial inputs confuse chip recognition or behavior analysis, enabling undetected strategies.
Interviews with casino personnel reveal heavy reliance on human oversight, despite AI promises. Only 8% of surveyed organizations secure AI effectively, versus 94% using it.
Lessons from the Inflection Point
Casinos transition to AI amid regulatory voids, amplifying risks. Harriet advocates integrating cyber lessons: robust testing beyond accuracy, incorporating security metrics.
Her findings stress governance: people and processes remain vital, yet overlooked. As societies embrace AI surveillance, vulnerabilities threaten equity and safety.
Harriet’s work urges cross-disciplinary approaches, blending cyber expertise with AI defenses to mitigate emerging dangers.
Links:
EN_DEFCON32MainStageTalks_011_013.md
[KotlinConf2024] Hacking Sony Cameras with Kotlin
At KotlinConf2024, Rahul Ravikumar, a Google software engineer, shared his adventure reverse-engineering Sony’s Bluetooth Low Energy (BLE) protocol to build a Kotlin Multiplatform (KMP) remote camera app. Frustrated by Sony’s bloated apps—Imaging Edge Mobile (1.9 stars) and Creators’ App (3.3 stars)—Rahul crafted a lean solution for his Sony Alpha a7r Mark 5, focusing on remote control. Using Compose Multiplatform for desktop and mobile, and Sony’s C SDK via cinterop, he demonstrated how KMP enables cross-platform innovation. His live demo, clicking a photo with a single button, thrilled the audience and showcased BLE’s potential for fun and profit.
Reverse-Engineering Sony’s BLE Protocol
Rahul’s journey began with Sony’s underwhelming app ecosystem, prompting him to reverse-engineer the camera’s undocumented BLE protocol. BLE’s Generic Access Profile (GAP) handles device discovery, with the camera (peripheral) advertising its presence and the phone (central) connecting. The Generic Attribute Profile (GATT) manages commands, using 16-bit UUIDs for services like Sony’s remote control (FF01 for commands, FF02 for notifications). Unable to use Android’s HCI Snoop logs due to Sony’s Wi-Fi Direct reliance, Rahul employed a USB BLE sniffer and Wireshark to capture GATT traffic. He identified Sony’s company ID (0x02D01) and camera marker (0x03000) in advertising packets. Key operations—reset (0x0106), focus (0x0107), and capture (0x0109)—form a state machine, with notifications (e.g., 0x023F) confirming actions. This meticulous process, decoding hexadecimal payloads, enabled Rahul to control the camera programmatically.
Building a KMP Remote Camera App
With the protocol cracked, Rahul built a KMP app using Compose Multiplatform, targeting Android and desktop. The app’s BLE scanner filters for Sony’s manufacturer data (0x03000), ignoring irrelevant metadata like model codes. Connection logic uses Kotlin Flows to monitor peripheral state, ensuring seamless reconnections. Capturing a photo involves sending reset and focus commands to FF01, awaiting focus confirmation on FF02, then triggering capture and shutter reset. For advanced features, Rahul integrated Sony’s C SDK via cinterop, navigating its complexities to access functions like interval shooting. His live demo, despite an initially powered-off camera, succeeded when the camera advertised, and a button click took a photo, earning audience cheers. The app’s simplicity contrasts Sony’s feature-heavy apps, proving KMP’s power for cross-platform development. Rahul’s GitHub repository offers the code, inviting developers to explore BLE and KMP for their own projects.
Links:
Hashtags: #KotlinMultiplatform #BluetoothLE
[DevoxxBE2024] Project Panama in Action: Building a File System by David Vlijmincx
At Devoxx Belgium 2024, David Vlijmincx delivered an engaging session on Project Panama, demonstrating its power by building a custom file system in Java. This practical, hands-on talk showcased how Project Panama simplifies integrating C libraries into Java applications, replacing the cumbersome JNI with a more developer-friendly approach. By leveraging Fuse, virtual threads, and Panama’s memory management capabilities, David walked attendees through creating a functional file system, highlighting real-world applications and performance benefits. His talk emphasized the ease of integrating C libraries and the potential to build high-performance, innovative solutions.
Why Project Panama Matters
David began by addressing the challenges of JNI, which many developers find frustrating due to its complexity. Project Panama, part of OpenJDK, offers a modern alternative for interoperating with native C libraries. With a vast ecosystem of specialized C libraries—such as io_uring for asynchronous file operations or libraries for AI and keyboard communication—Panama enables Java developers to access functionality unavailable in pure Java. David demonstrated this by comparing file reading performance: using io_uring with Panama, he read files faster than Java’s standard APIs (e.g., BufferedReader or Channels) in just two nights of work, showcasing Panama’s potential for performance-critical applications.
Building a File System with Fuse
The core of David’s demo was integrating the Fuse (Filesystem in Userspace) library to create a custom file system. Fuse acts as a middle layer, intercepting commands like ls from the terminal and passing them to a Java application via Panama. David explained how Fuse provides a C struct that Java developers can populate with pointers to Java methods, enabling seamless communication between C and Java. This struct, filled with method pointers, is mounted to a directory (e.g., ~/test), allowing the Java application to handle file system operations transparently to the user, who sees only the terminal output.
Memory Management with Arenas
A key component of Panama is its memory management via arenas, which David used to allocate memory for passing strings to Fuse. He demonstrated using Arena.ofShared(), which allows memory sharing across threads and explicit lifetime control via try-with-resources. Other arena types, like Arena.ofConfined() (single-threaded) or Arena.global() (unbounded lifetime), were mentioned for context. David allocated a memory segment to store pointers to a string array (e.g., ["-f", "-d", "~/test"]) and used Arena.allocateFrom() to create C-compatible strings. This ensured safe memory handling when interacting with Fuse, preventing leaks and simplifying resource management.
Downcalls and Upcalls: Bridging Java and C
David detailed the process of making downcalls (Java to C) and upcalls (C to Java). For downcalls, he created a function descriptor mirroring the C method’s signature (e.g., fuse_main_real, returning an int and taking parameters like string arrays and structs). Using Linker.nativeLinker(), he generated a platform-specific linker to invoke the C method. For upcalls, he recreated Fuse’s struct in Java using MemoryLayout.structLayout, populating it with pointers to Java methods like getattr. Tools like JExtract simplified this by generating bindings automatically, reducing boilerplate code. David showed how JExtract creates Java classes from C headers, though it requires an additional abstraction layer for user-friendly APIs.
Implementing File System Operations
David implemented two file system operations: reading files and creating directories. For reading, he extracted the file path from a memory segment using MemorySegment.getString(), checked if it was a valid file, and copied file contents into a buffer with MemorySegment.reinterpret() to handle size constraints. For directory creation, he added paths to a map, demonstrating simplicity. Running the application mounted the file system to ~/test, where commands like mkdir and echo worked seamlessly, with Fuse calling Java methods via upcalls. David unmounted the file system, showing its clean integration. Performance tips included reusing method handles and memory segments to avoid overhead, emphasizing careful memory management.
Links:
[DefCon32] The Edges of Surveillance System and Its Supply Chain
As artificial intelligence advances image processing, surveillance technologies proliferate, raising privacy concerns amid rising hacks and illicit collections. Chanin Kim and Myounghun Pak, offensive researchers, probe network video recorders (NVRs), pivotal yet understudied components. Their analysis targets market leaders Hikvision, Dahua, Uniview, and Synology’s Surveillance Station, uncovering vulnerabilities exploitable worldwide and amplifying impacts through OEM supply chains.
Chanin and Myounghun’s journey, spanning four months and yielding a substantial bounty, begins with firmware extraction challenges. They bypass U-Boot mitigations via UART, JTAG, and eMMC methods, extracting systems for scrutiny. HTTP communications and binary analyses reveal flaws, with OEM validations extending reach.
The surge in surveillance, valued at $4.1 billion globally, stems from pandemic demands for monitoring in retail, urban planning, and access control. NVRs store and manage footage from CCTVs and IP cameras, often internet-accessible, making them prime targets.
Limited prior research on NVRs, contrasted with extensive CCTV studies, motivates their focus. Shodan scans reveal over 30,000 exposed devices, echoing past exploits like Mirai botnets leveraging Hikvision RCEs.
Firmware Extraction Methodologies
Extraction proves arduous. Hikvision and Dahua support limited access but restrict root, necessitating alternatives. UART on Hikvision yields shells post-soldering, while Dahua requires JTAG for U-Boot interruption.
Uniview demands eMMC dumping via adapters, bypassing encryptions. Synology’s package simplifies via SSH.
Bypasses include patching U-Boot for boot delays and exploiting environment variables. These enable comprehensive analyses.
Discovered Vulnerabilities and Exploits
Analyses uncover stack overflows, command injections, and credential leaks. Hikvision’s overflows in HTTP handlers allow RCE, while Dahua’s injections in recovery endpoints enable reboots or shells.
Uniview exposes hardcoded credentials and overflows in authentication. Synology suffers overflows in login processes.
Chaining yields scenarios: credential stuffing grants admin, injections open reverses, scanning locates Windows hosts for plugin RCEs, mirroring wild attacks.
Demonstrations automate compromises, shutting devices or commandeering networks.
Supply Chain Ramifications
OEM dynamics amplify risks. Hikvision and Dahua supply rebranded products to Lorex, Luma, Amcrest, and EZVIZ. Similar logics and firmwares mean vulnerabilities propagate.
Purchased devices confirm: four of six Hikvision flaws affect Annke; four of seven Dahua issues impact Swann and Reolink.
CVEs assigned via Korean CNA increase recognized flaws by 61%. Shodan estimates 130,000 Hikvision and 100,000 Dahua units vulnerable.
Recommendations for Defense
Offensive researchers should prioritize HTTP and private protocols beyond studied surfaces. Users avoid external exposures, leveraging vendor DDNS for secure access.
Their findings underscore surveillance’s dual-edged nature, urging robust protections amid expanding deployments.
Links:
[DefCon32] Where’s the Money? Defeating ATM Disk Encryption
In an era where automated teller machines safeguard substantial sums, vulnerabilities in their protective mechanisms pose significant threats. Matt Burch, an independent security researcher specializing in IoT and hardware, unveils critical flaws in Diebold Nixdorf’s Vynamic Security Suite (VSS), the dominant solution in the sector. His investigation, conducted alongside a colleague, exposes six zero-day issues enabling full system compromise within minutes, highlighting systemic risks across financial, gaming, and retail domains.
Matt’s journey stems from a fascination with the surge in ATM crimes, up over 600% recently. Targeting enterprise-grade units holding up to $400,000, he dissects VSS’s full disk encryption, revealing offline code injection and decryption paths. Diebold Nixdorf, one of three primary North American manufacturers with global reach, deploys VSS widely, including in Las Vegas casinos.
ATM architecture divides into a fortified vault for currency and a less secure “top hat” housing computing elements. The vault features robust steel, multi-factor locks, and tamper sensors, while the upper section uses thin metal and vulnerable locks, facilitating entry via simple tools.
VSS integrates endpoint protection, whitelisting, and encryption, yet Matt identifies gaps in its pre-boot authentication (PBA). This layered integrity check, spanning phases, fails to prevent unauthorized access.
Pre-Boot Authentication Vulnerabilities
VSS’s PBA employs a custom Linux-based “Super Sheep” OS for initial validation. Phase one mounts the Windows partition read-only, computing SHA-256 sums for critical files. Successful checks lead to phase two, decrypting and booting Windows.
Matt exploits unencrypted Linux elements, mounting drives offline to inject code. CVE-2023-33204 allows execution via root’s .profile, bypassing sums by targeting non-checked files. Demonstrations show callback shells post-reboot.
Service releases introduce patches, yet recursive flaws emerge. CVE-2023-33205 leverages credential stuffing on weak defaults, enabling admin escalation and command injection.
Recursive Flaws and Persistence
Patches inadvertently create new vectors. Service Release 15 removes directories but overlooks symlinks, leading to CVE-2023-33206. Attackers create traversal links, injecting payloads into root’s directory for execution.
Further updates validate symlinks, yet CVE-2023-40261 strips execute permissions from integrity modules, disabling VSS entirely. This permits arbitrary unsigned binaries, affecting all versions.
Impacts span VSS iterations, with vulnerabilities identified from April to July 2023. Diebold’s responses include end-of-life for older variants and redesigns.
Mitigation and Broader Implications
Defending requires patching to latest releases, enabling security checks to withhold TPM keys, and monitoring physical access. Disabling unused ports and securing drives reduce risks. Alternatives like BitLocker offer robust encryption.
Matt’s findings influence Diebold’s Version 4.4, introducing full Super Sheep encryption. Yet, persistent unencrypted elements suggest ongoing challenges, with likely undiscovered exploits.
This research underscores the need for comprehensive security in high-value systems, where physical and digital safeguards must align.
Links:
[DefCon32] Welcome to DEF CON 32
Amid the vibrant energy of a gathering that has evolved over decades, Jeff Moss, founder of DEF CON, extends a heartfelt invitation to participants, emphasizing the essence of community and shared discovery. Drawing from his experiences since initiating the event 32 years ago, Jeff reflects on its growth from a modest assembly to a sprawling nexus of innovation. His remarks serve as an orientation, guiding attendees through the philosophy that underpins the conference, while encouraging them to forge their own paths in a landscape brimming with possibilities.
Jeff underscores the principle that the event’s value lies in individual contributions, acknowledging the impossibility of experiencing every facet. Early iterations allowed him to witness all activities, yet as attendance swelled, he embraced the reality of missing moments, transforming it into motivation for expanding offerings. This mindset fosters an environment where participants can prioritize personal interests, whether technical pursuits or interpersonal connections.
The structure facilitates meaningful interactions by segmenting the crowd into affinity clusters, such as those focused on automotive exploits or physical barriers. Such divisions enable deeper engagements, turning vast numbers into intimate collaborations. Jeff highlights the encouragement of inquiry, recognizing the specialization driven by technological complexity, which renders no single expert all-knowing.
Origins and Inclusivity
Tracing the roots, Jeff recounts how exclusion from invite-only gatherings inspired an open-door policy, rejecting seasonal naming to avoid constraints. This decision marked a pivotal divergence, prioritizing accessibility over restriction. Growth necessitated strategies to manage scale without diluting intimacy, leading to diverse tracks and villages that cater to niche passions.
The ethos promotes authenticity, allowing attendees to express themselves freely while respecting boundaries. Jeff shares anecdotes illustrating the blend of serendipity and intent that defines encounters, urging newcomers to engage without hesitation.
Global Perspectives and Accountability
Jeff broadens the view to international contexts, noting how varying educational systems influence entry into the field. In some regions, extended periods of exploration nurture creativity, contrasting with structured paths elsewhere. He celebrates the cultural embrace of setbacks as stepping stones, aligning with narratives of resilience.
To ensure trust, a code of conduct governs interactions, applicable universally. Enforcement through transparency reports holds organizers accountable, publicly detailing infractions to validate community standards. This mechanism reinforces integrity, even when confronting uncomfortable truths.
Jeff transitions to highlighting speakers like General Nakasone, whose insights demystify complex topics. Originating from efforts to verify online claims, these sessions connect attendees with authoritative voices, bridging gaps in understanding.
In closing, Jeff invites immersion, promising encounters that enrich beyond expectations.
Links:
[OxidizeConf2024] Certifying OxidOS for ISO26262 ASIL-D
Understanding OxidOS and ISO 26262
Georgiana Vlădulescu and Alexandru Radovici, representing OxidOS, shared their journey of certifying an operating system written entirely in Rust for ISO 26262 ASIL-D compliance at OxidizeConf2024. OxidOS, a professional derivative of the open-source Tock operating system, is designed for safety-critical applications in small electronic control units (ECUs). Their presentation provided a deep dive into the challenges and innovative solutions encountered while aligning OxidOS with the stringent requirements of ISO 26262, a standard that defines functional safety for automotive components.
ISO 26262 categorizes safety requirements into Automotive Safety Integrity Levels (ASILs), ranging from QM (Quality Management) to ASIL-D, based on factors like severity, exposure, and controllability. Georgiana and Alexandru explained that OxidOS aims to achieve ASIL-D certification out of context, meaning it can be used across various automotive applications, from infotainment to critical systems like airbags. This ambitious goal requires meticulous documentation and process adherence, which their team has approached with a focus on automation and open-source tools.
Challenges in Certification
The certification process for OxidOS revealed significant hurdles, primarily due to the automotive industry’s reliance on legacy tools and processes. Georgiana noted that traditional expectations, such as exchanging Excel documents, felt archaic to a team accustomed to modern software development practices. These manual processes were not only time-consuming but also prone to errors, as they lacked integration with development workflows. Additionally, the absence of Rust-specific tools in the certification domain posed a challenge, as most existing solutions cater to C-based systems.
Another significant obstacle was maintaining team motivation, as developers often prefer coding over documentation. To address this, the OxidOS team transformed the certification process into a challenge, encouraging developers to innovate by creating custom tools and workflows. This approach empowered the team, fostering a sense of autonomy and reducing burnout by allowing developers to switch between projects during downtime.
Innovative Solutions with Rust
Rust’s ecosystem proved instrumental in overcoming these challenges. The team adopted Sphinx Needs, an extension of the Sphinx documentation tool, to implement a “document as code” philosophy. This approach allowed them to version control requirements using GitHub, enabling asynchronous collaboration and historical tracking. By integrating tools like Cargo Doc, they automated the generation of certification documents from Rust code, significantly reducing manual effort.
Alexandru emphasized the importance of minimizing transitive dependencies in OxidOS, relying solely on libcore to maintain control over the codebase. This policy, coupled with Rust’s safety guarantees, ensured that the operating system remained robust and compliant. The team also developed custom scripts to link code changes to documentation, ensuring that updates to Tock’s upstream repository could be seamlessly integrated into the certification process without requiring extensive manual rework.
Future Directions and Industry Impact
Looking ahead, Georgiana and Alexandru outlined plans to enhance automation further, particularly in tracking code changes and their impact on certification requirements. By creating a link between code and documentation, they aim to streamline the process of updating requirements as Tock evolves. This approach addresses a critical industry challenge: certifying open-source software that lacks the controlled development environment of proprietary systems.
The presentation also highlighted broader industry implications. Alexandru noted that many automotive companies face similar tool-related challenges, often duplicating efforts due to non-disclosure agreements. By open-sourcing their tools in the future, the OxidOS team hopes to foster collaboration and reduce redundancy, potentially transforming how open-source software is certified in the automotive sector.
Links:
[DevoxxUK2024] Is It (F)ake?! Image Classification with TensorFlow.js by Carly Richmond
Carly Richmond, a Principal Developer Advocate at Elastic, captivated the DevoxxUK2024 audience with her engaging exploration of image classification using TensorFlow.js. Inspired by her love for the Netflix show Is It Cake?, Carly embarked on a project to build a model distinguishing cakes disguised as everyday objects from their non-cake counterparts. Despite her self-professed lack of machine learning expertise, Carly’s journey through data gathering, pre-trained models, custom model development, and transfer learning offers a relatable and insightful narrative for developers venturing into AI-driven JavaScript applications.
Gathering and Preparing Data
Carly’s project begins with the critical task of data collection, a foundational step in machine learning. To source images of cakes resembling other objects, she leverages Playwright, a JavaScript-based automation framework, to scrape images from bakers’ websites and Instagram galleries. For non-cake images, Carly utilizes the Unsplash API, which provides royalty-free photos with a rate-limited free tier. She queries categories like reptiles, candles, and shoes to align with the deceptive cakes from the show. However, Carly acknowledges limitations, such as inadvertently including biscuits or company logos in the dataset, highlighting the challenges of ensuring data purity with a modest set of 367 cake and 174 non-cake images.
Exploring Pre-Trained Models
To avoid building a model from scratch, Carly initially experiments with TensorFlow.js’s pre-trained models, Coco SSD and MobileNet. Coco SSD, trained on the Common Objects in Context (COCO) dataset, excels in object detection, identifying bounding boxes and classifying objects like cakes with reasonable accuracy. MobileNet, designed for lightweight classification, struggles with Carly’s dataset, often misclassifying cakes as cups or ice cream due to visual similarities like frosting. CORS issues further complicate browser-based MobileNet deployment, prompting Carly to shift to a Node.js backend, where she converts images into tensors for processing. These experiences underscore the trade-offs between model complexity and practical deployment.
Building and Refining a Custom Model
Undeterred by initial setbacks, Carly ventures into crafting a custom convolutional neural network (CNN) using TensorFlow.js. She outlines the CNN’s structure, which includes convolution layers to extract features, pooling layers to reduce dimensionality, and a softmax activation for binary classification (cake vs. not cake). Despite her efforts, the model’s accuracy languishes at 48%, plagued by issues like tensor shape mismatches and premature tensor disposal. Carly candidly admits to errors, such as mislabeling cakes as non-cakes, illustrating the steep learning curve for non-experts. This section of her talk resonates with developers, emphasizing perseverance and the iterative nature of machine learning.
Leveraging Transfer Learning
Recognizing the limitations of her dataset and custom model, Carly pivots to transfer learning, using MobileNet’s feature vectors as a foundation. By adding a custom classification head with ReLU and softmax layers, she achieves a significant improvement, with accuracy reaching 100% by the third epoch and correctly classifying 319 cakes. While not perfect, this approach outperforms her custom model, demonstrating the power of leveraging pre-trained models for specialized tasks. Carly’s comparison of human performance—90% accuracy by the DevoxxUK audience versus her model’s results—adds a playful yet insightful dimension, highlighting the gap between human intuition and machine precision.