Archive for the ‘General’ Category
[NDCOslo2024] Develop AI Agents with Semantic Kernel – Jakob Ehn
In the vanguard of generative paradigms, where language models orchestrate outcomes autonomously, Jakob Ehn, a seasoned solutions architect at Atea, illuminates Semantic Kernel’s symphony. A Microsoft MVP with a penchant for Azure and AI, Jakob unveils this open-source orchestrator—born in Microsoft’s copilot crucible—as a conduit for infusing intelligence into imperatives. His tutorial, threaded with live enactments, empowers .NET denizens to conjure agents that deliberate, delegate, and deliver, transcending prompts to proactive pursuits.
Jakob ignites with LLMs’ leap: from GPT’s generative gloss to agentic autonomy, where kernels coordinate calls, chaining capabilities contextually. Semantic Kernel, polyglot and potent, abstracts APIs into plugins, planners plot paths, memories mint moments—crafting copilots that converse, compute, and conclude sans ceaseless coding.
Foundations of Fusion: Kernel Configurations and Plugins
Semantic Kernel’s scaffold: a kernel encapsulating services—embeddings, chat completions—interfacing IConnectors for OpenAI or Azure. Jakob blueprints: var builder = Kernel.CreateBuilder(); builder.AddAzureOpenAIChatCompletion(…), yielding a kernel primed for prompts.
Plugins propel: native .NET assemblies or OpenAPI schemas, exposing functions as facets. Jakob instantiates: a weather plugin with GetForecastAsync, invoked via kernel.InvokeAsync. Semantic functions, YAML-yielded, infuse prompts with placeholders: “Current weather in {location}?”—blending blueprints with brains.
Memories memorialize: volatile or vectorized, persisting precedents for pertinence. Jakob’s vector store—Chroma—embeds episodes, retrieving resonances via similarity searches, enriching responses with recency.
Planners and Agents: Autonomous Assemblies
Planners pioneer paths: Sequential, Stepwise, or Action—dissecting directives into doables. Jakob’s Action planner: given “Book a flight,” it inventories functions—search, reserve—sequencing sans stalemates. Handles, he highlights, hallucination hurdles, grounding goals in grapples.
Agents amplify: collaborative cohorts, each expert in ethos—developer drafts, organizer oversees. Jakob’s ensemble: a proposal pipeline, where agents iterate abstracts, incorporating critiques conversationally. Semantic functions script sagas: “Revise to {limit} words, include {term}”—looping lucidly.
His demo: a conference copilot, soliciting sessions, surfacing summaries—agents alchemizing APIs into anecdotes.
Horizons and Hurdles: Refinements and Realities
Jakob tempers triumph: kernels kindle complexity—prompt precision, plugin parity. Yet, prospects gleam: multi-modal melds, enterprise embeddings. His behest: experiment, embracing Semantic Kernel’s scaffold to sculpt sentient solutions.
Links:
[DevoxxBE2024] How JavaScript Happened: A Short History of Programming Languages
In an engaging session at Devoxx Belgium 2024, Mark Rendle traced the evolution of programming languages leading to JavaScript’s creation in 1995. Titled “How JavaScript Happened: A Short History of Programming Languages,” the talk blended humor and history, from Ada Lovelace’s 1840s program to JavaScript’s rapid development for Netscape Navigator 2.0. Despite a brief battery scare during the presentation, Rendle’s storytelling and FizzBuzz examples across languages captivated the audience, offering insights into language design and JavaScript’s eclectic origins.
The Dawn of Programming
Rendle began in the 1830s with Ada Lovelace, who wrote the first program for Charles Babbage’s unbuilt Analytical Engine, introducing programming notation 120 years before computers existed. The 1940s saw programmable machines like Colossus, built to crack German ciphers, and ENIAC, programmed by women who deciphered its operation without manuals. These early systems, configured via patch cables, laid the groundwork for modern computing, though programming remained labor-intensive.
The Rise of High-Level Languages
The 1950s marked a shift with Fortran, created by John Backus to simplify machine code translation for IBM’s 701 mainframe. Fortran introduced if statements, the asterisk for multiplication (due to punch card limitations), and the iterator variable i, still ubiquitous today. ALGOL 58 and 60 followed, bringing block structures, if-then-else, and BNF grammar, formalized by Backus. Lisp, developed by John McCarthy, introduced first-class functions, the heap, and early garbage collection, while Simula pioneered object-oriented programming with classes and inheritance.
From APL to C and Beyond
Rendle highlighted APL’s concise syntax, enabled by its unique keyboard and dynamic typing, influencing JavaScript’s flexibility. The 1960s and 70s saw BCPL, B, and C, with C introducing curly braces, truthiness, and the iconic “hello world” program. Smalltalk added reflection, virtual machines, and the console, while ML introduced functional programming concepts like arrow functions. Scheme, a simplified Lisp, directly influenced JavaScript’s initial design as a browser scripting language, shaped to compete with Java applets.
JavaScript’s Hasty Creation
In 1995, Brendan Eich created JavaScript in ten days for Netscape Navigator 2.0, initially as a Scheme-like language with a DOM interface. To counter Java applets, it adopted a C-like syntax and prototypal inheritance (inspired by Self), as classical inheritance wasn’t feasible in Scheme. Rendle humorously speculated on advising Eich to add static typing and classical inheritance, noting JavaScript’s roots in Fortran, ALGOL, Lisp, and others. Despite its rushed origins, JavaScript inherited a rich legacy, from Fortran’s syntax to Smalltalk’s object model.
The Legacy and Future of JavaScript
Rendle concluded by reflecting on JavaScript’s dominance, driven by its browser integration, and its ongoing evolution, with features like async/await (from C#) and proposed gradual typing. He dismissed languages like COBOL and Pascal for lacking influential contributions, crediting BASIC for inspiring programmers despite adding little to language design. JavaScript, a synthesis of 70 years of innovation, continues to evolve, shaped by decisions from 1955 to today, proving no language is immune to historical influence.
Links:
Hashtags: #JavaScript #ProgrammingHistory #MarkRendle #DevoxxBE2024
[DefCon32] QuickShell: Sharing Is Caring About RCE Attack Chain on QuickShare
In the interconnected world of file sharing, Google’s QuickShare, bridging Android and Windows, presents a deceptively inviting attack surface. Or Yair and Shmuel Cohen, researchers at SafeBreach, uncover ten vulnerabilities, culminating in QuickShell, a remote code execution (RCE) chain exploiting five flaws. Their journey, sparked by QuickShare’s Windows expansion, reveals logical weaknesses that enable file writes, traffic redirection, and system crashes, culminating in a sophisticated RCE.
Or, a vulnerability research lead, and Shmuel, formerly of Check Point, dissect QuickShare’s Protobuf-based protocol. Initial fuzzing yields crashes but no exploits, prompting a shift to logical vulnerabilities. Their findings, responsibly disclosed to Google, lead to patches and two CVEs, addressing persistent Wi-Fi connections and file approval bypasses.
QuickShare’s design, facilitating seamless device communication, lacks robust validation, allowing attackers to manipulate file transfers and network connections. The RCE chain combines these flaws, achieving unauthorized code execution on Windows systems.
Protocol Analysis and Fuzzing
Or and Shmuel begin with QuickShare’s protocol, using hooks to decode Protobuf messages. Their custom fuzzer targets the Windows app, identifying crashes but lacking exploitable memory corruptions. This pivot to logical flaws uncovers issues like unauthenticated file writes and path traversals, exposing user directories.
Tools built for device communication enable precise vulnerability discovery, revealing weaknesses in QuickShare’s trust model.
Vulnerability Discoveries
The researchers identify ten issues: file write bypasses, denial-of-service (DoS) crashes, and Wi-Fi redirection via crafted access points. Notable vulnerabilities include forcing file approvals without user consent and redirecting traffic to malicious networks.
A novel HTTPS MITM technique amplifies the attack, intercepting communications to escalate privileges. These flaws, present in both Android and Windows, highlight systemic design oversights.
Crafting the RCE Chain
QuickShell chains five vulnerabilities: a DoS to destabilize QuickShare, a file write to plant malicious payloads, a path traversal to target system directories, a Wi-Fi redirection to control connectivity, and a final exploit triggering RCE. This unconventional chain leverages seemingly minor bugs, transforming them into a potent attack.
Demonstrations show persistent connections and code execution, underscoring the chain’s real-world impact.
Takeaways for Developers and Defenders
Or and Shmuel emphasize that minor bugs, often dismissed, can cascade into severe threats. The DoS flaw, critical to their chain, exemplifies how non-security issues enable attacks. They advocate holistic security assessments, beyond memory corruptions, to evaluate logical behaviors.
Google’s responsive fixes, completed by January 2025, validate the research’s impact. The team’s open-source tools invite further exploration, urging developers to prioritize robust validation in file-sharing systems.
Links:
[KotlinConf2024] Why We Can’t Have Nice Things in Kotlin: Library Challenges
Vsevolod Tolstopyatov, a Kotlin libraries team member at JetBrains, entertained KotlinConf2024 with a witty exploration of why Kotlin’s standard libraries evolve slowly. From Turkish locale bugs breaking compilers to time zone quirks skipping entire days, Vsevolod revealed the hidden complexities of API design. Through anecdotes like Twitter polls on naming and a friend’s missed alarm, he highlighted how human language, historical quirks, and real-world constraints challenge library development, urging developers to appreciate the slow, thoughtful process behind robust APIs.
Human Language: The Turkish I Debacle
Kotlin’s string functions, like toUpperCase and toLowerCase, rely on system locales, leading to unexpected issues. Vsevolod recounted a bug where code compiled in Turkey failed due to the Turkish dotted and dotless I. Uppercasing an English lowercase i in Turkish yields a dotted uppercase İ, breaking method names like intArray. This forced the deprecation of locale-dependent functions, replaced with uppercase and lowercase using a root locale. The lesson: human languages’ diversity demands careful API design to avoid breaking code across cultures.
Naming Woes: The Capitalize Conundrum
Naming APIs is deceptively hard. Vsevolod shared how the deprecated capitalize function lacked a clear replacement because “capitalize” means different things to different developers. Twitter polls revealed no consensus, and other languages like Python and Ruby use capitalize inconsistently. With all good names taken or ambiguous, Kotlin’s team struggles to craft intuitive APIs without confusing users. This naming challenge slows library updates, as new functions risk misinterpretation or overlap with existing conventions.
Time Zones: Samoa’s Missing Day
Time zone complexities plague datetime APIs. Vsevolod described a case in Samoa, which skipped December 30, 2011, to align with Australia’s time zone, jumping from UTC-10 to UTC+14. A simple date addition (e.g., December 29 + 2 days) yields inconsistent hours due to the missing day. Adding time zone parameters fixes this but complicates APIs, especially on devices crossing time zone boundaries, like aircraft. These real-world quirks force Kotlin to balance simplicity with correctness, often at the cost of elegance.
Daylight Saving Time: Bug Hunting’s Legacy
Daylight saving time (DST) adds further headaches. Vsevolod traced DST to George Hudson, a 19th-century entomologist who proposed time shifts to hunt bugs in New Zealand. During DST transitions, times like 2:30 AM may not exist (spring forward) or occur twice (fall back), breaking datetime calculations. An anecdote about Vsevolod’s friend Vlad, whose smart home set a 9:30 AM alarm instead of 8:30 AM during a DST shift, underscored the need for explicit API parameters to handle ambiguous times, sacrificing simplicity.
API Design: Slow and Steady Wins
Vsevolod concluded that Kotlin’s libraries evolve slowly to avoid mistakes. Rushed APIs lead to bugs, like Vlad’s missed alarm, requiring years of maintenance. Thoughtful design, considering edge cases like locales, time zones, and DST, ensures reliability but delays releases. Twitter polls and community feedback help, but the real world’s complexity—political, historical, and cultural—demands patience. By prioritizing robustness, Kotlin’s team crafts APIs that won’t “get anyone fired,” even if it means fewer “nice things” per release.
Links:
[DevoxxBE2024] Thinking Like an Architect
In a reflective talk at Devoxx Belgium 2024, Gregor Hohpe, a veteran architect, shared insights from two decades of experience in “Thinking Like an Architect.” Hohpe debunked the myth of architects as all-knowing decision-makers, instead portraying them as “IQ boosters” who enhance team decision-making through models, metaphors, and multi-level communication. Despite a minor issue with a clicker during the presentation, his engaging delivery and relatable examples, like the “architect elevator,” offered practical strategies for navigating complex organizational and technical landscapes.
Connecting Levels with the Architect Elevator
Hohpe introduced the “architect elevator,” a metaphor for architects’ role in bridging organizational layers—from developers to executives. He argued that the most valuable architects connect business strategy to technical implementation, translating complex trade-offs into terms executives understand without oversimplifying. For example, automation and frequent releases (developer priorities) enable security and cost-efficiency (executive concerns). This connection counters the isolation caused by layered organizations, where management may assume all is well due to buzzwords like Kubernetes, while developers operate with unchecked freedom.
Seeing More Dimensions in Decision-Making
Architects expand solution spaces by revealing additional dimensions, Hohpe explained. Using a sketch of a cylinder mistaken as a circle or rectangle, he showed how architects resolve debates—like speed versus quality—by introducing options like automated testing. At AWS, Hohpe tackled vendor lock-in by framing it as a two-dimensional trade-off: switching costs versus benefits. This approach, inspired by Adrian Cockcroft’s analogy of marriage as “accepted lock-in,” fosters rational discussions, avoiding binary thinking and helping teams find balanced solutions.
Selling Options to Defer Decisions
Hohpe likened architects to options traders, deferring decisions to reduce uncertainty. For instance, standard APIs allow language flexibility, sacrificing some protocol options to gain adaptability. In a financial firm, he explained this to executives using options trading, noting that options’ value rises with volatility—a concept they instantly grasped via the Black-Scholes formula. This metaphor underscores architecture’s increasing relevance in uncertain environments, aligning it with agile methodologies, which thrive under similar conditions. However, options come at the cost of complexity, a trade-off architects must weigh.
Zooming In and Out for System-Wide Perspective
To tackle complexity, architects must zoom in and out, balancing local and global optima. Hohpe illustrated this with two systems using identical components but different connections, yielding opposite characteristics (e.g., latency versus resilience). Local optimization, like perfecting a single component, often fails to ensure system-wide success, as seen in operations where “all lights are green, but nothing works.” By viewing systems holistically, architects ensure decisions align with broader goals, avoiding pitfalls like excessive layering that propagates changes unnecessarily.
Using Models to Navigate Uncertainty
Hohpe emphasized models as architects’ best tools for simplifying complexity. Comparing geocentric and heliocentric solar system models, he showed how the right model makes decisions obvious, even if imperfect. Models vary by purpose—topographical maps for hiking, population density for logistics—requiring architects to choose based on the question at hand. In uncertain environments, models shine by forcing assumptions, enabling scenario-based planning (e.g., low, medium, high user loads). Hohpe urged architects to avoid absolutes, embracing shades of gray to find optimal trade-offs.
Links:
- Devoxx Belgium 2024
- The Architect Elevator
- Cloud Strategy by Gregor Hohpe
- Adrian Cockcroft’s Blog
- Black-Scholes Formula
Hashtags: #SoftwareArchitecture #ArchitectMindset #AgileArchitecture #DevoxxBE2024
[DevoxxBE2024] Project Leyden: Improving Java’s Startup Time by Per Minborg, Sébastien Deleuze
Per Minborg and Sébastien Deleuze delivered an insightful joint presentation at Devoxx Belgium 2024, unveiling the transformative potential of Project Leyden to enhance Java application startup time, warmup, and footprint. Per, from Oracle’s Java Core Library team, and Sébastien, a Spring Framework core committer at Broadcom, explored how Leyden shifts computation across time to optimize performance. Despite minor demo hiccups, such as Wi-Fi-related delays, their talk combined technical depth with practical demonstrations, showcasing how Spring Boot 3.3 leverages Leyden’s advancements, cutting startup times significantly and paving the way for future Java optimizations.
Understanding Project Leyden’s Mission
Project Leyden, an open-source initiative under OpenJDK, aims to address long-standing Java performance challenges: startup time, warmup time, and memory footprint. Per explained startup as the duration from launching a program to its first useful operation, like displaying “Hello World” or serving a Spring app’s initial request. Warmup, conversely, is the time to reach peak performance via JIT compilation. Leyden’s approach involves shifting computations earlier (e.g., at build time) or later (e.g., via lazy initialization) while preserving Java’s dynamic nature. Unlike GraalVM Native Image or Project CRaC, which sacrifice dynamism for speed, Leyden maintains compatibility, allowing developers to balance performance and flexibility.
Class Data Sharing (CDS) and AOT Cache: Today’s Solutions
Per introduced Class Data Sharing (CDS), a feature available since JDK 5, and its evolution into the Ahead-of-Time (AOT) Cache, a cornerstone of Leyden’s strategy. CDS preloads JDK classes, while AppCDS, introduced in JDK 10, extends this to application classes. The AOT Cache, an upcoming enhancement, stores class objects, resolved linkages, and method profiles, enabling near-instant startup. Sébastien demonstrated this with a Spring Boot Pet Clinic application, reducing startup from 3.2 seconds to 800 milliseconds using CDS and AOT Cache. The process involves a training run to generate the cache, which is then reused for faster deployments, though it requires consistent JVM and classpath configurations.
Spring Boot’s Synergy with Leyden
Sébastien highlighted the collaboration between the Spring and Leyden teams, initiated after a 2023 JVM Language Summit case study. Spring Boot 3.3 introduces features to simplify CDS and AOT Cache usage, such as extracting executable JARs into a CDS-friendly layout. A demo showed how a single command extracts the JAR, runs a training phase, and generates a cache, which is then embedded in a container image. This reduced startup times by up to 4x and memory usage by 20% when combined with Spring’s AOT optimizations. Sébastien also demonstrated how AOT Cache retains JIT “warmness,” enabling near-peak performance from startup, though a minor performance plateau gap is being addressed.
Future Horizons and Trade-offs
Looking ahead, Leyden plans to introduce stable values, a hybrid between mutable and immutable fields, offering final-like performance with flexible initialization. Per emphasized that Leyden avoids the heavy constraints of GraalVM (e.g., limited reflection) or CRaC (e.g., Linux-only, security concerns with serialized secrets). While CRaC achieves millisecond startups, its lifecycle complexities and security risks limit adoption. Leyden’s AOT Cache, conversely, offers significant gains (2–4x faster startups) with minimal constraints, making it ideal for most use cases. Developers can experiment with Leyden’s early access builds to optimize their applications, with further enhancements like code cache storage on the horizon.
Links:
Hashtags: #ProjectLeyden #Java #SpringBoot #AOTCache #CDS #StartupTime #JVM #DevoxxBE2024 #PerMinborg #SébastienDeleuze
[DefCon32] Windows Downdate: Downgrade Attacks Using Windows Updates
The notion of a “fully patched” system crumbles under the weight of downgrade attacks, as revealed by Alon Leviev, a self-taught security researcher at SafeBreach. His exploration of Windows Updates uncovers a flaw allowing attackers to revert critical components—DLLs, drivers, kernels, and virtualization stacks—to vulnerable versions, bypassing verification and exposing privilege escalations. Alon’s tool, Windows Downdate, renders the term “updated” obsolete, compromising systems worldwide.
Alon, a former Brazilian Jiu-Jitsu champion, leverages his expertise in OS internals and reverse engineering to dissect Windows Update mechanisms. Inspired by the BlackLotus UEFI bootkit, which bypassed Secure Boot via downgrades, he investigates whether similar vulnerabilities plague other components. His findings reveal a systemic design flaw, enabling unprivileged attackers to manipulate updates and disable protections like Virtualization-Based Security (VBS).
The implications are profound: downgraded systems report as fully updated, evade recovery tools, and block future patches, leaving them exposed to thousands of known vulnerabilities.
BlackLotus and the Downgrade Threat
Alon traces the research to BlackLotus, which exploited a patched Secure Boot flaw by reverting components. Secure Boot verifies boot chain signatures, but BlackLotus’s downgrade bypassed this, prompting Alon to probe Windows Updates for similar weaknesses.
He discovers that update packages, lacking robust validation, allow crafted downgrades. By manipulating update manifests, attackers revert critical files, exploiting old vulnerabilities without triggering alerts.
Compromising the Virtualization Stack
Targeting Hyper-V, Secure Kernel, and Credential Guard, Alon achieves downgrades that expose privilege escalations. VBS, designed to isolate sensitive operations, relies on UEFI locks, yet his methods disable these protections, a first in known research.
The attack exploits design flaws allowing less privileged rings to update higher ones, a remnant since VBS’s 2015 debut. Demonstrations show downgraded hypervisors, undermining Windows’ security architecture.
Restoration Vulnerabilities
A secondary flaw in update restoration scenarios amplifies the threat. Unprivileged users can trigger rollbacks, embedding malicious updates that persist across reboots. Recovery tools fail to detect these, as the system registers as compliant.
Alon’s Windows Downdate tool automates this, crafting updates that downgrade entire systems, from drivers to kernels, without administrative rights.
Industry Implications and Mitigations
The research exposes a gap in downgrade attack awareness. Alon urges thorough design reviews, emphasizing that unexamined surfaces, like update mechanisms, harbor risks. Linux and macOS may face similar threats, necessitating preemptive scrutiny.
Mitigations include enhanced validation, privilege restrictions, and monitoring for anomalous updates. His findings, shared responsibly with Microsoft, highlight the need for systemic changes to restore trust in patching.
Links:
[DevoxxBE2024] Java Language Futures by Gavin Bierman
Gavin Bierman, from Oracle’s Java Platform Group, captivated attendees at Devoxx Belgium 2024 with a forward-looking talk on Java’s evolution under Project Amber. Focusing on productivity-oriented language features, Gavin outlined recent additions like records, sealed classes, and pattern matching, while previewing upcoming enhancements like simplified main methods and flexible constructor bodies. His session illuminated Java’s design philosophy—prioritizing readability, explicit programmer intent, and compatibility—while showcasing how these features enable modern, data-oriented programming paradigms suited for today’s microservices architectures.
Project Amber’s Mission: Productivity and Intent
Gavin introduced Project Amber as a vehicle for delivering smaller, productivity-focused Java features, leveraging the six-month JDK release cadence to preview and finalize enhancements. Unlike superficial syntax changes, Amber emphasizes exposing programmer intent to improve code readability and reduce bugs. Compatibility is paramount, with breaking changes minimized, as Java evolves to address modern challenges distinct from its 1995 origins. Gavin highlighted how features like records and sealed classes make intent explicit, enabling the compiler to enforce constraints and provide better error checking, aligning with the needs of contemporary applications.
Records: Simplifying Data Carriers
Records, introduced to streamline data carrier classes, were a key focus. Gavin demonstrated how a Point class with two integers requires verbose boilerplate (constructors, getters, equals, hashCode) that obscures intent. Records (record Point(int x, int y)) eliminate this by auto-generating a canonical constructor, accessor methods, and value-based equality, ensuring immutability and transparency. This explicitness allows the compiler to enforce a contract: constructing a record from its components yields an equal instance. Records also support deserialization via the canonical constructor, ensuring domain-specific constraints, making them safer than traditional classes.
Sealed Classes and Pattern Matching
Sealed classes, shipped in JDK 17, allow developers to restrict class hierarchies explicitly. Gavin showed a Shape interface sealed to permit only Circle and Rectangle implementations, preventing unintended subclasses at compile or runtime. This clarity enhances library design by defining precise interfaces. Pattern matching, enhanced in JDK 21, further refines this by enabling type patterns and record patterns in instanceof and switch statements. For example, a switch over a sealed Shape interface requires exhaustive cases, eliminating default clauses and reducing errors. Nested record patterns allow sophisticated data queries, handling nulls safely without exceptions.
Data-Oriented Programming with Amber Features
Gavin illustrated how records, sealed classes, and pattern matching combine to support data-oriented programming, ideal for microservices exchanging pure data. He reimagined the Future class’s get method, traditionally complex due to multiple control paths (success, failure, timeout, interruption). By modeling the return type as a sealed AsyncReturn interface with four record implementations (Success, Failure, Timeout, Interrupted), and using pattern matching in a switch, developers handle all cases uniformly. This approach simplifies control flow, ensures exhaustiveness, and leverages Java’s type safety, contrasting with error-prone exception handling in traditional designs.
Future Features: Simplifying Java for All
Looking ahead, Gavin previewed features in JDK 23 and beyond. Simplified main methods allow beginners to write void main() without boilerplate, reducing cognitive load while maintaining full Java compatibility. The with expression for records enables concise updates (e.g., doubling a component) without redundant constructor calls, preserving domain constraints. Flexible constructor bodies (JEP 482) relax top-down initialization, allowing pre-super call logic to validate inputs, addressing issues like premature field access in subclass constructors. Upcoming enhancements include patterns for arbitrary classes, safe template programming, and array pattern matching, promising further productivity gains.
Links:
[DefCon32] Your AI Assistant Has a Big Mouth: A New Side-Channel Attack
As AI assistants like ChatGPT reshape human-technology interactions, their security gaps pose alarming risks. Yisroel Mirsky, a Zuckerman Faculty Scholar at Ben-Gurion University, alongside graduate students Daniel Eisenstein and Roy Weiss, unveils a novel side-channel attack exploiting token length in encrypted AI responses. Their research exposes vulnerabilities in major platforms, including OpenAI, Microsoft, and Cloudflare, threatening the confidentiality of personal and sensitive communications.
Yisroel’s Offensive AI Research Lab focuses on adversarial techniques, and this discovery highlights how subtle data leaks can undermine encryption. By analyzing network traffic, they intercept encrypted responses, reconstructing conversations from medical queries to document edits. Their findings, disclosed responsibly, prompted swift vendor patches, underscoring the urgency of securing AI integrations.
The attack leverages predictable token lengths in JSON responses, allowing adversaries to infer content despite encryption. Demonstrations reveal real-world impacts, from exposing personal advice to compromising corporate data, urging a reevaluation of AI security practices.
Understanding the Side-Channel Vulnerability
Yisroel explains the attack’s mechanics: AI assistants transmit responses as JSON objects, with token lengths correlating to content size. By sniffing HTTPS traffic, attackers deduce these lengths, mapping them to probable outputs. For instance, a query about a medical rash yields distinct packet sizes, enabling reconstruction.
Vulnerable vendors, unaware of this flaw until February 2025, included OpenAI and Quora. The team’s tool, GPTQ Logger, automates traffic analysis, highlighting the ease of exploitation in unpatched systems.
Vendor Responses and Mitigations
Post-disclosure, vendors acted decisively. OpenAI implemented padding to the nearest 32-byte value, obscuring token lengths. Cloudflare adopted random padding, further disrupting patterns. By March 2025, patches neutralized the threat, with five vendors offering bug bounties.
Yisroel emphasizes simple defenses: random padding, fixed-size packets, or increased buffering. These measures, easily implemented, prevent length-based inference, safeguarding user privacy.
Implications for AI Security
The discovery underscores a broader issue: AI services, despite their sophistication, inherit historical encryption pitfalls. Yisroel draws parallels to past side-channel attacks, where minor details like timing betrayed secrets. AI’s integration into sensitive domains demands rigorous security, akin to traditional software.
The work encourages offensive research to uncover similar weaknesses, advocating AI’s dual role in identifying and mitigating vulnerabilities. As new services emerge, proactive design is critical to prevent data exposure.
Broader Call to Action
Yisroel’s team urges the community to explore additional side channels, from compression ratios to processing delays. Their open-source tools invite further scrutiny, fostering a collaborative defense against evolving threats.
This research redefines AI assistant security, emphasizing meticulous data handling to protect user trust.
Links:
[DevoxxBE2024] Performance-Oriented Spring Data JPA & Hibernate by Maciej Walkowiak
At Devoxx Belgium 2024, Maciej Walkowiak delivered a compelling session on optimizing Spring Data JPA and Hibernate for performance, a critical topic given Hibernate’s ubiquity and polarizing reputation in Java development. With a focus on practical solutions, Maciej shared insights from his extensive consulting experience, addressing common performance pitfalls such as poor connection management, excessive queries, and the notorious N+1 problem. Through live demos and code puzzles, he demonstrated how to configure Hibernate and Spring Data JPA effectively, ensuring applications remain responsive and scalable. His talk emphasized proactive performance tuning during development to avoid production bottlenecks.
Why Applications Slow Down
Maciej opened by debunking myths about why applications lag, dismissing outdated notions that Java or databases are inherently slow. Instead, he pinpointed the root cause: misuse of technologies like Hibernate. Common issues include poor database connection management, which can halt applications, and issuing excessive or slow queries due to improper JPA mappings or over-fetching data. Maciej stressed the importance of monitoring tools like DataDog APM, which revealed thousands of queries in a single HTTP request in one of his projects, taking over 7 seconds. He urged developers to avoid guessing and use tracing tools or SQL logging to identify issues early, ideally during testing with tools like Digma’s IntelliJ plugin.
Optimizing Database Connection Management
Effective connection management is crucial for performance. Maciej explained that establishing database connections is costly due to network latency and authentication overhead, especially in PostgreSQL, where each connection spawns a new OS process. Connection pools, standardized in Spring Boot, mitigate this by creating a fixed number of connections (default: 10) at startup. However, developers must ensure connections are released promptly to avoid exhaustion. Using FlexyPool and Spring Boot Data Source Decorator, Maciej demonstrated logging connection acquisition and release times. In one demo, a transactional method unnecessarily held a connection for 273 milliseconds due to an external HTTP call within the transaction. Disabling spring.jpa.open-in-view reduced this to 61 milliseconds, freeing the connection after the transaction completed.
Transaction Management for Efficiency
Maciej highlighted the pitfalls of default transaction settings and nested transactions. By default, Spring Boot’s auto-commit mode triggers commits after each database interaction, but disabling it (spring.datasource.auto-commit=false) delays connection acquisition until the first database interaction, reducing connection hold times. For complex workflows, he showcased the TransactionTemplate for programmatic transaction management, allowing developers to define transaction boundaries within a method without creating artificial service layers. This approach avoids issues with @Transactional(propagation = Propagation.REQUIRES_NEW), which can occupy multiple connections unnecessarily, as seen in a demo where nested transactions doubled connection usage, risking pool exhaustion.
Solving the N+1 Problem and Over-Fetching
The N+1 query problem, a common Hibernate performance killer, occurs when lazy-loaded relationships trigger additional queries per entity. In a banking application demo, Maciej showed a use case where fetching bank transfers by sender ID resulted in multiple queries due to eager fetching of related accounts. By switching @ManyToOne mappings to FetchType.LAZY and using explicit JOIN FETCH in custom JPQL queries, he reduced queries to a single, efficient one. Additionally, he addressed over-fetching by using getReferenceById() instead of findById(), avoiding unnecessary queries when only entity references are needed, and introduced the @DynamicUpdate annotation to update only changed fields, optimizing updates for large tables.
Projections and Tools for Long-Term Performance
For read-heavy operations, Maciej advocated using projections to fetch only necessary data, avoiding the overhead of full entity loading. Spring Data JPA supports projections via records or interfaces, automatically generating queries based on method names or custom JPQL. Dynamic projections further simplify repositories by allowing runtime specification of return types. To maintain performance, he recommended tools like Hypersistence Optimizer (a commercial tool by Vlad Mihalcea) and QuickPerf (an open-source library, though unmaintained) to enforce query expectations in tests. These tools help prevent regressions, ensuring optimizations persist despite team changes or project evolution.