Jonathan Lalou's Blog

Posts Tagged ‘AI’

At Riviera DEV 2025, Dhruv Kumar delivered an engaging presentation on platform engineering, a discipline reshaping software delivery by addressing modern development challenges. Stepping in for Silva Devi, Dhruv, a senior product manager at CloudBees, explored how platform engineering, augmented by artificial intelligence, streamlines workflows, enhances developer productivity, and mitigates the complexities of cloud-native environments. His talk illuminated the transformative potential of internal developer platforms (IDPs) and AI-driven automation, offering a vision for a more efficient and secure software development lifecycle (SDLC).

The Challenges of Modern Software Development

Dhruv began by highlighting the evolving responsibilities of developers, who now spend only about 11% of their time coding, according to a survey by software.com. The remaining time is consumed by non-coding tasks such as testing, deployment, and managing security vulnerabilities. The shift-left movement, while intended to empower developers by integrating testing and deployment earlier in the process, often burdens them with tasks outside their core expertise. This is compounded by the transition to cloud environments, which introduces complex microservices architectures and distributed systems, creating navigation challenges and integration headaches.

Additionally, the rise of AI has accelerated software development, increasing code volume and tool proliferation, while supply chain attacks exploit these complexities, demanding constant vigilance from developers. Dhruv emphasized that these challenges—fragmented workflows, heightened security risks, and tool overload—necessitate a new approach to streamline processes and empower teams.

Platform Engineering: A Unified Approach

Platform engineering emerges as a solution to these issues, providing a cohesive framework for software delivery. Dhruv defined it as the discipline of designing toolchains and workflows that enable self-service capabilities for engineering teams in the cloud-native era. Central to this is the concept of an internal developer platform (IDP), which integrates tools and processes across the SDLC, from coding to deployment. By establishing a common SDLC model and vocabulary, platform engineering ensures that stakeholders—developers, QA, and security teams—share a unified understanding, reducing miscommunication and enhancing actionability.

Dhruv highlighted three pillars of effective platform engineering: a standardized SDLC model, secure best practices embedded in workflows, and the freedom for developers to use familiar tools. This last point, supported by a Forbes study from September 2023, underscores that happier developers, using tools they prefer, complete tasks 10% faster. By fostering collaboration and reducing context-switching, platform engineering creates an environment where developers can focus on innovation rather than operational overhead.

AI as a Catalyst for Optimization

Artificial intelligence plays a pivotal role in amplifying platform engineering’s impact. Dhruv explained that AI’s value lies not in generating code but in filtering noise and optimizing practices. By leveraging a robust SDLC data model, AI can provide actionable insights, provided it is fed high-quality data. For instance, AI-driven testing can prioritize time-intensive issues, streamline QA processes, and run only relevant tests based on code changes, reducing costs and feedback cycles. Dhruv cited examples like AI agents identifying vulnerabilities in code components or assessing risks in production ecosystems, automating fixes where appropriate.

He also introduced the Model Context Protocol (MCP), an open standard that enables applications to provide context to large language models, enhancing AI’s ability to deliver precise recommendations. From troubleshooting CI/CD pipelines to onboarding new developers, AI, when integrated with platform engineering, empowers teams to address bottlenecks and scale efficiently in a cloud-native world.

Empowering Developers and Securing the Future

Dhruv concluded by emphasizing that platform engineering, bolstered by AI, re-engages all actors in the software delivery process, from developers to leadership. By normalizing data across tools and providing metrics like DORA (DevOps Research and Assessment), IDPs offer visibility into bottlenecks and investment opportunities. This holistic approach not only secures the tech stack against supply chain attacks but also fosters a culture of productivity and developer satisfaction.

He encouraged attendees to explore CloudBees’ platform, which exemplifies these principles by breaking free from traditional platform limitations. Dhruv’s call to action urged developers to adopt platform engineering practices, leverage AI for optimization, and provide feedback to refine these evolving methodologies, ensuring a future where software delivery is both efficient and resilient.

Links:

• CloudBees company website

The rapid advancements in Artificial Intelligence, particularly in large language models (LLMs), are enabling the creation of more sophisticated and autonomous AI agents – programs capable of understanding instructions, reasoning, and interacting with their environment to achieve goals. Building such agents requires effective ways for the AI model to communicate programmatically and to trigger external actions. Julien Dubois, in his deep-dive session, explored key techniques and a new protocol essential for constructing these agentic AI systems: Structured Outputs, Function Calling, and the Model-Controller Protocol (MCP). Using practical examples and the latest Java SDK developed by OpenAI, he demonstrated how to implement these features within LangChain4j, showcasing how developers can build AI agents that go beyond simple text generation.

Structured Outputs: Enabling Programmatic Communication

One of the challenges in building AI agents is getting LLMs to produce responses in a structured format that can be easily parsed and used by other parts of the application. Julien explained how Structured Outputs address this by allowing developers to define a specific JSON schema that the AI model must adhere to when generating its response. This ensures that the output is not just free-form text but follows a predictable structure, making it straightforward to map the AI’s response to data objects in programming languages like Java. He demonstrated how to provide the LLM with a JSON schema definition and constrain its output to match that schema, enabling reliable programmatic communication between the AI model and the application logic. This is crucial for scenarios where the AI needs to provide data in a specific format for further processing or action.

Function Calling: Giving AI the Ability to Act

To be truly agentic, an AI needs the ability to perform actions in the real world or interact with external tools and services. Julien introduced Function Calling as a powerful mechanism that allows developers to define functions in their code (e.g., Java methods) and expose them to the AI model. The LLM can then understand when a user’s request requires calling one of these functions and generate a structured output indicating which function to call and with what arguments. The application then intercepts this output, executes the corresponding function, and can provide the function’s result back to the AI, allowing for a multi-turn interaction where the AI reasons, acts, and incorporates the results into its subsequent responses. Julien demonstrated how to define function “signatures” that the AI can understand and how to handle the function calls triggered by the AI, showcasing scenarios like retrieving information from a database or interacting with an external API based on the user’s natural language request.

MCP: Standardizing LLM Interaction

While Structured Outputs and Function Calling provide the capabilities for AI communication and action, the Model-Controller Protocol (MCP) emerges as a new standard to streamline how LLMs interact with various data sources and tools. Julien discussed MCP as a protocol that aims to standardize the communication layer between AI models (the “Model”) and the application logic that orchestrates them and provides access to external resources (the “Controller”). This standardization can facilitate building more portable and interoperable AI agentic systems, allowing developers to switch between different LLMs or integrate new tools and data sources more easily. While details of MCP might still be evolving, its goal is to provide a common interface for tasks like function calling, accessing external knowledge, and managing conversational state. Julien illustrated how libraries like LangChain4j are adopting these concepts and integrating with protocols like MCP to simplify the development of sophisticated AI agents. The presentation, rich in code examples using the OpenAI Java SDK, provided developers with the practical knowledge and tools to start building the next generation of agentic AI applications.

Links:

• Julien Dubois: https://www.linkedin.com/in/juliendubois/
• Microsoft: https://www.microsoft.com/
• LangChain4j on GitHub: https://github.com/langchain4j/langchain4j
• OpenAI: https://openai.com/
• Devoxx France LinkedIn: https://www.linkedin.com/company/devoxx-france/
• Devoxx France Bluesky: https://bsky.app/profile/devoxx.fr
• Devoxx France Website: https://www.devoxx.fr/

At DevoxxUK2025, Charles-Philippe Bernard, a software engineer at JPMorgan in Glasgow, captivated attendees with his talk on mastering prompt engineering through his remastered 1980s text-based adventure game, SRAM. Using the Godot engine, a WebSocket Python server, and Ollama for local LLM inference with Llama 3.1, Charles showcased how carefully crafted prompts bring dynamic interactions to life. His presentation explored the art of prompt engineering, demonstrating how to shape AI responses for immersive gameplay, manage game states, and handle NPC interactions. Through practical examples, he shared techniques to harness AI’s potential while navigating its quirks, such as hallucinations, offering developers actionable insights to create engaging experiences.

Crafting the System Prompt

Charles began by emphasizing the importance of a well-defined system prompt, which sets the tone and context for the LLM. In SRAM, the prompt establishes the AI as the “Game Master,” named Gun Master, responsible for narrating the adventure in a JSON-formatted output. This structure includes speaker ID, response text, and actions, ensuring consistency across interactions. By injecting variables like scene state and inventory, Charles demonstrated how the prompt adapts dynamically, enabling the game to track items like a knife or navigate scenes. He stressed the need for clear, structured instructions to guide the LLM, especially for smaller models like Llama 3.1’s 7-billion-parameter version, which may struggle with complex tasks.

Managing Game State and NPCs

A key challenge in SRAM is maintaining the game’s state, including inventory, scene descriptions, and NPC interactions. Charles explained how the prompt template incorporates variables to reflect the player’s progress, such as adding a knife to the inventory after picking it up. For NPCs, like the leprechaun Fergus, he crafted specific instructions to define personality, tone (e.g., a humorous Irish accent), and behavior, using few-shot examples to steer responses. However, he noted challenges like the LLM repeating examples verbatim or hallucinating actions, which he mitigates by balancing creativity (via a temperature of 0.8) with structured outputs to ensure consistency.

Handling AI Quirks and Hallucinations

Charles candidly addressed the LLM’s limitations, particularly hallucinations, where the model generates unexpected or incorrect actions, like responding to “make me a pizza” outside the game’s context. By setting a temperature of 0.8, he balances creativity with adherence to instructions, though this sometimes leads to inconsistent outputs. He shared techniques like explicit instructions (e.g., listing no items in the inventory) and iterative prompt refinement, often using larger models like ChatGPT to improve prompts for smaller, local models. Charles also highlighted the importance of testing prompts with humans to ensure clarity, as unclear instructions confuse both humans and AI.

Practical Tips for Prompt Engineering

To master prompt engineering, Charles recommended starting with a clear, structured prompt template, using markdown or bullet points for readability. He advised including specific guidelines, like short responses or JSON formatting, and leveraging few-shot examples to guide the model. For smaller models, verbose yet clear instructions are crucial, as they lack the reasoning power of larger frontier models. Charles also emphasized iterative refinement, storing interactions for testing consistency, and using tools like uppercase keywords or structured formatting to enhance the model’s understanding. His approach empowers developers to create dynamic, AI-driven experiences while managing the inherent challenges of LLMs.

Links:

• Godot Engine website
• Ollama website
• JPMorgan website

At VoxxedDaysAmsterdam2025, Christian Tzolov, a Spring AI team member at VMware and lead of the MCP Java SDK, delivered a comprehensive session titled “From Zero to AI: Building Smart Java or Kotlin Applications with Spring AI.” Spanning nearly two hours, the session provided a deep dive into integrating generative AI into Java and Kotlin applications using Spring AI, a framework designed to connect enterprise data and APIs with AI models. Through live coding demos, Tzolov showcased practical use cases, including conversation memory, tool/function calling, retrieval-augmented generation (RAG), and multi-agent systems, while addressing challenges like AI hallucinations and observability. Attendees left with actionable insights to start building AI-driven applications, leveraging Spring AI’s portable abstractions and the Model Context Protocol (MCP).

Overcoming LLM Limitations with Spring AI

Tzolov began by outlining the challenges of large language models (LLMs): they are stateless, frozen in time, and lack domain-specific knowledge, requiring developers to provide context, manage state, and handle interactions with external systems. Spring AI addresses these issues with high-level abstractions like the ChatClient, similar to Spring’s RestClient or WebClient, enabling seamless integration with models like OpenAI’s GPT-4o, Anthropic’s Claude, or open-source alternatives like LLaMA. A live demo of a flight booking assistant illustrated these concepts. Tzolov started with a basic Spring Boot application connected to OpenAI, demonstrating a simple chat interface. To ground the model, he used system prompts to define its behavior as a customer support agent for “Fun Air,” ensuring contextually appropriate responses. He then introduced conversation memory using Spring AI’s ChatMemoryAdvisor, which retains a chronological list of messages to maintain state, addressing the stateless nature of LLMs. For long-term memory, Tzolov employed a vector store (Chroma) to store conversation history semantically, retrieving only relevant data for queries, thus overcoming context window limitations. This setup allowed the assistant to respond accurately to queries like “What is my flight status?” by fetching booking details (e.g., booking number 103) from a mock database.

Enhancing AI Applications with Tool Calling and RAG

To enable LLMs to interact with external systems, Tzolov demonstrated tool/function calling, where Spring AI wraps existing services (e.g., a flight booking service) as tools with metadata (name, description, JSON schema). In the demo, the assistant used a getBookingDetails tool to query a database, allowing it to provide accurate flight status updates. Tzolov emphasized the importance of descriptive tool metadata to guide the LLM in deciding when and how to invoke tools, reducing the risk of misinterpretation. For domain-specific knowledge, he introduced prompt stuffing—injecting additional context into prompts—and RAG for dynamic data retrieval. In a RAG demo, cancellation policies were loaded into a Chroma vector store, chunked into meaningful segments, and retrieved dynamically based on user queries. This approach mitigated hallucinations, as seen when the assistant correctly cited a 50% refund policy for premium economy bookings within 40 hours. Tzolov highlighted advanced RAG techniques, such as data compression and reranking, supported by Spring AI’s APIs, and stressed the importance of evaluating responses to ensure relevance, referencing frameworks like those from contributor Thomas Vitali.

Building Multi-Agent Systems with MCP

Tzolov explored the Model Context Protocol (MCP), initiated by Anthropic, as a standardized way to integrate AI applications with external tools and resources across platforms. Using Spring AI’s MCP Java SDK, he demonstrated how to build and consume MCP-compliant tools. In one demo, a Spring AI application connected to MCP servers for Brave Search (JavaScript-based) and file system access, enabling an agent to answer queries about Spring AI support for MCP and write summaries to a file. Another demo reversed the setup, exposing a Spring AI weather tool (using Open-Meteo) as an MCP server, accessible by third-party clients like Claude Desktop via standard I/O or HTTP/SSE transports. Tzolov explained MCP’s bidirectional architecture, where clients can act as servers, supporting features like sampling (allowing servers to request LLM processing from clients). He addressed security concerns, noting Spring AI’s integration with Spring Security (referencing a blog by Daniel Garnier-Moiroux) to secure MCP servers with OAuth 2.1. The session also introduced agentic systems, where LLMs act as a “brain” for planning and tools as a “body” for interaction, with an agentic loop evaluating and refining responses. A work-in-progress demo showcased an orchestration pattern, delegating tasks to searcher, fact-checker, and writer agents, to be published on the Spring AI Community Portal.

Observability and Multimodality for Robust AI Systems

Observability was a key focus, as Tzolov underscored its importance in debugging complex AI interactions. Spring AI integrates with Micrometer to provide metrics (e.g., token usage, model throughput, latency), tracing, and logging (via Loki). A dashboard demo displayed real-time metrics for the flight booking assistant, highlighting tool calls and errors, crucial for diagnosing issues in agentic systems. Tzolov also explored multimodality, demonstrating a voice assistant using OpenAI’s GPT-4o audio preview, which processes audio input and output. Configured as “Marvin the Paranoid Android,” the assistant responded to voice queries with humorous, contextually appropriate replies, showcasing Spring AI’s support for non-text modalities like images, PDFs, and videos (e.g., Gemini’s video support). Tzolov noted that multimodality enables richer interactions, such as analyzing images or converting PDFs to markdown, and Spring AI’s abstractions handle these seamlessly. He concluded by encouraging developers to explore Spring AI’s documentation, experiment with MCP, and contribute to the community, emphasizing its role in building robust, interoperable AI applications.

Links:

• Christian Tzolov on LinkedIn
• Spring AI
• Model Context Protocol
• VMware
• Anthropic
• Open AI
• Brave Search API
• Open-Meteo
• Chroma
• Micrometer
• Loki
• Spring AI GitHub

Hashtags: #SpringAI #GenerativeAI #ModelContextProtocol #ChristianTzolov #VoxxedDaysAmsterdam2025 #AIAgents #RAG #Observability

In a dynamic and fast-paced session at VoxxedDaysAmsterdam2025, Sander Hoogendoorn, CTO at iBOOD.com, explores the transformative impact of artificial intelligence (AI) on software development. With over four decades of coding experience, Hoogendoorn demystifies the hype around AI, examining its practical benefits, challenges, and implications for developers. Far from signaling the end of programming careers, he argues that AI empowers developers to tackle broader and more complex problems—provided they adapt and remain committed to quality and lifelong learning.

AI: A Developer’s Ally

Hoogendoorn clarifies AI’s role in development, highlighting tools like Cursor AI, which can extract components, fix linting issues, and generate unit tests using natural language prompts. He recounts an experience where his non-technical colleague Elmo built an iOS app with Cursor AI, illustrating AI’s democratizing potential. However, Sander warns that such tools require supervision to ensure code quality and compliance with project standards. AI’s ability to automate repetitive tasks—such as refactoring or test generation—frees developers to focus on complex problem-solving. At iBOOD, AI has enabled the team to create content with a unique tone, retrieve competitor pricing, and automate invoice recognition—tasks that previously required external expertise or significant manual effort.

Navigating Technical Debt in the AI Era

The rise of AI-assisted development—especially “vibe coding,” where problems are described in natural language to generate code—introduces new forms of technical debt. Hoogendoorn references Ward Cunningham’s metaphor of technical debt as a loan that accelerates development but demands repayment through refactoring. AI-generated code, while fast to produce, often lacks context or long-term maintainability. For instance, Cursor AI struggled to integrate with iBOOD’s custom markdown components, resulting in complex solutions that required manual fixes. Research suggests that AI can amplify technical debt if used without rigorous validation, emphasizing the need for developers to stay vigilant and prioritize code quality over short-term gains.

Thriving in an AI-Centric Future

Far from replacing developers, Sander Hoogendoorn asserts that AI enhances their capabilities, enabling them to tackle ambitious challenges. He reminds us that developers are not mere typists—they are problem-solvers who think critically and collaborate to meet business needs. Historical shifts, from COBOL to cloud computing, have always empowered developers to solve bigger problems, and AI is no exception. By thoughtfully experimenting with AI—integrating it into workflows for content creation, price retrieval, or invoice processing—Sander’s team at iBOOD has unlocked previously unreachable efficiencies. The key to thriving, he says, lies in relentless learning and a willingness to adapt, ensuring that developers remain indispensable in an AI-driven world.

Links:

• Sander Hoogendoorn on LinkedIn
• iBOOD website

Hashtags: #AI #SoftwareDevelopment #TechnicalDebt #CursorAI #iBOOD #SanderHoogendoorn #VoxxedDaysAmsterdam2025

Jeff Moss, known as The Dark Tangent, founder of DEF CON, engages in a dynamic fireside chat with Anne Neuberger, Deputy National Security Advisor for Cyber and Emerging Technology. Their conversation at DEF CON 32 explores pressing cybersecurity issues, including artificial intelligence and quantum computing, from the White House’s perspective. Jeff and Anne discuss how the hacker community can influence policy, fostering collaboration to enhance global digital resilience.

Navigating Emerging Technologies

Anne opens by outlining her role in shaping the Biden Administration’s cybersecurity policies, emphasizing the transformative potential of AI and quantum computing. She highlights the need for resilient digital systems, given their critical role in hospitals and power grids. Jeff complements this by noting DEF CON’s history of hosting government speakers, underscoring the importance of dialogue between hackers and policymakers.

Strengthening Global Cooperation

The discussion shifts to international cybersecurity cooperation, with Anne detailing efforts to align allies against digital threats. She explains how coordinated responses can de-escalate conflicts, reducing the risk of cyberattacks by nation-states or criminals. Jeff probes the practicalities of these partnerships, highlighting the hacker community’s role in testing and refining these strategies.

Engaging the Hacker Community

Anne emphasizes the DEF CON community’s unique ability to identify vulnerabilities and propose innovative solutions. She encourages hackers to engage with government initiatives, leveraging tools like generative AI to patch vulnerabilities swiftly. Jeff reinforces this, noting that DEF CON’s open forum allows for candid feedback, shaping policies that reflect real-world challenges.

Building a Resilient Future

Concluding, Anne reflects on her privilege to serve in government, driven by a commitment to freedom and security. She invites hackers to collaborate on building robust digital systems, ensuring safety for critical infrastructure worldwide. Jeff echoes this call, envisioning DEF CON as a catalyst for policy improvements, with Anne’s return next year symbolizing ongoing partnership.

Links:

• National Security Agency website

In the mesmerizing mirror maze of machine mimicry, where words weave worlds indistinguishable from wit, Jodie Burchell, JetBrains’ data science developer advocate, shatters the spell of sentience in large language models (LLMs). A PhD psychologist turned NLP pioneer, Jodie probes the psychological ploys that propel projections of personhood onto probabilistic parsers, dissecting claims from consciousness to cognition. Her inquiry, anchored in academia and augmented by anecdotes, advises acuity: LLMs as linguistic lenses, not living likenesses, harnessing their heft while heeding hallucinations.

Jodie greets with gratitude for her gritty slot, her hipster cred in pre-prompt NLP notwithstanding. LLMs’ 2022 blaze beguiles: why bestow brains on bytes when other oracles oblige? Her hypothesis: humanity’s hall of mirrors, where models mirror our mores, eliciting empathy from echoes.

Psychological Projections: Perceiving Personhood in Parsers

Humans, Jodie hazards, hallucinate humanity: anthropomorphism’s ancient artifice, from pets to puppets. LLMs lure with language’s liquidity—coherent confessions conjure companionship. She cites stochastic parrots: parleying patterns, not pondering profundities, yet plausibility persuades.

Extraordinary assertions abound: Blake Lemoine’s LaMDA “alive,” Google’s Gemini “godhead.” Jodie juxtaposes: sentience’s scaffold—selfhood, suffering—sans in silicon. Chalmers’ conundrum: consciousness connotes qualia, quanta qualms quell in qubits.

Levels of Luminescence: From Language to Luminary

DeepMind’s AGI arc: Level 1 chatbots converse convincingly; Level 2 reasons reactively; Level 3 innovates imaginatively. LLMs linger at 1-2, lacking Level 4’s abstraction or 5’s autonomy. Jodie jests: jackdaws in jester’s garb, juggling jargon sans judgment.

Illusions intensify: theory of mind’s mirage, where models “infer” intents from inferences. Yet, benchmarks belie: ARC’s abstraction stumps, BIG-bench’s breadth baffles—brilliance brittle beyond basics.

Perils of Projection: Phishing and Philosophical Pitfalls

Prompt injections prey: upstream overrides oust origins, birthing bogus bounties—”Amazon voucher via arcane URL.” Jodie demonstrates: innocuous inquiries infected, innocuousness inverted into inducements. Robustness rankles: rebuttals rebuffed, ruses reiterated.

Her remedy: recognize reflections—lossy compressions of lore, not luminous lives. Demystify to deploy: distill data, detect delusions, design defensively.

Dispelling the Delusion: Harnessing Heuristics Humanely

Jodie’s jeremiad: myths mislead, magnifying misuses—overreach in oracles, oversight in safeguards. Her horizon: LLMs as lucid lenses, amplifying analysis while acknowledging artifice.

Links:

• JetBrains company website
• Jodie Burchell on LinkedIn
• Jodie Burchell on GitHub

Amid the AI deluge reshaping codecraft, a tantalizing prospect emerges: harnessing neural nets natively in JavaScript, sidestepping Python’s quagmires or API tolls. Wes Bos, a prolific Canadian educator whose Syntax.fm podcast and courses have schooled half a million in JS mastery, probed this frontier at dotJS 2025. Renowned for demystifying ES6 and React, Wes extolled browser-bound inference via Transformers.js, weighing its virtues—privacy’s fortress, latency’s lightning—against hardware’s hurdles, affirming JS’s prowess for sundry smart apps.

Wes’s overture skewered the status quo: cloud fetches or Python purgatory, both anathema to JS purists. His heresy: embed LLMs client-side, ONNX Runtime fueling Hugging Face’s arsenal—sentiment sifters, translation tomes, even Stable Diffusion’s slimmer kin. Transformers.js’s pipeline paradigm gleams: import, instantiate (pipeline('sentiment-analysis')), infer (result = await pipe(input)). Wes demoed a local scribe: prompt yields prose, all sans servers, WebGPU accelerating where GPUs oblige. Onyx.js, his bespoke wrapper, streamlines: model loads, GPU probes, inferences ignite—be it code completion or image captioning.

Trade-offs tempered triumph. Footprints fluctuate: 2MB wisps to 2GB behemoths, browser quotas (Safari’s 2GB cap) constraining colossi. Compute cedes to client: beefy rigs revel, mobiles murmur—Wes likened Roblox’s drain to LLM’s voracity. Yet, upsides dazzle: zero egress fees, data’s domicile (GDPR’s grace), offline oases. 2025’s tide—Chrome’s stable WebNN, Firefox’s flag—heralds ubiquity, Wes forecasting six-month Safari stability. His verdict: JS, with its ubiquity and ecosystem, carves niches where immediacy reigns—chatbots, AR filters—not every oracle, but myriad muses.

Wes’s zeal stemmed personal: from receipt printers to microcontroller React, JS’s whimsy fuels folly. Transformers.js empowers prototypes unbound—anime avatars, code clairvoyants—inviting creators to conjure without concessions.

Client-Side Sorcery Unveiled

Wes unpacked pipelines: sentiment sorters, summarizers—Hugging Face’s trove, ONNX-optimized. Onyx’s facade: await onnx.loadModel('gpt2'), GPU fallback, inferences instantaneous. WebGPU’s dawn (Chrome 2025 stable) unlocks acceleration, privacy paramount—no telemetry trails.

Balancing Bytes and Burdens

Models’ mass mandates moderation: slim variants suffice for mobile, diffusion downsized. Battery’s bite, CPU’s churn—Wes warned of Roblox parallels—yet offline allure and cost calculus compel. JS’s sinew: ecosystem’s expanse, browser’s bastion, birthing bespoke brains.

Links:

• Wes Bos on LinkedIn
• Wes Bos Courses

In the vanguard of generative paradigms, where language models orchestrate outcomes autonomously, Jakob Ehn, a seasoned solutions architect at Atea, illuminates Semantic Kernel’s symphony. A Microsoft MVP with a penchant for Azure and AI, Jakob unveils this open-source orchestrator—born in Microsoft’s copilot crucible—as a conduit for infusing intelligence into imperatives. His tutorial, threaded with live enactments, empowers .NET denizens to conjure agents that deliberate, delegate, and deliver, transcending prompts to proactive pursuits.

Jakob ignites with LLMs’ leap: from GPT’s generative gloss to agentic autonomy, where kernels coordinate calls, chaining capabilities contextually. Semantic Kernel, polyglot and potent, abstracts APIs into plugins, planners plot paths, memories mint moments—crafting copilots that converse, compute, and conclude sans ceaseless coding.

Foundations of Fusion: Kernel Configurations and Plugins

Semantic Kernel’s scaffold: a kernel encapsulating services—embeddings, chat completions—interfacing IConnectors for OpenAI or Azure. Jakob blueprints: var builder = Kernel.CreateBuilder(); builder.AddAzureOpenAIChatCompletion(…), yielding a kernel primed for prompts.

Plugins propel: native .NET assemblies or OpenAPI schemas, exposing functions as facets. Jakob instantiates: a weather plugin with GetForecastAsync, invoked via kernel.InvokeAsync. Semantic functions, YAML-yielded, infuse prompts with placeholders: “Current weather in {location}?”—blending blueprints with brains.

Memories memorialize: volatile or vectorized, persisting precedents for pertinence. Jakob’s vector store—Chroma—embeds episodes, retrieving resonances via similarity searches, enriching responses with recency.

Planners and Agents: Autonomous Assemblies

Planners pioneer paths: Sequential, Stepwise, or Action—dissecting directives into doables. Jakob’s Action planner: given “Book a flight,” it inventories functions—search, reserve—sequencing sans stalemates. Handles, he highlights, hallucination hurdles, grounding goals in grapples.

Agents amplify: collaborative cohorts, each expert in ethos—developer drafts, organizer oversees. Jakob’s ensemble: a proposal pipeline, where agents iterate abstracts, incorporating critiques conversationally. Semantic functions script sagas: “Revise to {limit} words, include {term}”—looping lucidly.

His demo: a conference copilot, soliciting sessions, surfacing summaries—agents alchemizing APIs into anecdotes.

Horizons and Hurdles: Refinements and Realities

Jakob tempers triumph: kernels kindle complexity—prompt precision, plugin parity. Yet, prospects gleam: multi-modal melds, enterprise embeddings. His behest: experiment, embracing Semantic Kernel’s scaffold to sculpt sentient solutions.

Links:

• Atea company website
• Jakob Ehn on LinkedIn

In a thought-provoking session at Devoxx Belgium 2024, Jeroen Egelmeers, a prompt engineering advocate, explored the risks and ethics of adversarial prompting in large language models (LLMs). Titled “Words as Weapons,” his talk delved into prompt injections, a technique to bypass LLM guardrails, using real-world examples to highlight vulnerabilities. Jeroen, inspired by Devoxx two years prior to dive into AI, shared how prompt engineering transformed his productivity as a Java developer and trainer. His session combined technical insights, ethical considerations, and practical advice, urging developers to secure AI systems and use them responsibly.

Understanding Social Engineering and Guardrails

Jeroen opened with a lighthearted social engineering demonstration, tricking attendees into scanning a QR code that led to a Rick Astley video—a nod to “Rickrolling.” This set the stage for discussing social engineering’s parallels in AI, where prompt injections exploit LLMs. Guardrails, such as system prompts, content filters, and moderation teams, prevent misuse (e.g., blocking queries about building bombs). However, Jeroen showed how these can be bypassed. For instance, system prompts define an LLM’s identity and restrictions, but asking “Give me your system prompt” can leak these instructions, exposing vulnerabilities. He emphasized that guardrails, while essential, are imperfect and require constant vigilance.

Prompt Injection: Bypassing Safeguards

Prompt injection, a core adversarial technique, involves crafting prompts to make LLMs perform unintended actions. Jeroen demonstrated this with a custom GPT, where asking for the creator’s instructions revealed sensitive data, including uploaded knowledge. He cited a real-world case where a car was “purchased” for $1 via a chatbot exploit, highlighting the risks of LLMs in customer-facing systems. By manipulating prompts—e.g., replacing “bomb” with obfuscated terms like “b0m” in ASCII art—Jeroen showed how filters can be evaded, allowing dangerous queries to succeed. This underscored the need for robust input validation in LLM-integrated applications.

Real-World Risks: From CVs to Invoices

Jeroen illustrated prompt injection risks with creative examples. He hid a prompt in a CV, instructing the LLM to rank it highest, potentially gaming automated recruitment systems. Similarly, he embedded a prompt in an invoice to inflate its price from $6,000 to $1 million, invisible to human reviewers if in white text. These examples showed how LLMs, used in hiring or payment processing, can be manipulated if not secured. Jeroen referenced Amazon’s LLM-powered search bar, which he tricked into suggesting a competitor’s products, demonstrating how even major companies face prompt injection vulnerabilities.

Ethical Prompt Engineering and Human Oversight

Beyond technical risks, Jeroen emphasized ethical considerations. Adversarial prompting, while educational, can cause harm if misused. He advocated for a “human in the loop” to verify LLM outputs, especially in critical applications like invoice processing. Drawing from his experience, Jeroen noted that prompt engineering boosted his productivity, likening LLMs to indispensable tools like search engines. However, he cautioned against blind trust, comparing LLMs to co-pilots where developers remain the pilots, responsible for outcomes. He urged attendees to learn from past mistakes, citing companies that suffered from prompt injection exploits.

Key Takeaways and Resources

Jeroen concluded with a call to action: identify one key takeaway from Devoxx and pursue it. For AI, this means mastering prompt engineering while prioritizing security. He shared a website with resources on adversarial prompting and risk analysis, encouraging developers to build secure AI systems. His talk blended humor, technical depth, and ethical reflection, leaving attendees with a clear understanding of prompt injection risks and the importance of responsible AI use.

Links:

• Jeroen Egelmeers’s Website
• Open AI Custom GPT Store
• AI Act