Jonathan Lalou's Blog

Posts Tagged ‘AI’

At Devoxx Greece 2024, Henry Lagarde, a senior software engineer at Criteo, introduced audiences to LangChain, a versatile framework for building AI-powered applications. With infectious enthusiasm and live demonstrations, Henry showcased how LangChain simplifies interactions with large language models (LLMs), enabling developers to create context-aware, reasoning-driven tools. His talk, rooted in his experience at Criteo, a leader in retargeting and retail media, highlighted LangChain’s composability and community-driven evolution, offering a practical guide for AI integration.

LangChain’s Ecosystem and Composability

Henry began by defining LangChain as a framework for building context-aware reasoning applications. Unlike traditional LLM integrations, LangChain provides modular components—prompt templates, LLM abstractions, vector stores, text splitters, and document loaders—that integrate with external services rather than hosting them. This composability allows developers to switch LLMs seamlessly, adapting to changes in cost or performance without rewriting code. Henry emphasized LangChain’s open-source roots, launched in late 2022, and its rapid growth, with versions in Python, TypeScript, Java, and more, earning it the 2023 New Tool of the Year award.

The ecosystem extends beyond core modules to include LangServe for REST API deployment, LangSmith for monitoring, and a community hub for sharing prompts and agents. This holistic approach supports developers from prototyping to production, making LangChain a cornerstone for AI engineering.

Building a Chat Application

In a live demo, Henry showcased LangChain’s simplicity by recreating a ChatGPT-like application in under 10 lines of Python code. He instantiated an OpenAI client using GPT-3.5 Turbo, implemented chat history for context awareness, and used prompt templates to define system and human messages. By combining these components, he enabled streaming responses, mimicking ChatGPT’s real-time output without the $20 monthly subscription. This demonstration highlighted LangChain’s ability to handle memory, input/output formatting, and LLM interactions with minimal effort, empowering developers to build cost-effective alternatives.

Henry noted that LangChain’s abstractions, such as strong typing and output parsing, eliminate manual prompt engineering, ensuring robust integrations even when APIs change. The demo underscored the framework’s accessibility, inviting developers to experiment with its capabilities.

Creating an AI Agent for PowerPoint Generation

Henry’s second demo illustrated LangChain’s advanced features by building an AI agent to generate PowerPoint presentations. Using TypeScript, he configured a system prompt from LangSmith’s community hub, defining the agent’s tasks: researching a topic via the Serper API and generating a structured PowerPoint. He defined tools with Zod for runtime type checking, ensuring consistent outputs, and integrated callbacks for UI tracing and monitoring.

The agent, powered by Anthropic’s Claude model, performed internet research on Google Cloud, compiled findings, and generated a presentation with sourced information. Despite minor delays, the demo showcased LangChain’s ability to orchestrate complex workflows, combining research, data processing, and content creation. Henry’s use of LangSmith for prompt optimization and monitoring highlighted the framework’s production-ready capabilities.

Community and Cautions

Henry emphasized LangChain’s vibrant community, which drives its multi-language support and rapid evolution. He encouraged attendees to contribute, noting the framework’s open-source ethos and resources like GitHub for further exploration. However, he cautioned against over-reliance on LLMs, citing their occasional laziness or errors, as seen in ChatGPT’s simplistic responses. LangChain, he argued, augments developer workflows but requires careful integration to ensure reliability in production environments.

His vision for LangChain is one of empowerment, enabling developers to enhance applications incrementally while maintaining control over AI-driven processes. By sharing his demo code on GitHub, Henry invited attendees to experiment and contribute to LangChain’s growth.

Conclusion

Henry’s presentation at Devoxx Greece 2024 was a compelling introduction to LangChain’s potential. Through practical demos and insightful commentary, he demonstrated how the framework simplifies AI development, from basic chat applications to sophisticated agents. His emphasis on composability, community, and cautious integration resonated with developers eager to explore AI. As LangChain continues to evolve, Henry’s talk serves as a blueprint for harnessing its capabilities in real-world applications.

Links:

• Criteo company website
• LangChain website

Akshata Sawant, a Senior Developer Advocate at Salesforce, delivered an insightful presentation at DevoxxUK2024, illuminating the transformative potential of AI-enabled APIs. With a career spanning seven years in API development and a recent co-authored book on MuleSoft for Salesforce developers, Akshata expertly navigates the convergence of artificial intelligence and application programming interfaces. Her talk explores how AI-powered APIs are reshaping industries by enhancing automation, data analysis, and user experiences, while also addressing critical ethical and security considerations. Through practical examples and a clear framework, Akshata demonstrates how these technologies synergize to create smarter, more connected systems.

The Evolution of APIs and AI Integration

Akshata begins by likening APIs to a waiter, facilitating seamless communication between disparate systems, such as a customer ordering food and a kitchen preparing it. This analogy underscores the fundamental role of APIs in enabling interoperability across applications. She traces the evolution of APIs from the cumbersome Enterprise JavaBeans (EJB) and SOAP-based systems to the more streamlined REST APIs, noting their pervasive adoption across industries. The advent of AI has further accelerated this evolution, leading to what Akshata terms “API sprawling,” where APIs are integral to integration ecosystems. She introduces three key aspects of AI-enabled APIs: consuming pre-built AI APIs, using AI to streamline API development, and embedding AI models into custom APIs to enhance functionality.

Practical Applications of AI-Enabled APIs

The first aspect Akshata explores is the use of pre-built AI APIs, which are readily available from providers like Google Cloud and Microsoft Azure. These APIs, encompassing generative AI, text, language, image, and video processing, allow developers to integrate advanced capabilities without building complex models from scratch. For instance, Google Cloud’s AI APIs offer use-case-specific endpoints that can be embedded into applications, enabling rapid deployment of intelligent features. Akshata highlights the accessibility of these APIs, which come with pricing models and trial options, making them viable for businesses seeking to enhance automation or data processing. She engages the audience by inquiring about their experience with such APIs, emphasizing their growing relevance in modern development.

The second dimension involves leveraging AI to accelerate API development. Akshata describes the API management lifecycle—designing, simulating, publishing, and documenting APIs—as a complex, iterative process. AI tools can simplify these stages, particularly in generating OpenAPI specifications and documentation. She provides an example where a simple prompt to an AI model produces a comprehensive OpenAPI specification for an order management system, streamlining a traditionally time-consuming task. Additionally, AI-driven intelligent document processing can scan invoices or purchase orders, extract relevant fields, and generate REST APIs with GET and POST methods, complete with auto-generated documentation. This approach significantly reduces manual effort and enhances efficiency.

Embedding AI into Custom APIs

The third aspect focuses on embedding AI models, such as large language models (LLMs) or custom co-pilot solutions, into APIs to create sophisticated applications. Akshata showcases Salesforce’s Einstein Assistant, which integrates with OpenAI’s models to process natural language requests. For example, querying “customer details for Mark” triggers an API call that matches the request to predefined actions, retrieves relevant data, and delivers a response. This seamless integration exemplifies how AI can elevate APIs beyond mere data transfer, enabling dynamic, context-aware interactions. Akshata emphasizes that such embeddings allow developers to create tailored solutions that enhance user experiences, such as personalized customer service or automated workflows.

Ethical and Security Considerations

While celebrating the potential of AI-enabled APIs, Akshata candidly addresses their challenges. She underscores the importance of ethical considerations, such as ensuring unbiased AI outputs and protecting user privacy. Security is another critical concern, as integrating AI into APIs introduces vulnerabilities that must be mitigated through robust authentication and data encryption. Akshata’s balanced perspective highlights the need for responsible development practices to maximize benefits while minimizing risks, ensuring that AI-driven solutions remain trustworthy and secure.

Links:

• Salesforce website
• Akshata Sawant on LinkedIn

Mark Needham, from ClickHouse’s product team, and Dunith Danushka, a Senior Developer Advocate at Redpanda, presented an innovative experiment at DevoxxUK2024, showcasing an AI-driven co-pilot for live sports commentary. Inspired by the BBC’s live text commentary for sports like tennis and football, their solution automates repetitive summarization tasks, freeing human commentators to focus on nuanced insights. By integrating Redpanda for streaming, ClickHouse for analytics, and a large language model (LLM) for text generation, they demonstrate a scalable architecture for real-time commentary. Their talk details the technical blueprint, practical implementation, and broader applications, offering a compelling pattern for generative AI in streaming data contexts.

Real-Time Data Streaming with Redpanda

Dunith introduces Redpanda, a Kafka-compatible streaming platform written in C++ to maximize modern hardware efficiency. Unlike Kafka, Redpanda consolidates components like the broker, schema registry, and HTTP proxy into a single binary, simplifying deployment and management. Its web-based console and CLI (rpk) facilitate debugging and administration, such as creating topics and inspecting payloads. In their demo, Mark and Dunith simulate a tennis match by feeding JSON-formatted events into a Redpanda topic named “points.” These events, capturing match details like scores and players, are published at 20x speed using a Python script with the Twisted library. Redpanda’s ability to handle high-throughput streams—hundreds of thousands of messages per second—ensures robust real-time data ingestion, setting the stage for downstream processing.

Analytics with ClickHouse

Mark explains ClickHouse’s role as a column-oriented analytics database optimized for aggregation queries. Unlike row-oriented databases like PostgreSQL, ClickHouse stores columns contiguously, enabling rapid processing of operations like counts or averages. Its vectorized query execution processes column chunks in parallel, enhancing performance for analytics tasks. In the demo, events from Redpanda are ingested into ClickHouse via a Kafka engine table, which mirrors the “points” topic. A materialized view transforms incoming JSON data into a structured table, converting timestamps and storing match metadata. Mark also creates a “matches” table for historical context, demonstrating ClickHouse’s ability to ingest streaming data in real time without batch processing, a key feature for dynamic applications.

Generating Commentary with AI

The core innovation lies in generating human-like commentary using an LLM, specifically OpenAI’s model. Mark and Dunith design a Streamlit-based web application, dubbed the “Live Text Commentary Admin Center,” where commentators can manually input text or trigger AI-generated summaries. The application queries ClickHouse for recent events (e.g., the last minute or game) using SQL, converts results to JSON, and feeds them into the LLM with a prompt instructing it to write concise, present-tense summaries for tennis fans. For example, a query retrieving the last game’s events might yield, “Zverev and Alcaraz slug it out in an epic five-set showdown.” While effective with frontier models like GPT-4, smaller models like Llama 3 struggled, highlighting the need for robust LLMs. The generated text is published to a Redpanda “live_text” topic, enabling flexible consumption.

Broadcasting and Future Applications

To deliver commentary to end users, Mark and Dunith employ Server-Sent Events (SSE) via a FastAPI server, streaming Redpanda’s “live_text” topic to a Streamlit web app. This setup mirrors real-world applications like Wikipedia’s recent changes feed, ensuring low-latency updates. The demo showcases commentary appearing in real time, with potential extensions like tweeting updates or storing them in a data warehouse. Beyond sports, Dunith highlights the architecture’s versatility for domains like live auctions, traffic updates, or food delivery tracking (e.g., Uber Eats notifications). Future enhancements include fine-tuning smaller LLMs, integrating fine-grained statistics via text-to-SQL, or summarizing multiple matches for comprehensive coverage, demonstrating the pattern’s adaptability for real-time generative applications.

Links:

• Redpanda website
• ClickHouse website

The Aspiring Speakers 2024 session at DevoxxUK2024, organized in collaboration with the London Java Community, showcased five emerging talents sharing fresh perspectives on technology and leadership. Rajani Rao explores serverless architectures, Yemurai Rabvukwa bridges chemistry and cybersecurity, Farhath Razzaque delves into AI-driven productivity, Manogna Machiraju tackles imposter syndrome in leadership, and Leena Mooneeram offers strategies for platform team synergy. Each 10-minute talk delivers actionable insights, reflecting the diversity and innovation within the tech community. This session highlights the power of new voices in shaping the future of software development.

Serverless Revolution with Rajani Rao

Rajani Rao, a principal technologist at Viva and founder of the Women Coding Community, presents a compelling case for serverless computing. Using a restaurant analogy—contrasting home cooking (traditional computing) with dining out (serverless)—Rajani illustrates how serverless eliminates infrastructure management, enhances scalability, and optimizes costs. She shares a real-world example of porting a REST API from Windows EC2 instances to AWS Lambda, handling 6 billion monthly requests. This shift, completed in a day, resolved issues like CPU overload and patching failures, freeing the team from maintenance burdens. The result was not only operational efficiency but also a monetized service, boosting revenue and team morale. Rajani advocates starting small with serverless to unlock creativity and improve developer well-being.

Chemistry Meets Cybersecurity with Yemurai Rabvukwa

Yemurai Rabvukwa, a cybersecurity engineer and TikTok content creator under STEM Bab, draws parallels between chemistry and cybersecurity. Her squiggly career path—from studying chemistry in China to pivoting to tech during a COVID-disrupted study abroad—highlights transferable skills like analytical thinking and problem-solving. Yemurai identifies three intersections: pharmaceuticals, healthcare, and energy. In pharmaceuticals, both fields use a prevent-detect-respond framework to safeguard systems and ensure quality. The 2017 WannaCry attack on the NHS underscores a multidisciplinary approach in healthcare, involving stakeholders to restore services. In energy, geopolitical risks and ransomware target renewable sectors, emphasizing cybersecurity’s critical role. Yemurai’s journey inspires leveraging diverse backgrounds to tackle complex tech challenges.

AI-Powered Productivity with Farhath Razzaque

Farhath Razzaque, a freelance full-stack engineer and AI enthusiast, explores how generative AI can transform developer productivity. Quoting DeepMind’s Demis Hassabis, Farhath emphasizes AI’s potential to accelerate innovation. He outlines five levels of AI adoption: zero-shot prompting for quick error resolution, AI apps like Cursor IDE for streamlined coding, prompt engineering for precise outputs, agentic workflows for collaborative AI agents, and custom solutions using frameworks like LangChain. Farhath highlights open-source tools like NoAI Browser and MakeReal, which rival commercial offerings at lower costs. By automating repetitive tasks and leveraging domain expertise, developers can achieve 10x productivity gains, preparing for an AI-driven future.

Overcoming Imposter Syndrome with Manogna Machiraju

Manogna Machiraju, head of engineering at Domestic & General, shares a candid exploration of imposter syndrome in leadership roles. Drawing from her 2017 promotion to engineering manager, Manogna recounts overworking to prove her worth, only to face project failure and team burnout. This prompted reflection on her role’s expectations, realizing she wasn’t meant to code but to enable her team. She advocates building clarity before acting, appreciating team efforts, and embracing tolerable imperfection. Manogna also addresses the challenge of not being the expert in senior roles, encouraging curiosity and authenticity over faking expertise. Her principle—leaning into discomfort with determination—offers a roadmap for navigating leadership doubts.

Platform Happiness with Leena Mooneeram

Leena Mooneeram, a platform engineer at Chainalysis, presents a developer’s guide to platform happiness, emphasizing mutual engagement between engineers and platform teams. Viewing platforms as products, Leena suggests three actions: be an early adopter to shape tools and build relationships, contribute by fixing documentation or small bugs, and question considerately with context and urgency details. These steps enhance platform robustness and reduce friction. For instance, early adopters provide critical feedback, while contributions like PRs for typos streamline workflows. Leena’s mutual engagement model fosters collaboration, ensuring platforms empower engineers to build software joyfully and efficiently.

Links:

• Women Coding Community website
• Chainalysis website

The fusion of artificial intelligence and frontend development is reshaping how we conceive interactive experiences, placing JavaScript engineers at the vanguard of this transformation. Malte Ubl, CTO at Vercel, captivated audiences at dotJS 2024 with a compelling exploration of Generative UI, a pivotal advancement in Vercel’s AI SDK. Originally hailing from Germany and now entrenched in Silicon Valley’s innovation hub, Ubl reflected on the serendipitous echoes of past tech eras—from CGI uploads via FTP to his own contributions like Whiz at Google—before pivoting to AI’s seismic impact. His message was unequivocal: frontend expertise isn’t obsolete in the AI surge; it’s indispensable, empowering developers to craft dynamic, context-aware interfaces that transcend textual exchanges.

Ubl framed the narrative around a paradigm shift from Software 1.0’s laborious machine learning to Software 2.0’s accessible, API-driven intelligence. Where once PhD-level Python tinkering dominated, today’s landscape favors TypeScript applications invoking large language models (LLMs) as services. Models have ballooned in scale and savvy, rendering fine-tuning optional and prompting paramount. This velocity—shipping products in days rather than years—democratizes AI development, yet disrupts traditional roles. Ubl’s optimism stems from a clear positioning: frontend developers as architects of human-AI symbiosis, leveraging React components to ground abstract prompts in tangible interactions.

Central to his demonstration was a conversational airline booking interface, where users query seat changes via natural language. Conventional AI might bombard with options like 14C or 19D, overwhelming without context. Generative UI elevates this: the LLM invokes React server components as functions, streaming a interactive seat map pre-highlighting viable window seats. Users manipulate the UI directly—selecting, visualizing availability—bypassing verbose back-and-forth. Ubl showcased the underlying simplicity: a standard React project with TypeScript files for boarding passes and seat maps, hot-module-reloading enabled, running locally. The magic unfolds via AI functions—React server components that embed client-side state, synced back to the LLM through an “AI state” mechanism. Selecting 19C triggers a callback: “User selected seat 19C,” enabling seamless continuations like checkout flows yielding digital boarding passes.

This isn’t mere novelty; Ubl underscored practical ramifications. End-user chatbots gain depth, support teams wield company-specific components for real-time adjustments, and search engines like the open-source Wary (a Perplexity analog) integrate existing product renderers for enriched results. Accessibility leaps forward too: retrofitting legacy sites with AI state turns static pages into voice-navigable experiences, empowering non-traditional input modalities. Ubl likened AI to a potent backend—API calls fetching not raw data, but rendered intelligence—amplifying human-computer dialogue beyond text. As models from OpenAI, Google Gemini, Anthropic’s Claude, and Mistral proliferate, frontend differentiation via intuitive UIs becomes the competitive edge, uplifting the stack’s user-facing stratum.

Ubl’s closing exhortation: embrace this disruption by viewing React components as AI-native building blocks. Vercel’s AI SDK examples offer starter chatbots primed for customization, accelerating prototyping. In a world where AI smarts escalate, frontend artisans—adept at state orchestration and visual storytelling—emerge as the revolution’s fulcrum, forging empathetic, efficient digital realms.

The Dawn of AI-Infused Interfaces

Ubl vividly contrasted archaic AI chats with generative prowess, using an airline scenario to highlight contextual rendering’s superiority. Prompts yield not lists, but explorable maps—streamed via server components—where selections feed back into the AI loop. This bidirectional flow, powered by AI state, ensures coherence, transforming passive queries into collaborative sessions. Ubl’s live demo, from flight selection to boarding pass issuance, revealed the unobtrusive elegance: plain React, no arcane setups, just LLM-orchestrated functions bridging intent and action.

Empowering Developers in the AI Era

Beyond demos, Ubl advocated for strategic adoption, spotlighting use cases like e-commerce search enhancements and accessibility overlays. Existing components slot into AI workflows effortlessly, while diverse models foster toolkit pluralism. The SDK’s documentation and examples lower barriers, inviting experimentation. Ubl’s thesis: as AI commoditizes logic, frontend’s artistry—crafting modality-agnostic interactions—secures its primacy, heralding an inclusive future where developers orchestrate intelligence with familiar tools.

Links:

• Malte Ubl on LinkedIn
• Vercel

Moderated by Oleg Tsal-Tsalko, Senior Solution Architect at EPAM, the Devoxx Ukraine 2023 panel discussion, AI: Friend or Foe?, brought together experts Evgeny Borisov, Mary Grygleski, Andriy Mulyar, and Sean Phillips to explore the transformative impact of AI on software development and society. The discussion delves into AI’s potential to augment or disrupt, addressing ethical concerns, practical applications, and the skills developers need to thrive in an AI-driven world. This engaging session aligns with the conference’s focus on AI’s role in shaping technology’s future.

AI’s Impact on Software Development

The panel opens with a provocative question: does AI threaten software development jobs? Evgeny and Andriy assert that AI will not replace developers but rather enhance their productivity, acting as a “third arm.” Evgeny notes that many developers, especially juniors, already use tools like ChatGPT alongside their IDEs, streamlining tasks like code generation and documentation lookup. This shift, he argues, allows developers to focus on creative problem-solving rather than rote tasks, making development more engaging and efficient.

Mary reinforces this, suggesting that AI may create new roles, such as prompt engineers, to manage and optimize AI interactions. The panel agrees that while fully autonomous AI agents are still distant, current tools empower developers to deliver higher-quality code faster, transforming the development process into a more strategic and innovative endeavor.

Ethical and Societal Implications

The discussion shifts to AI’s ethical challenges, with Andriy highlighting the risk of “hallucinations”—incorrect or fabricated outputs from LLMs due to incomplete data. Mary adds that unintentional harm, such as misusing generated content, is a significant concern, urging developers to approach AI with caution and responsibility. Sean emphasizes the need for regulation, noting that the lack of oversight could lead to misuse, such as generating misleading content or exploiting personal data.

The panelists stress the importance of transparency, with Evgeny questioning the trustworthiness of AI providers like OpenAI, which may use user inputs to improve their models. This raises concerns about data privacy and intellectual property, prompting a call for developers to be mindful of the tools they use and the data they share.

Educating for an AI-Driven Future

A key theme is the need for broader AI literacy. Andriy advocates for basic machine learning education, even for non-technical users, to demystify AI systems. He suggests resources like MIT’s introductory ML courses to help individuals understand the “black box” of AI, enabling informed interactions. Mary agrees, emphasizing that understanding AI’s implications—without needing deep technical knowledge—can prevent unintended consequences, such as misinterpreting AI outputs.

The panelists encourage developers to learn prompt engineering, as well-formulated prompts significantly improve AI outputs. Evgeny shares that a well-named class or minimal context can yield better results than overly detailed prompts, highlighting the importance of clarity and precision in AI interactions.

Preparing Developers for AI Integration

The panel concludes with practical advice for developers. Sean recommends exploring AI tools to stay competitive, echoing the sentiment that “AI will not replace you, but people using AI will.” Evgeny suggests starting with simple resources, like YouTube tutorials, to master prompt engineering and understand AI capabilities. Mary highlights emerging tools like LangStream, an open-source library for event streaming in RAG patterns, showcasing how AI can integrate with real-time data processing.

The discussion, moderated with skill by Oleg, inspires developers to embrace AI as a collaborative tool while remaining vigilant about its challenges. By fostering education, ethical awareness, and technical proficiency, the panelists envision a future where AI empowers developers to innovate responsibly.

Links:

• EPAM website
• DataStax website
• Nomic AI website
• Oleg Tsal-Tsalko on Twitter
• Evgeny Borisov on Twitter
• Mary Grygleski on Twitter
• Andriy Mulyar on Twitter
• Sean Phillips on Twitter

Hashtags: #AI #SoftwareDevelopment #Ethics #MachineLearning #PromptEngineering #EPAM #DataStax #NomicAI #OlegTsalTsalko #EvgenyBorisov #MaryGrygleski #AndriyMulyar #SeanPhillips #DevoxxUkraine2023