Posts Tagged ‘LangChain’
[VoxxedDaysTicino2026] Securing RAG Pipelines with Fine-Grained Authorization
Lecturer
Sohan Maheshwar is a Lead Developer Advocate at AuthZed, based in the Netherlands. With over 15 years in the IT industry, including a decade in developer relations, Sohan began his career developing mobile applications and has since focused on emerging technologies, developer ecosystems, and authorization systems. He frequently speaks on AI security and access control. Relevant links include his personal website (https://www.sohan.co/) and LinkedIn profile (https://www.linkedin.com/in/sohan-maheshwar/).
Abstract
This article scrutinizes Sohan Maheshwar’s examination of securing retrieval-augmented generation (RAG) pipelines using relationship-based access control (ReBAC). It elucidates authorization paradigms, Zanzibar’s influence, and practical implementations with tools like Pinecone, LangChain, OpenAI, and SpiceDB. Through contextual analysis, methodological demos, and audience interactions, it assesses efficiency, security enhancements, and enterprise implications.
Evolution of Authorization Models and ReBAC’s Role
Sohan delineates authorization from authentication: the former governs access post-identity verification. Historical approaches like access control lists (ACLs)—enumerating users per resource—prove unscalable. Role-based access control (RBAC) assigns permissions via roles (e.g., software engineer accessing code), but reorgs necessitate new roles, complicating management.
Attribute-based access control (ABAC) evaluates attributes (location, IP) at decision points, yet scales poorly with complexity. ReBAC, popularized by Google’s 2019 Zanzibar paper, expresses permissions as relationships between users and objects (e.g., viewer, editor).
Zanzibar, handling 10 million queries per second in 2019, underpins Google services. Relationships form graphs: users connect to objects via relations like “member” or “owner.” Tuples formalize these: “user:3#owner@document:123.” Objects relate to others, enabling hierarchies.
Contexts: Inspired by Google+’s circles, ReBAC facilitates dynamic permissions, easing reorgs by rewriting relationships. Implications: Graphs enable efficient traversals, surpassing list-based models for scale.
Integrating ReBAC into RAG Pipelines
RAG augments LLMs with external data: queries retrieve vectors from databases, enriching prompts for accurate responses. Security demands users access only authorized data, preventing leaks.
Sohan advocates ReBAC for fine-grained control, superior to RBAC/ABAC in relational precision. Zanzibar schemas define relations (e.g., document viewers as owners or parent viewers), supporting unions, intersections for complex policies.
In RAG, pre-filtering queries with authorized IDs enhances efficiency: SpiceDB, an open-source Zanzibar implementation, checks permissions, filtering vectors before retrieval. Post-filtering verifies results, though less efficient.
Demo methodology: Using Pinecone for vectors, LangChain for orchestration, OpenAI for LLMs, and SpiceDB for authorization. Documents store metadata like IDs; SpiceDB relationships (e.g., “user:sohan#viewer@document:hr_policy”) enable checks.
Code integrates:
authorized_docs = spicedb.check_permissions(user, "viewer", documents)
filtered_query = vector_db.query_with_filter(authorized_docs)
Contexts: Enterprises risk breaches without such controls. Implications: Pre-filtering scales better for large datasets, though metadata in vectors can bloat if not optimized via external graphs.
Practical Demonstrations and Error Handling
Sohan’s live demo simulates an HR chatbot: Unsecured, it leaks sensitive data (e.g., salaries). With ReBAC, queries filter to authorized documents (vacation policies for non-HR users).
Pre-filtering: SpiceDB lists permitted IDs, constraining vector queries. Post-filtering: Retrieves broadly, then filters results—viable for small sets but inefficient at scale.
Audience queries highlight extensions: Subset permissions for agents via schema intersections/minuses. Hallucinations mitigated by prompt engineering (“reply only from these documents”) or fail-closed policies (deny access on errors).
Prompt engineering can’t bypass filters, as unauthorized data never reaches the LLM.
Contexts: Multi-agent scenarios require nuanced schemas reflecting privileges. Implications: Schemas model complexities (e.g., GitHub’s owners/repositories), but agent identities remain unresolved—user-linked or independent?
Enterprise Implications and Future Considerations
ReBAC safeguards RAG by centralizing permissions, akin to Zanzibar’s cross-app checks (e.g., Google Docs in Gmail). SpiceDB, handling billions of relationships, offers caching, watch APIs for reactivity.
Implications: Enhances security/efficiency, but demands schema expertise. At scale, vector metadata for users falters; graphs excel. Enterprises must address agent authorization—subsets prevent overreach.
Future: Industry evolves toward agent-specific identities, avoiding user dependency. Tools like Kubernetes controllers leveraging SpiceDB (via operators) indicate broader adoption.
In essence, Sohan’s approach fortifies AI pipelines, balancing access with protection in relational contexts.
Links:
[DevoxxGR2025] Simplifying LLM Integration: A Blueprint for Effective AI Systems
Efstratios Marinos captivated attendees at Devoxx Greece 2025 with a masterclass on streamlining large language model (LLM) integrations. By focusing on practical, modular patterns, Efstratios demonstrated how to construct robust, scalable AI systems that prioritize simplicity without sacrificing functionality, offering actionable strategies for developers.
Exploring the Complexity Continuum
Efstratios introduced the concept of a complexity continuum for LLM integrations, spanning from straightforward single calls to sophisticated agentic frameworks. At its simplest, a system comprises an LLM, a retrieval mechanism, and tool capabilities, delivering maintainability and ease of updates with minimal overhead. More intricate setups incorporate routers, APIs, and vector stores, enhancing functionality but complicating debugging. Efstratios emphasized that simplicity is a strategic choice, enabling rapid adaptation to evolving AI technologies. He showcased a concise Python implementation, where a single function manages retrieval and response generation in a handful of lines, contrasting this with a multi-step retrieval-augmented generation (RAG) workflow that involves encoding, indexing, and embedding, adding layers of complexity that demand careful justification.
Crafting Robust Interfaces
Central to Efstratios’s philosophy is the design of clean interfaces for LLMs, retrieval systems, tools, and memory components. He compared prompt crafting to API design, advocating for structured formats that clearly separate instructions, context, and queries. Well-documented tools, complete with detailed descriptions and practical examples, empower LLMs to perform effectively, while vague documentation leads to errors. Efstratios underscored the need for resilient error handling, such as fallback strategies for failed retrievals or tool invocations, to ensure system reliability. For example, a system might respond to a failed search by suggesting alternatives or retrying with adjusted parameters, improving usability and simplifying troubleshooting in production environments.
Enhancing Capabilities with Workflow Patterns
Efstratios explored three foundational workflow patterns—prompt chaining, routing, and parallelization—to optimize performance while managing complexity. Prompt chaining divides complex tasks into sequential steps, such as outlining, drafting, and refining content, enhancing clarity at the expense of increased latency. Routing employs an LLM to categorize inputs and direct them to specialized handlers, like a customer support bot distinguishing technical from financial queries, improving efficiency through focused processing. Parallelization, encompassing sectioning and voting, distributes tasks across multiple LLM instances, such as analyzing document segments concurrently, though it incurs higher computational costs. These patterns provide incremental enhancements, ideal for tasks requiring moderate sophistication.
Advanced Patterns and Decision-Making Principles
For more demanding scenarios, Efstratios presented two advanced patterns: orchestrator-workers and evaluator-optimizer. The orchestrator-workers pattern dynamically breaks down tasks, with a central LLM coordinating specialized workers, perfect for complex coding projects or multi-faceted content creation. The evaluator-optimizer pattern establishes a feedback loop, where a generator LLM produces content and an evaluator refines it iteratively, mirroring human iterative processes. Efstratios outlined six decision-making principles—use case alignment, development effort, maintainability, performance granularity, latency, and cost—to guide pattern selection. Simple solutions suffice for tasks like summarization, while multi-step workflows excel in knowledge-intensive applications. He encouraged starting with minimal solutions, establishing performance baselines, identifying specific limitations, and adding complexity only when validated by measurable gains.
Links:
[DevoxxGR2024] Meet Your New AI Best Friend: LangChain at Devoxx Greece 2024 by Henry Lagarde
At Devoxx Greece 2024, Henry Lagarde, a senior software engineer at Criteo, introduced audiences to LangChain, a versatile framework for building AI-powered applications. With infectious enthusiasm and live demonstrations, Henry showcased how LangChain simplifies interactions with large language models (LLMs), enabling developers to create context-aware, reasoning-driven tools. His talk, rooted in his experience at Criteo, a leader in retargeting and retail media, highlighted LangChain’s composability and community-driven evolution, offering a practical guide for AI integration.
LangChain’s Ecosystem and Composability
Henry began by defining LangChain as a framework for building context-aware reasoning applications. Unlike traditional LLM integrations, LangChain provides modular components—prompt templates, LLM abstractions, vector stores, text splitters, and document loaders—that integrate with external services rather than hosting them. This composability allows developers to switch LLMs seamlessly, adapting to changes in cost or performance without rewriting code. Henry emphasized LangChain’s open-source roots, launched in late 2022, and its rapid growth, with versions in Python, TypeScript, Java, and more, earning it the 2023 New Tool of the Year award.
The ecosystem extends beyond core modules to include LangServe for REST API deployment, LangSmith for monitoring, and a community hub for sharing prompts and agents. This holistic approach supports developers from prototyping to production, making LangChain a cornerstone for AI engineering.
Building a Chat Application
In a live demo, Henry showcased LangChain’s simplicity by recreating a ChatGPT-like application in under 10 lines of Python code. He instantiated an OpenAI client using GPT-3.5 Turbo, implemented chat history for context awareness, and used prompt templates to define system and human messages. By combining these components, he enabled streaming responses, mimicking ChatGPT’s real-time output without the $20 monthly subscription. This demonstration highlighted LangChain’s ability to handle memory, input/output formatting, and LLM interactions with minimal effort, empowering developers to build cost-effective alternatives.
Henry noted that LangChain’s abstractions, such as strong typing and output parsing, eliminate manual prompt engineering, ensuring robust integrations even when APIs change. The demo underscored the framework’s accessibility, inviting developers to experiment with its capabilities.
Creating an AI Agent for PowerPoint Generation
Henry’s second demo illustrated LangChain’s advanced features by building an AI agent to generate PowerPoint presentations. Using TypeScript, he configured a system prompt from LangSmith’s community hub, defining the agent’s tasks: researching a topic via the Serper API and generating a structured PowerPoint. He defined tools with Zod for runtime type checking, ensuring consistent outputs, and integrated callbacks for UI tracing and monitoring.
The agent, powered by Anthropic’s Claude model, performed internet research on Google Cloud, compiled findings, and generated a presentation with sourced information. Despite minor delays, the demo showcased LangChain’s ability to orchestrate complex workflows, combining research, data processing, and content creation. Henry’s use of LangSmith for prompt optimization and monitoring highlighted the framework’s production-ready capabilities.
Community and Cautions
Henry emphasized LangChain’s vibrant community, which drives its multi-language support and rapid evolution. He encouraged attendees to contribute, noting the framework’s open-source ethos and resources like GitHub for further exploration. However, he cautioned against over-reliance on LLMs, citing their occasional laziness or errors, as seen in ChatGPT’s simplistic responses. LangChain, he argued, augments developer workflows but requires careful integration to ensure reliability in production environments.
His vision for LangChain is one of empowerment, enabling developers to enhance applications incrementally while maintaining control over AI-driven processes. By sharing his demo code on GitHub, Henry invited attendees to experiment and contribute to LangChain’s growth.
Conclusion
Henry’s presentation at Devoxx Greece 2024 was a compelling introduction to LangChain’s potential. Through practical demos and insightful commentary, he demonstrated how the framework simplifies AI development, from basic chat applications to sophisticated agents. His emphasis on composability, community, and cautious integration resonated with developers eager to explore AI. As LangChain continues to evolve, Henry’s talk serves as a blueprint for harnessing its capabilities in real-world applications.
Links:
[DevoxxBE2023] Making Your @Beans Intelligent: Spring AI Innovations
At DevoxxBE2023, Dr. Mark Pollack delivered an insightful presentation on integrating artificial intelligence into Java applications using Spring AI, a project inspired by advancements in AI frameworks like LangChain and LlamaIndex. Mark, a seasoned Spring developer since 2003 and leader of the Spring Data project, explored how Java developers can harness pre-trained AI models to create intelligent applications that address real-world challenges. His talk introduced the audience to Spring AI’s capabilities, from simple “Hello World” examples to sophisticated use cases like question-and-answer systems over custom documents.
The Genesis of Spring AI
Mark began by sharing his journey into AI, sparked by the transformative impact of ChatGPT. Unlike traditional AI development, which often required extensive data cleaning and model training, pre-trained models like those from OpenAI offer accessible APIs and vast knowledge bases, enabling developers to focus on application engineering rather than data science. Mark highlighted how Spring AI emerged from his exploration of code generation, leveraging the structured nature of code within these models to create a framework tailored for Java developers. This framework abstracts the complexity of AI model interactions, making it easier to integrate AI into Spring-based applications.
Spring AI draws inspiration from Python’s AI ecosystem but adapts these concepts to Java’s idioms, emphasizing component abstractions and pluggability. Mark emphasized that this is not a direct port but a reimagination, aligning with the Spring ecosystem’s strengths in enterprise integration and batch processing. This approach positions Spring AI as a bridge between Java’s robust software engineering practices and the dynamic world of AI.
Core Components of AI Applications
A significant portion of Mark’s presentation focused on the architecture of AI applications, which extends beyond merely calling a model. He introduced a conceptual framework involving contextual data, AI frameworks, and models. Contextual data, akin to ETL (Extract, Transform, Load) processes, involves parsing and transforming data—such as PDFs—into embeddings stored in vector databases. These embeddings enable efficient similarity searches, crucial for use cases like question-and-answer systems.
Mark demonstrated a simple AI client in Spring AI, which abstracts interactions with various AI models, including OpenAI, Hugging Face, Amazon Bedrock, and Google Vertex. This portability allows developers to switch models without significant code changes. He also showcased the Spring CLI, a tool inspired by JavaScript’s Create React App, which simplifies project setup by generating starter code from existing repositories.
Prompt Engineering and Its Importance
Prompt engineering emerged as a critical theme in Mark’s talk. He explained that crafting effective prompts is essential for directing AI models to produce desired outputs, such as JSON-formatted responses or specific styles of answers. Spring AI’s PromptTemplate class facilitates this by allowing developers to create reusable, stateful templates with placeholders for dynamic content. Mark illustrated this with a demo where a prompt template generated a joke about a raccoon, highlighting the importance of roles (system and user) in defining the context and tone of AI responses.
He also touched on the concept of “dogfooding,” where AI models are used to refine prompts, creating a feedback loop that enhances their effectiveness. This iterative process, combined with evaluation techniques, ensures that applications deliver accurate and relevant responses, addressing challenges like model hallucinations—where AI generates plausible but incorrect information.
Retrieval Augmented Generation (RAG)
Mark introduced Retrieval Augmented Generation (RAG), a technique to overcome the limitations of AI models’ context windows, which restrict the amount of data they can process. RAG involves pre-processing data into smaller fragments, converting them into embeddings, and storing them in vector databases for similarity searches. This approach allows developers to provide only relevant data to the model, improving efficiency and accuracy.
In a demo, Mark showcased RAG with a bicycle shop dataset, where a question about city-commuting bikes retrieved relevant product descriptions from a vector store. This process mirrors traditional search engines but leverages AI to synthesize answers, demonstrating how Spring AI integrates with vector databases like Milvus and PostgreSQL to handle complex queries.
Real-World Applications and Future Directions
Mark highlighted practical applications of Spring AI, such as enabling question-and-answer systems for financial documents, medical records, or government programs like Medicaid. These use cases illustrate AI’s potential to make complex information more accessible, particularly for non-technical users. He also discussed the importance of evaluation in AI development, advocating for automated scoring mechanisms to assess response quality beyond simple test passing.
Looking forward, Mark outlined Spring AI’s roadmap, emphasizing robust core abstractions and support for a growing number of models and vector databases. He encouraged developers to explore the project’s GitHub repository and participate in its evolution, underscoring the rapid pace of AI advancements and the need for community involvement.