Jonathan Lalou's Blog

Posts Tagged ‘KevinDubois’

Lecturer

Roberto Carratalá is a Principal AI Architect at Red Hat, specializing in container orchestration, AI/ML, and cloud-native platforms. Kevin Dubois is a Senior Principal Developer Advocate at Red Hat, with expertise in improving developer experiences through open-source tools and containerization.

Abstract

This discourse addresses obstacles in maintaining local AI development amid cloud reliance, identifying solutions for offline model execution and code assistance. It explains innovations in local inference tools and model comparisons, framed by desires for autonomy in workflows. Detailing approaches for hardware optimization and framework integrations like Quarkus, it scrutinizes effects on experimentation and privacy. Ramifications for cost-effective innovation and ethical data handling are discussed, guiding sustainable AI practices.

Barriers to Offline AI Workflows

AI’s integration into creation workflows has heightened dependencies on remote services, introducing delays, expenses, and data risks. Developers prefer local environments for mastery over factors like connectivity and setups, but model demands often require clouds.

Roberto and Kevin emphasize local alternatives to preserve independence. Contextually, this counters API costs and quotas, enabling unrestricted trials. Implications: enhanced privacy for proprietary code, vital in secure sectors.

Challenges: hardware constraints limit large models; quantization compresses them for consumer devices. Methodologically, tools like Ollama manage deployments, allowing terminal or IDE interactions.

Deploying and Assessing Local Models

Local deployment uses Ollama for simplicity: installing and running models like Phi-3. Commands:

ollama install phi3
ollama run phi3

Assessment compares sizes: 3.8B Phi-3 versus 70B Llama 3, trading depth for speed. Smaller models run on CPUs, suiting laptops; GPUs accelerate via frameworks.

Code assistants like Continue.dev integrate, configuring for VS Code with local backends. Demos generate Java code, refining via prompts.

For apps, Quarkus with LangChain4j embeds AI. Agents use local models for tasks, code:

AiServices.create(Assistant.class)
    .withChatModel(OllamaChatModel.builder()
        .url("http://localhost:11434")
        .model("phi3")
        .build())
    .withTools(Calculator.class)
    .build();

This enables offline agents. Analysis: smaller models suffice for dev, with tool calls enhancing functionality.

Model Comparisons and Security Considerations

Comparisons: Microsoft’s Phi for compactness, Meta’s Llama for versatility. Quantization (FP32 to INT4) fits 7B models on 8GB RAM.

Assistants: Continue for flexibility, Cursor for editing, but local variants ensure offline use.

Security: reputable sources like Hugging Face prevent malware. Implications: balanced performance-accuracy for local runs.

Enhancing Developer Autonomy and Prospects

Local AI maintains control, reducing barriers. Implications: cost savings, secure trials.

Future: NPUs optimize inference; open models spur community advances.

In essence, local strategies empower efficient AI adoption, merging independence with progress.

Links:

• Lecture video: https://www.youtube.com/watch?v=HeQErLzvnhc
• Roberto Carratalá on LinkedIn: https://es.linkedin.com/in/rcarrata
• Kevin Dubois on LinkedIn: https://ch.linkedin.com/in/kevindubois
• Kevin Dubois on Twitter/X: https://twitter.com/kevin_dubois
• Red Hat website: https://www.redhat.com/

Lecturer

Kevin Dubois is a Senior Principal Developer Advocate at IBM, previously with Red Hat, focusing on Java, AI, and cloud-native development. As a Java Champion and Technical Lead for the CNCF Developer Experience Technical Advisory Group, Kevin authors content, speaks internationally, and contributes to open-source projects. Mario Fusco, co-presenter, is a Senior Principal Software Engineer at IBM (Red Hat), leading the Drools project. A Java Champion with expertise in functional programming and domain-specific languages, Mario coordinates the Milano Java User Group and frequently speaks on software engineering topics. Relevant links include Kevin’s LinkedIn profile (https://ch.linkedin.com/in/kevindubois), Mario’s LinkedIn profile (https://it.linkedin.com/in/mario-fusco-3467213), and Mario’s X account (https://x.com/mariofusco).

Abstract

This article investigates patterns in agentic AI systems as presented by Kevin Dubois and Mario Fusco, emphasizing orchestration of AI services for complex tasks. It delineates foundational components, workflow-based orchestration, autonomous agent models, and extensible planners. Through analysis of methodologies in LangChain4j with Quarkus, it elucidates contexts, implementations, and ramifications for building sophisticated AI applications.

Foundations of AI Services and Agentic Systems

Kevin and Mario initiate their discourse by establishing core elements of AI-infused applications, particularly within Java ecosystems using LangChain4j and Quarkus. An AI service fundamentally interfaces with a large language model (LLM) to process inputs and yield responses. However, effective integration demands more: precise prompting to elicit desired outputs, memory management to sustain conversational context, tool invocation for external actions, and data augmentation via retrieval-augmented generation (RAG).

Prompting emerges as pivotal; vague instructions yield suboptimal results, whereas structured prompts enhance accuracy. Memory, absent in standalone LLMs, requires client-side tracking—LangChain4j automates this, customizable via caching. Tools enable LLMs to perform actions like database queries or email dispatch, via function calling where LLMs request tool usage.

RAG integrates proprietary data: embeddings store vectorized information in databases like Pinecone, retrieved to enrich prompts. Moderation filters harmful content, ensuring ethical outputs.

Agentic systems extend this: agents, autonomous entities with goals, leverage these components. Patterns categorize into workflows (predefined paths) and autonomous agents (dynamic LLM-directed processes). Contexts include scenarios needing multi-step reasoning, like trip planning involving weather, flights, and accommodations.

Implications: These foundations enable modular, scalable AI, but demand careful design to mitigate errors like hallucinations.

Code illustrates basics:

@RegisterAiService
interface WeatherAgent {
    String getWeather(String city);
}

This defines an AI service interfacing with an LLM for weather queries.

Workflow-Based Orchestration of Agents

Workflow patterns orchestrate agents through coded sequences, suitable for predictable tasks. Kevin and Mario detail sequential, parallel, conditional, and looping workflows in LangChain4j.

Sequential invokes agents in order: e.g., weather retrieval followed by outfit suggestion. Parallel executes concurrently, aggregating outputs—useful for independent subtasks like multi-city weather checks.

Conditional branches based on outputs: if weather is rainy, suggest indoor activities. Looping iterates until conditions met, like refining content via reviewer-critic cycles.

Methodology employs builders:

AgenticSystem system = AgenticSystem.builder()
    .sequence(weatherAgent, outfitAgent)
    .build();

Execution yields structured results, with event logs for monitoring.

Contexts: Workflows suit deterministic processes, reducing LLM variability. Implications: Enhance efficiency but limit adaptability; error handling via retries or prompt adjustments is crucial.

Autonomous and Dynamic Agent Orchestration

Autonomous patterns empower an LLM-orchestrator to dynamically select agents, ideal for unstructured tasks. The orchestrator evaluates inputs, plans invocations, and executes, adapting via reasoning.

Mario explains: Orchestrator prompts guide tool (agent) selection. Execution involves planning, tool calls, and result integration until resolution.

AgenticSystem system = AgenticSystem.builder()
    .autonomous(orchestrator)
    .agents(agent1, agent2)
    .build();

Contexts: Handles ambiguity, like open-ended queries. Implications: Increases flexibility but risks infinite loops or off-track reasoning; human-in-the-loop mitigates via approvals.

Multimodal extensions process PDFs or generate images, expanding applicability.

Extensible Planners for Custom Agentic Patterns

To accommodate diverse needs, Mario introduces pluggable planners, abstracting orchestration. This service provider interface (SPI) allows custom implementations, like goal-oriented patterns using A* search.

Planners initialize with agents, determining next actions: invoke agents (sequentially/parallel) or conclude. Existing patterns refactor atop this.

Goal-oriented example: Define prerequisites and goals; algorithm generates invocation graphs.

Planner customPlanner = new GoalOrientedPlanner(agents);
AgenticSystem system = AgenticSystem.builder()
    .planner(() -> customPlanner)
    .build();

Hybridization combines patterns, e.g., goal-oriented with loops for refinement.

Contexts: Custom scenarios like adaptive learning systems. Implications: Fosters innovation, but requires algorithmic expertise; promotes modularity in AI design.

In summary, Kevin and Mario’s patterns advance agentic AI, blending structure with dynamism for robust applications.

Links:

• Lecture Video
• Kevin Dubois on LinkedIn
• Mario Fusco on LinkedIn
• Mario Fusco on X