Jonathan Lalou's Blog

Posts Tagged ‘Performance’

Kévin Dunglas, a seasoned developer at Les-Tilleuls.coop and creator of API Platform, presented an innovative exploration of FrankenPHP at PHP Forum Paris 2022. Blending PHP with Go, Kévin introduced a groundbreaking server solution that pushes PHP’s boundaries. His talk delved into the technical intricacies of integrating Go’s threading model with PHP’s interpreter, offering a glimpse into a future where PHP applications achieve unprecedented performance and flexibility.

Introducing FrankenPHP

Kévin opened with the origins of FrankenPHP, a project born from his passion for both PHP and Go. Inspired by Les-Tilleuls’ developer Loris Sorio, who designed its logo, FrankenPHP aims to combine PHP’s ease of use with Go’s performance capabilities. Kévin explained how it leverages Go’s threading to overcome PHP-FPM’s limitations, enabling features like concurrent request handling. This fusion, he argued, unlocks new possibilities for PHP applications, particularly in high-performance scenarios.

Overcoming Technical Challenges

Delving into the technical core, Kévin described the complexities of integrating PHP’s Zend Thread Safe (ZTS) mode with Go’s threading model. He highlighted challenges like signal conflicts and the lack of OPcache support, which required custom modifications to PHP’s source code. By isolating PHP processes within Go threads, Kévin’s team achieved stable communication, though he noted the solution remains experimental. His transparency about these hurdles provided valuable insights for developers exploring similar integrations.

Performance and Future Directions

Kévin showcased FrankenPHP’s performance potential, demonstrating how enabling OPcache by modifying PHP’s SAPI list significantly reduced compilation overhead. He outlined future goals, including support for Laravel Octane and Symfony’s CLI, while acknowledging Windows compatibility challenges. Kévin’s call for community contributions to refine FrankenPHP underscored its open-source ethos, inviting developers to explore its code and report issues to enhance its stability.

Community Engagement and Collaboration

Concluding, Kévin emphasized the collaborative spirit driving FrankenPHP’s development. He encouraged developers to contribute via GitHub, highlighting the project’s experimental nature and potential for growth. By sharing Les-Tilleuls’ vision, Kévin inspired attendees to experiment with FrankenPHP, fostering a community-driven effort to redefine PHP’s role in modern web development.

Links:

• Les-Tilleuls.coop company website
• API Platform website
• FrankenPHP on GitHub

Lecturer

Vladimir Dejanović is a seasoned software professional with over a decade of experience in the IT industry, having contributed to diverse projects since 2006. As Senior Director of B2C Technology at PVH, a fashion technology firm overseeing brands like Tommy Hilfiger and Calvin Klein, he focuses on scalable systems and innovative solutions. Beyond his corporate role, Vladimir founded and leads the Amsterdam Java User Group, fostering community engagement in Java technologies. He is recognized as an Oracle Code One Star and Java Rockstar, frequently delivering presentations at international conferences on topics like GraphQL and Java development.

Abstract

This article delves into the intricacies of GraphQL as applied within Java environments, examining its specification, implementation strategies, and practical applications through code demonstrations. It analyzes the advantages of GraphQL over traditional REST APIs, such as enhanced query flexibility and schema validation, while addressing potential pitfalls like cyclic dependencies and security concerns. Drawing from real-world examples, the discussion highlights methodologies for schema design, resolver integration, and performance optimization, underscoring GraphQL’s role in modern API development and its implications for system architecture.

Understanding GraphQL: Beyond the Basics

GraphQL emerges as a pivotal specification in API design, originating from Facebook in 2015 to address inefficiencies in data fetching encountered during mobile application development. Unlike conventional REST APIs, which often result in over-fetching or under-fetching of data, GraphQL empowers clients to request precisely the information needed, thereby optimizing network usage and enhancing performance. The specification defines a query language that allows for declarative data retrieval, where clients specify the structure of the response, aligning closely with application requirements.

At its core, GraphQL is not a full-fledged framework but a set of guidelines that various languages implement differently. In Java, implementations like GraphQL Java provide the engine for processing queries, while tools such as GraphQL Java Kickstarters facilitate integration with existing infrastructures, such as Spring Boot. This flexibility means developers must be cognizant of implementation-specific nuances, including coverage of the specification and additional features not mandated by the core rules. For instance, while the specification mandates schema validation, implementations may vary in handling extensions like custom scalars or error propagation.

The schema definition language (SDL) stands out as GraphQL’s most potent feature, surpassing alternatives like OpenAPI in expressiveness. It requires a mandatory schema that describes types, fields, and relationships, ensuring both client and server adhere to a contract. Upon connection, the server transmits the schema, enabling clients to validate requests locally before transmission, which reduces invalid traffic and conserves resources. This schema-first approach, preferred for its mockability, allows teams to prototype APIs independently: backend developers define the schema, while frontend teams use mocks to simulate responses.

Consider a practical scenario involving a conference application with entities like attendees, speakers, and talks. The schema might define types as follows:

type Attendee {
  id: ID!
  name: String
}

type Speaker {
  id: ID!
  name: String
  twitter: String
}

type Talk {
  id: ID!
  title: String
  description: String
  speakers: [Speaker]
}

Here, the exclamation mark denotes mandatory fields, and arrays indicate relationships. This structure not only documents the API but also enforces consistency, preventing outdated documentation—a common issue in REST environments.

Implementing Queries and Resolvers in Java

Transitioning to code, integrating GraphQL in Java involves wiring the schema to business logic. Using Spring Boot and GraphQL Java, one initializes a servlet mapped to “/graphql”, parsing the schema and registering resolvers. Resolvers act as the bridge, implementing interfaces like GraphQLQueryResolver for read operations.

For the conference example, a Query class might look like this:

@Component
public class Query implements GraphQLQueryResolver {
  private final TalkService talkService;
  private final SpeakerService speakerService;
  private final AttendeeService attendeeService;

  @RequiredArgsConstructor
  public Query(TalkService talkService, SpeakerService speakerService, AttendeeService attendeeService) {
    this.talkService = talkService;
    this.speakerService = speakerService;
    this.attendeeService = attendeeService;
  }

  public List<Talk> allTalks() {
    return talkService.findAll();
  }

  public List<Speaker> allSpeakers() {
    return speakerService.findAll();
  }

  public List<Attendee> allAttendees() {
    return attendeeService.findAll();
  }
}

This setup enables queries like fetching all talks with specific fields:

query {
  allTalks {
    id
    title
    description
    speakers {
      name
      twitter
    }
  }
}

The response mirrors the query structure in JSON, promoting predictability. Clients can alias fields (e.g., renaming “title” to “myTitle”) or conditionally include them using directives like @include(if: $variable), where variables are passed separately for dynamic behavior.

Resolvers for relationships, such as linking talks to speakers, extend GraphQLResolver:

@Component
public class TalkResolver implements GraphQLResolver<Talk> {
  private final SpeakerService speakerService;

  @RequiredArgsConstructor
  public TalkResolver(SpeakerService speakerService) {
    this.speakerService = speakerService;
  }

  public List<Speaker> speakers(Talk talk) {
    return speakerService.findAllSpeakersForTalk(talk);
  }
}

This modular approach allows for granular control, but it introduces risks like cyclic queries if bidirectional links (e.g., speakers to talks) are added without safeguards. Such cycles can lead to infinite loops, necessitating depth limits during validation.

Mutations: Enabling Data Modification

While queries handle reads, mutations facilitate writes, mirroring CRUD operations but with GraphQL’s precision. Defined similarly in the schema:

type Mutation {
  addTalk(input: TalkInput!): Talk
}

Mutations require explicit input types to encapsulate parameters, ensuring type safety. In Java, a Mutation resolver implements GraphQLMutationResolver:

@Component
public class Mutation implements GraphQLMutationResolver {
  private final TalkService talkService;

  @RequiredArgsConstructor
  public Mutation(TalkService talkService) {
    this.talkService = talkService;
  }

  public Talk addTalk(TalkInput input) {
    // Logic to create and persist talk
    return talkService.save(input.toTalk());
  }
}

This method processes inputs, validates them, and returns the updated entity. Unlike queries, which can execute in parallel for optimization, mutations are sequential to maintain data integrity. Errors in mutations propagate similarly to queries, with customizable handlers to continue processing or halt execution.

The implications are profound: mutations reduce boilerplate compared to REST’s multiple endpoints, centralizing logic while allowing clients to request related data in the same response, such as fetching the newly added talk’s speakers.

Subscriptions: Real-Time Data Streams

Subscriptions introduce reactive capabilities, enabling server-push updates over WebSockets. The schema defines:

type Subscription {
  scores(title: String!): Score
}

type Score {
  title: String
  score: Int
}

In Java, using Reactor for reactivity:

@Component
public class Subscription implements GraphQLSubscriptionResolver {
  public Publisher<Score> scores(String title) {
    return Flux.interval(Duration.ofSeconds(2))
      .map(i -> Score.builder()
        .title(title)
        .score(ThreadLocalRandom.current().nextInt(1, 6))
        .build());
  }
}

This generates scores every two seconds, demonstrating backpressure handling to prevent client overload. Subscriptions transform static APIs into dynamic ones, ideal for live updates like conference feedback, though implementation varies—GraphQL Java uses WebSockets, but the specification leaves transport open.

Security and Performance Considerations

Security in GraphQL demands vigilance, as the schema’s public nature exposes potential attack vectors. Authentication and authorization occur via custom contexts:

public class MyGraphQLContext extends GraphQLContext {
  private final User user;

  public MyGraphQLContext(User user) {
    this.user = user;
  }
}

Resolvers access this context via DataFetchingEnvironment to enforce roles. For schema protection, directives or instrumentation filter visibility, though non-standard approaches risk interoperability.

Performance pitfalls include N+1 queries, mitigated by batching or caching in services. Instrumentation traces execution:

public class TracingInstrumentation extends SimpleInstrumentation {
  @Override
  public InstrumentationContext<ExecutionResult> beginExecution(InstrumentationExecutionParameters parameters) {
    // Start timer, log query
    return super.beginExecution(parameters);
  }
}

Query complexity analysis during validation prevents denial-of-service attacks by capping depth or computational cost.

Schema management in large systems involves stitching or extensions to avoid monolithic files:

extend type Speaker {
  twitter: String
}

This federates schemas across services, though conflicts require governance.

Implications for Modern Development

GraphQL’s client-centric model shifts power from servers, fostering agile development but requiring robust safeguards. In Java, its integration with Spring Boot streamlines adoption, yet demands awareness of implementation variances. By enabling precise data fetching and real-time interactions, it addresses REST’s limitations, promoting efficient, scalable architectures. Future specification enhancements, like improved subscription standards, promise broader applicability.

Links:

• Lecture Video

Lecturer

Eric Cochran is an Android developer at Pinterest, focusing on performance across the app stack. He contributes to open-source projects, notably the Moshi JSON library. Relevant links: Pinterest Engineering Blog (publications); LinkedIn Profile (professional page).

Abstract

This article analyzes Eric Cochran’s exploration of Kotlin Serialization for multiplatform projects, emphasizing its role in enhancing code reuse across platforms. Set in the context of Pinterest’s performance-driven Android development, it examines methodologies for integrating serialization with data formats and frameworks. The analysis highlights innovations in type safety and performance, with implications for cross-platform scalability and library evolution.

Introduction and Context

Eric Cochran presented at KotlinConf 2018, focusing on Kotlin Serialization’s potential to unify code in multiplatform environments. As an Android developer at Pinterest, Cochran’s work on serialization formats like Moshi informed his advocacy for Kotlin’s experimental library. The context is the growing need for shared logic in apps targeting JVM, JS, and Native, where serialization ensures seamless data handling across diverse runtimes.

Methodological Approaches to Serialization

Cochran outlined Kotlin Serialization’s setup: Annotate data classes with @Serializable to generate compile-time adapters, supporting JSON, Protobuf, and CBOR. Integration with frameworks like OkHttp or Ktor involves custom serializers for complex types. He demonstrated parsing dynamic JSON structures, emphasizing compile-time safety over Moshi’s runtime reflection. Performance optimizations included minimizing allocations and leveraging inline classes. Cochran compared Moshi’s factory-based API, noting its JVM-centric limitations versus Kotlin Serialization’s multiplatform readiness.

Analysis of Innovations and Features

Kotlin Serialization innovates with compile-time code generation, avoiding reflection’s overhead, unlike Moshi’s Java type reliance. It supports multiple formats, enhancing flexibility compared to JSON-centric libraries. Inline classes reduce boxing, boosting performance. Limitations include poor dynamic type handling and manual serializer implementation for custom cases. Compared to Moshi, it offers broader platform support but lacks mature metadata APIs.

Implications and Consequences

The library implies greater code sharing in multiplatform apps, reducing duplication and maintenance. Its performance focus suits high-throughput systems like Pinterest’s. Consequences include a shift toward compile-time solutions, though experimental status requires caution. Future integration with Okio’s multiplatform efforts could resolve reflection issues, broadening adoption.

Conclusion

Cochran’s insights position Kotlin Serialization as a cornerstone for multiplatform data handling, offering a performant, type-safe alternative that promises to reshape cross-platform development.

Links

• Lecture video: https://www.youtube.com/watch?v=p8Wt_atMA50
• Lecturer’s X/Twitter: @ericcochran
• Lecturer’s LinkedIn: Eric Cochran
• Organization’s X/Twitter: @PinterestEng
• Organization’s LinkedIn: Pinterest

Lecturers

Mathieu Pouymerol and Pierre Baillet were the technical backbone of Fotopedia, a photo-sharing platform that, at its peak, served over five million monthly visitors using a Ruby on Rails application that had been in production for six years. Mathieu, armed with degrees from École Centrale Paris and a background in building custom data stores for dictionary publishers, brought a deep understanding of database design, indexing, and performance optimization. Pierre, also from Centrale and with experience at Cambridge, had spent nearly a decade managing infrastructure, tuning Tomcat, configuring memcached, and implementing geoDNS systems. Together, they faced the ultimate challenge: keeping a legacy Rails monolith responsive under massive, unpredictable traffic while maintaining content freshness and developer velocity.

Abstract

This article presents an exhaustively detailed expansion of Mathieu Pouymerol and Pierre Baillet’s 2012 DevoxxFR presentation, “MongoDB et Mustache, vers la mort du cache ?”, reimagined as a definitive case study in high-traffic web architecture and the evolution of caching strategies. The Fotopedia team inherited a Rails application plagued by slow ORM queries, complex cache invalidation logic, and frequent stale data. Their initial response—edge-side includes (ESI), fragment caching, and multi-layered memcached—bought time but introduced fragility and operational overhead. The breakthrough came from a radical rethinking: use MongoDB as a real-time document store and Mustache as a logic-less templating engine to assemble pages dynamically, eliminating cache for the most volatile content.

This analysis walks through every layer of their architecture: from database schema design to template composition, from CDN integration to failure mode handling. It includes performance metrics, post-mortem analyses, and lessons learned from production incidents. Updated for 2025, it maps their approach to modern tools: MongoDB 7.0 with Atlas, server-side rendering with HTMX, edge computing via Cloudflare Workers, and Spring Boot with Mustache, offering a complete playbook for building cache-minimized, real-time web applications at scale.

The Legacy Burden: A Rails Monolith Under Siege

Fotopedia’s core application was built on Ruby on Rails 2.3, a framework that, while productive for startups, began to show its age under heavy load. The database layer relied on MySQL with aggressive sharding and replication, but ActiveRecord queries were slow, and joins across shards were impractical. The presentation layer used ER 15–20 partials per page, each with its own caching logic. The result was a cache dependency graph so complex that a single user action—liking a photo—could invalidate dozens of cache keys across multiple servers.

The team’s initial strategy was defense in depth:
– Varnish at the edge with ESI for including dynamic fragments.
– Memcached for fragment and row-level caching.
– Custom invalidation daemons to purge stale cache entries.

But this created a house of cards. A missed invalidation led to stale comments. A cache stampede during a traffic spike brought the database to its knees. As Pierre put it, “We were not caching to improve performance. We were caching to survive.”

The Paradigm Shift: Real-Time Data with MongoDB

The turning point came when the team migrated dynamic, user-generated content—photos, comments, tags, likes—to MongoDB. Unlike MySQL, MongoDB stored data as flexible JSON-like documents, allowing embedded arrays and atomic updates:

{
  "_id": "photo_123",
  "title": "Sunset",
  "user_id": "user_456",
  "tags": ["paris", "sunset"],
  "likes": 1234,
  "comments": [
    { "user": "Alice", "text": "Gorgeous!", "timestamp": "2013-04-01T12:00:00Z" }
  ]
}

This schema eliminated joins and enabled single-document reads for most pages. Updates used atomic operators:

db.photos.updateOne(
  { _id: "photo_123" },
  { $inc: { likes: 1 }, $push: { comments: { user: "Bob", text: "Nice!" } } }
);

Indexes on user_id, tags, and timestamp ensured sub-millisecond query performance.

Mustache: The Logic-Less Templating Revolution

The second pillar was Mustache, a templating engine that enforced separation of concerns by allowing no logic in templates—only iteration and conditionals:

{{#photo}}
  <h1>{{title}}</h1>
  <img src="{{url}}" alt="{{title}}" />
  <p>By {{user.name}} • {{likes}} likes</p>
  <ul class="comments">
    {{#comments}}
      <li><strong>{{user}}</strong>: {{text}}</li>
    {{/comments}}
  </ul>
{{/photo}}

Because templates contained no business logic, they could be cached indefinitely in Varnish. Only the data changed—and that came fresh from MongoDB on every request.

data = mongo.photos.find(_id: params[:id]).first
html = Mustache.render(template, data)

The Hybrid Architecture: Cache Where It Makes Sense

The final system was a hybrid of caching and real-time rendering:
– Static assets (CSS, JS, images) → CDN with long TTL.
– Static page fragments (headers, footers, sidebars) → Varnish ESI with 1-hour TTL.
– Dynamic content (photo, comments, likes) → MongoDB + Mustache, no cache.

This reduced cache invalidation surface by 90% and average response time from 800ms to 180ms.

2025: The Evolution of Cache-Minimized Architecture

EDIT:
The principles pioneered by Fotopedia are now mainstream:
– Server-side rendering with HTMX for dynamic updates.
– Edge computing with Cloudflare Workers to assemble pages.
– MongoDB Atlas with change streams for real-time UIs.
– Spring Boot + Mustache for Java backends.

Links

• MongoDB Atlas
• Mustache.java
• HTMX
• Video: MongoDB et Mustache

Lecturer

Emmanuel Lecharny is a member of the Apache Software Foundation, contributing to projects like Apache Directory Server and Apache MINA. He also mentors incubating projects such as Deft and Syncope. As founder of his own company, he collaborates on OpenLDAP development through partnerships.

Abstract

Emmanuel Lecharny’s presentation delves into the intricacies of network input/output (NIO) in Java, contrasting it with blocking I/O (BIO) and asynchronous I/O (AIO). Through detailed explanations and code examples, he explores concurrency management, scalability, encoding/decoding, and performance in building efficient servers using Apache MINA. The talk emphasizes practical challenges and solutions, advocating framework use to simplify complex implementations while highlighting system-level considerations like buffers and selectors.

Fundamentals of I/O Models: BIO, NIO, and AIO Compared

Lecharny begins by outlining the three primary I/O paradigms in Java: blocking I/O (BIO), non-blocking I/O (NIO), and asynchronous I/O (AIO). BIO, the traditional model, assigns a thread per connection, blocking until data arrives. This simplicity suits low-connection scenarios but falters under high load, as threads consume resources—up to 1MB stack each—leading to context switching overhead.

NIO introduces selectors and channels, enabling a single thread to monitor multiple connections via events like OP_READ or OP_WRITE. This non-blocking approach scales better, handling thousands of connections without proportional threads. However, it requires manual state management, as partial reads/writes necessitate buffering.

AIO, added in Java 7, builds on NIO with callbacks or futures for completion notifications, reducing polling needs. Yet, it demands careful handler design to avoid blocking the callback thread, often necessitating additional threading for processing.

These models address concurrency differently: BIO is straightforward but resource-intensive; NIO offers efficiency through event-driven multiplexing; AIO provides true asynchrony but with added complexity in callback handling.

Building Scalable Servers with Apache MINA: Core Components and Configuration

Apache MINA simplifies NIO/AIO development by abstracting low-level details. Lecharny demonstrates a basic UDP server: instantiate IoAcceptor, bind to a port, and set a handler for messages. The framework manages buffers, threading, and protocol encoding/decoding.

Key components include IoService (for acceptors/connectors), IoHandler (for events like messageReceived), and filters (e.g., logging, protocol codecs). Configuration involves thread pools: one for I/O (typically one thread suffices due to selectors), another for application logic to prevent blocking.

Scalability hinges on proper setup: use direct buffers for large data to avoid JVM heap copies, but heap buffers for small payloads in Java 7 for speed. MINA’s executor filter offloads heavy computations, maintaining responsiveness.

Code example:

DatagramAcceptor acceptor = new NioDatagramAcceptor();
acceptor.setHandler(new MyHandler());
SocketAddress address = new InetSocketAddress(port);
acceptor.bind(address);

This binds a UDP acceptor, ready for incoming datagrams.

Handling Data: Encoding, Decoding, and Buffer Management

Encoding/decoding is pivotal; MINA’s ProtocolCodecFilter uses encoders/decoders for byte-to-object conversion. Lecharny explains cumulative decoding for fragmented messages: maintain a buffer, append incoming data, and decode when complete (e.g., via length prefixes).

Buffers in NIO are crucial: ByteBuffer for data storage, with position, limit, and capacity. Direct buffers (allocateDirect) bypass JVM heap for zero-copy I/O, ideal for large transfers, but allocation is costlier. Heap buffers (allocate) are faster for small sizes.

Performance tests show Java 7 heap buffers outperforming direct ones up to 64KB; beyond, direct excels. UDP limits (64KB max) favor heap buffers.

Partial writes require looping until completion, tracking written bytes. MINA abstracts this, but understanding underlies effective use.

public class LengthPrefixedDecoder extends CumulativeProtocolDecoder {
    protected boolean doDecode(IoSession session, IoBuffer in, ProtocolDecoderOutput out) {
        if (in.remaining() < 4) return false;
        int length = in.getInt();
        if (in.remaining() < length) return false;
        // Decode data
        return true;
    }
}

This decoder checks for complete messages via prefixed length.

Concurrency and Performance Optimization in High-Load Scenarios

Concurrency management involves separating I/O from processing: MINA’s single I/O thread uses selectors for event polling, dispatching to worker pools. Avoid blocking in handlers; use executors for database queries or computations.

Scalability tests: on a quad-core machine, MINA handles 10,000+ connections efficiently. UDP benchmarks show Java 7 20-30% faster than Java 6, nearing native speeds. TCP may lag BIO slightly due to overhead, but NIO/AIO shine in connection volume.

Common pitfalls: over-allocating threads (match to cores), ignoring backpressure (queue overloads), and poor buffer sizing. Monitor via JMX: MINA exposes metrics for queued events, throughput.

Lecharny stresses: network rarely bottlenecks; focus on application I/O (databases, disks). 10Gbps networks outpace SSDs, so optimize backend.

Practical Examples: From Simple Servers to Real-World Applications

Lecharny presents realistic servers: a basic echo server with MINA requires minimal code—set acceptor, handler, bind. For protocols like LDAP, integrate codecs for ASN.1 encoding.

In Directory Server, NIO enables handling massive concurrent searches without thread explosion. MINA’s modularity allows stacking filters: SSL for security, compression for efficiency.

For UDP-based services (e.g., DNS), similar setup but with DatagramAcceptor. Handle datagram fragmentation manually if exceeding MTU.

AIO variant: Use AsyncIoAcceptor with CompletionHandlers for callbacks, reducing selector polling.

These examples illustrate MINA’s brevity: functional servers in under 50 lines, versus hundreds in raw NIO.

Implications and Recommendations for NIO Adoption

NIO/AIO demand understanding OS-level mechanics: epoll (Linux) vs. kqueue (BSD) for selectors, impacting portability. Java abstracts this, but edge cases (e.g., IPv6) require vigilance.

Performance gains are situational: BIO suffices for <1000 connections; NIO for scalability. Frameworks like MINA or Netty mitigate complexity, encapsulating best practices.

Lecharny concludes: embrace frameworks to avoid reinventing; comprehend fundamentals for troubleshooting. Java 7+ enhancements make NIO more viable, but test rigorously under load.

Relevant Links and Hashtags

Links:

• Video of the lecture
• Emmanuel Lecharny on LinkedIn

Lecturers

Mathieu Pouymerol and Pierre Baillet were the technical backbone of Fotopedia, a photo-sharing platform that, at its peak, served over five million monthly visitors using a Ruby on Rails application that had been in production for six years. Mathieu, armed with degrees from École Centrale Paris and a background in building custom data stores for dictionary publishers, brought a deep understanding of database design, indexing, and performance optimization. Pierre, also from Centrale and with experience at Cambridge, had spent nearly a decade managing infrastructure, tuning Tomcat, configuring memcached, and implementing geoDNS systems. Together, they faced the ultimate challenge: keeping a legacy Rails monolith responsive under massive, unpredictable traffic while maintaining content freshness and developer velocity.

Abstract

This article presents an exhaustively detailed expansion of Mathieu Pouymerol and Pierre Baillet’s 2012 DevoxxFR presentation, “MongoDB et Mustache, vers la mort du cache ?”, reimagined as a definitive case study in high-traffic web architecture and the evolution of caching strategies. The Fotopedia team inherited a Rails application plagued by slow ORM queries, complex cache invalidation logic, and frequent stale data. Their initial response—edge-side includes (ESI), fragment caching, and multi-layered memcached—bought time but introduced fragility and operational overhead. The breakthrough came from a radical rethinking: use MongoDB as a real-time document store and Mustache as a logic-less templating engine to assemble pages dynamically, eliminating cache for the most volatile content.

This analysis walks through every layer of their architecture: from database schema design to template composition, from CDN integration to failure mode handling. It includes performance metrics, post-mortem analyses, and lessons learned from production incidents. Updated for 2025, it maps their approach to modern tools: MongoDB 7.0 with Atlas, server-side rendering with HTMX, edge computing via Cloudflare Workers, and Spring Boot with Mustache, offering a complete playbook for building cache-minimized, real-time web applications at scale.

The Legacy Burden: A Rails Monolith Under Siege

Fotopedia’s core application was built on Ruby on Rails 2.3, a framework that, while productive for startups, began to show its age under heavy load. The database layer relied on MySQL with aggressive sharding and replication, but ActiveRecord queries were slow, and joins across shards were impractical. The presentation layer used ER 15–20 partials per page, each with its own caching logic. The result was a cache dependency graph so complex that a single user action—liking a photo—could invalidate dozens of cache keys across multiple servers.

The team’s initial strategy was defense in depth:
– Varnish at the edge with ESI for including dynamic fragments.
– Memcached for fragment and row-level caching.
– Custom invalidation daemons to purge stale cache entries.

But this created a house of cards. A missed invalidation led to stale comments. A cache stampede during a traffic spike brought the database to its knees. As Pierre put it, “We were not caching to improve performance. We were caching to survive.”

The Paradigm Shift: Real-Time Data with MongoDB

The turning point came when the team migrated dynamic, user-generated content—photos, comments, tags, likes—to MongoDB. Unlike MySQL, MongoDB stored data as flexible JSON-like documents, allowing embedded arrays and atomic updates:

{
  "_id": "photo_123",
  "title": "Sunset",
  "user_id": "user_456",
  "tags": ["paris", "sunset"],
  "likes": 1234,
  "comments": [
    { "user": "Alice", "text": "Gorgeous!", "timestamp": "2013-04-01T12:00:00Z" }
  ]
}

This schema eliminated joins and enabled single-document reads for most pages. Updates used atomic operators:

db.photos.updateOne(
  { _id: "photo_123" },
  { $inc: { likes: 1 }, $push: { comments: { user: "Bob", text: "Nice!" } } }
);

Indexes on user_id, tags, and timestamp ensured sub-millisecond query performance.

Mustache: The Logic-Less Templating Revolution

The second pillar was Mustache, a templating engine that enforced separation of concerns by allowing no logic in templates—only iteration and conditionals:

{{#photo}}
  <h1>{{title}}</h1>
  <img src="{{url}}" alt="{{title}}" />
  <p>By {{user.name}} • {{likes}} likes</p>
  <ul class="comments">
    {{#comments}}
      <li><strong>{{user}}</strong>: {{text}}</li>
    {{/comments}}
  </ul>
{{/photo}}

Because templates contained no business logic, they could be cached indefinitely in Varnish. Only the data changed—and that came fresh from MongoDB on every request.

data = mongo.photos.find(_id: params[:id]).first
html = Mustache.render(template, data)

The Hybrid Architecture: Cache Where It Makes Sense

The final system was a hybrid of caching and real-time rendering:
– Static assets (CSS, JS, images) → CDN with long TTL.
– Static page fragments (headers, footers, sidebars) → Varnish ESI with 1-hour TTL.
– Dynamic content (photo, comments, likes) → MongoDB + Mustache, no cache.

This reduced cache invalidation surface by 90% and average response time from 800ms to 180ms.

2025: The Evolution of Cache-Minimized Architecture

EDIT:
The principles pioneered by Fotopedia are now mainstream:
– Server-side rendering with HTMX for dynamic updates.
– Edge computing with Cloudflare Workers to assemble pages.
– MongoDB Atlas with change streams for real-time UIs.
– Spring Boot + Mustache for Java backends.

Links

• MongoDB Atlas
• Mustache.java
• HTMX
• Video: MongoDB et Mustache