Posts Tagged ‘PierreBaillet’
[DevoxxFR2013] MongoDB and Mustache: Toward the Death of the Cache? A Comprehensive Case Study in High-Traffic, Real-Time Web Architecture
Lecturers
Mathieu Pouymerol and Pierre Baillet were the technical backbone of Fotopedia, a photo-sharing platform that, at its peak, served over five million monthly visitors using a Ruby on Rails application that had been in production for six years. Mathieu, armed with degrees from École Centrale Paris and a background in building custom data stores for dictionary publishers, brought a deep understanding of database design, indexing, and performance optimization. Pierre, also from Centrale and with experience at Cambridge, had spent nearly a decade managing infrastructure, tuning Tomcat, configuring memcached, and implementing geoDNS systems. Together, they faced the ultimate challenge: keeping a legacy Rails monolith responsive under massive, unpredictable traffic while maintaining content freshness and developer velocity.
Abstract
This article presents an exhaustively detailed expansion of Mathieu Pouymerol and Pierre Baillet’s 2012 DevoxxFR presentation, “MongoDB et Mustache, vers la mort du cache ?”, reimagined as a definitive case study in high-traffic web architecture and the evolution of caching strategies. The Fotopedia team inherited a Rails application plagued by slow ORM queries, complex cache invalidation logic, and frequent stale data. Their initial response—edge-side includes (ESI), fragment caching, and multi-layered memcached—bought time but introduced fragility and operational overhead. The breakthrough came from a radical rethinking: use MongoDB as a real-time document store and Mustache as a logic-less templating engine to assemble pages dynamically, eliminating cache for the most volatile content.
This analysis walks through every layer of their architecture: from database schema design to template composition, from CDN integration to failure mode handling. It includes performance metrics, post-mortem analyses, and lessons learned from production incidents. Updated for 2025, it maps their approach to modern tools: MongoDB 7.0 with Atlas, server-side rendering with HTMX, edge computing via Cloudflare Workers, and Spring Boot with Mustache, offering a complete playbook for building cache-minimized, real-time web applications at scale.
The Legacy Burden: A Rails Monolith Under Siege
Fotopedia’s core application was built on Ruby on Rails 2.3, a framework that, while productive for startups, began to show its age under heavy load. The database layer relied on MySQL with aggressive sharding and replication, but ActiveRecord queries were slow, and joins across shards were impractical. The presentation layer used ER 15–20 partials per page, each with its own caching logic. The result was a cache dependency graph so complex that a single user action—liking a photo—could invalidate dozens of cache keys across multiple servers.
The team’s initial strategy was defense in depth:
– Varnish at the edge with ESI for including dynamic fragments.
– Memcached for fragment and row-level caching.
– Custom invalidation daemons to purge stale cache entries.
But this created a house of cards. A missed invalidation led to stale comments. A cache stampede during a traffic spike brought the database to its knees. As Pierre put it, “We were not caching to improve performance. We were caching to survive.”
The Paradigm Shift: Real-Time Data with MongoDB
The turning point came when the team migrated dynamic, user-generated content—photos, comments, tags, likes—to MongoDB. Unlike MySQL, MongoDB stored data as flexible JSON-like documents, allowing embedded arrays and atomic updates:
{
"_id": "photo_123",
"title": "Sunset",
"user_id": "user_456",
"tags": ["paris", "sunset"],
"likes": 1234,
"comments": [
{ "user": "Alice", "text": "Gorgeous!", "timestamp": "2013-04-01T12:00:00Z" }
]
}
This schema eliminated joins and enabled single-document reads for most pages. Updates used atomic operators:
db.photos.updateOne(
{ _id: "photo_123" },
{ $inc: { likes: 1 }, $push: { comments: { user: "Bob", text: "Nice!" } } }
);
Indexes on user_id, tags, and timestamp ensured sub-millisecond query performance.
Mustache: The Logic-Less Templating Revolution
The second pillar was Mustache, a templating engine that enforced separation of concerns by allowing no logic in templates—only iteration and conditionals:
{{#photo}}
<h1>{{title}}</h1>
<img src="{{url}}" alt="{{title}}" />
<p>By {{user.name}} • {{likes}} likes</p>
<ul class="comments">
{{#comments}}
<li><strong>{{user}}</strong>: {{text}}</li>
{{/comments}}
</ul>
{{/photo}}
Because templates contained no business logic, they could be cached indefinitely in Varnish. Only the data changed—and that came fresh from MongoDB on every request.
data = mongo.photos.find(_id: params[:id]).first
html = Mustache.render(template, data)
The Hybrid Architecture: Cache Where It Makes Sense
The final system was a hybrid of caching and real-time rendering:
– Static assets (CSS, JS, images) → CDN with long TTL.
– Static page fragments (headers, footers, sidebars) → Varnish ESI with 1-hour TTL.
– Dynamic content (photo, comments, likes) → MongoDB + Mustache, no cache.
This reduced cache invalidation surface by 90% and average response time from 800ms to 180ms.
2025: The Evolution of Cache-Minimized Architecture
EDIT:
The principles pioneered by Fotopedia are now mainstream:
– Server-side rendering with HTMX for dynamic updates.
– Edge computing with Cloudflare Workers to assemble pages.
– MongoDB Atlas with change streams for real-time UIs.
– Spring Boot + Mustache for Java backends.
Links
[DevoxxFR2012] MongoDB and Mustache: Toward the Death of the Cache? A Comprehensive Case Study in High-Traffic, Real-Time Web Architecture
Lecturers
Mathieu Pouymerol and Pierre Baillet were the technical backbone of Fotopedia, a photo-sharing platform that, at its peak, served over five million monthly visitors using a Ruby on Rails application that had been in production for six years. Mathieu, armed with degrees from École Centrale Paris and a background in building custom data stores for dictionary publishers, brought a deep understanding of database design, indexing, and performance optimization. Pierre, also from Centrale and with experience at Cambridge, had spent nearly a decade managing infrastructure, tuning Tomcat, configuring memcached, and implementing geoDNS systems. Together, they faced the ultimate challenge: keeping a legacy Rails monolith responsive under massive, unpredictable traffic while maintaining content freshness and developer velocity.
Abstract
This article presents an exhaustively detailed expansion of Mathieu Pouymerol and Pierre Baillet’s 2012 DevoxxFR presentation, “MongoDB et Mustache, vers la mort du cache ?”, reimagined as a definitive case study in high-traffic web architecture and the evolution of caching strategies. The Fotopedia team inherited a Rails application plagued by slow ORM queries, complex cache invalidation logic, and frequent stale data. Their initial response—edge-side includes (ESI), fragment caching, and multi-layered memcached—bought time but introduced fragility and operational overhead. The breakthrough came from a radical rethinking: use MongoDB as a real-time document store and Mustache as a logic-less templating engine to assemble pages dynamically, eliminating cache for the most volatile content.
This analysis walks through every layer of their architecture: from database schema design to template composition, from CDN integration to failure mode handling. It includes performance metrics, post-mortem analyses, and lessons learned from production incidents. Updated for 2025, it maps their approach to modern tools: MongoDB 7.0 with Atlas, server-side rendering with HTMX, edge computing via Cloudflare Workers, and Spring Boot with Mustache, offering a complete playbook for building cache-minimized, real-time web applications at scale.
The Legacy Burden: A Rails Monolith Under Siege
Fotopedia’s core application was built on Ruby on Rails 2.3, a framework that, while productive for startups, began to show its age under heavy load. The database layer relied on MySQL with aggressive sharding and replication, but ActiveRecord queries were slow, and joins across shards were impractical. The presentation layer used ER 15–20 partials per page, each with its own caching logic. The result was a cache dependency graph so complex that a single user action—liking a photo—could invalidate dozens of cache keys across multiple servers.
The team’s initial strategy was defense in depth:
– Varnish at the edge with ESI for including dynamic fragments.
– Memcached for fragment and row-level caching.
– Custom invalidation daemons to purge stale cache entries.
But this created a house of cards. A missed invalidation led to stale comments. A cache stampede during a traffic spike brought the database to its knees. As Pierre put it, “We were not caching to improve performance. We were caching to survive.”
The Paradigm Shift: Real-Time Data with MongoDB
The turning point came when the team migrated dynamic, user-generated content—photos, comments, tags, likes—to MongoDB. Unlike MySQL, MongoDB stored data as flexible JSON-like documents, allowing embedded arrays and atomic updates:
{
"_id": "photo_123",
"title": "Sunset",
"user_id": "user_456",
"tags": ["paris", "sunset"],
"likes": 1234,
"comments": [
{ "user": "Alice", "text": "Gorgeous!", "timestamp": "2013-04-01T12:00:00Z" }
]
}
This schema eliminated joins and enabled single-document reads for most pages. Updates used atomic operators:
db.photos.updateOne(
{ _id: "photo_123" },
{ $inc: { likes: 1 }, $push: { comments: { user: "Bob", text: "Nice!" } } }
);
Indexes on user_id, tags, and timestamp ensured sub-millisecond query performance.
Mustache: The Logic-Less Templating Revolution
The second pillar was Mustache, a templating engine that enforced separation of concerns by allowing no logic in templates—only iteration and conditionals:
{{#photo}}
<h1>{{title}}</h1>
<img src="{{url}}" alt="{{title}}" />
<p>By {{user.name}} • {{likes}} likes</p>
<ul class="comments">
{{#comments}}
<li><strong>{{user}}</strong>: {{text}}</li>
{{/comments}}
</ul>
{{/photo}}
Because templates contained no business logic, they could be cached indefinitely in Varnish. Only the data changed—and that came fresh from MongoDB on every request.
data = mongo.photos.find(_id: params[:id]).first
html = Mustache.render(template, data)
The Hybrid Architecture: Cache Where It Makes Sense
The final system was a hybrid of caching and real-time rendering:
– Static assets (CSS, JS, images) → CDN with long TTL.
– Static page fragments (headers, footers, sidebars) → Varnish ESI with 1-hour TTL.
– Dynamic content (photo, comments, likes) → MongoDB + Mustache, no cache.
This reduced cache invalidation surface by 90% and average response time from 800ms to 180ms.
2025: The Evolution of Cache-Minimized Architecture
EDIT:
The principles pioneered by Fotopedia are now mainstream:
– Server-side rendering with HTMX for dynamic updates.
– Edge computing with Cloudflare Workers to assemble pages.
– MongoDB Atlas with change streams for real-time UIs.
– Spring Boot + Mustache for Java backends.