Jonathan Lalou's Blog

Archive for the ‘en-US’ Category

If you’re working with the AWS SDK for Java and you’ve seen the following log message:

WARN --- AmazonS3Client : No content length specified for stream data. Stream contents will be buffered in memory and could result in out of memory errors.

…you’re not alone. This warning might seem harmless at first, but it can lead to serious issues, especially in production environments.

What’s Really Happening?

This message appears when you upload a stream to Amazon S3 without explicitly setting the content length in the request metadata.

When that happens, the SDK doesn’t know how much data it’s about to upload, so it buffers the entire stream into memory before sending it to S3. If the stream is large, this could lead to:

• Excessive memory usage
• Slow performance
• OutOfMemoryError crashes

✅ How to Fix It

Whenever you upload a stream, make sure you calculate and set the content length using ObjectMetadata.

Example with Byte Array:

byte[] bytes = ...; // your content
ByteArrayInputStream inputStream = new ByteArrayInputStream(bytes);

ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(bytes.length);

PutObjectRequest request = new PutObjectRequest(bucketName, key, inputStream, metadata);
s3Client.putObject(request);

Example with File:

File file = new File("somefile.txt");
FileInputStream fileStream = new FileInputStream(file);

ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(file.length());

PutObjectRequest request = new PutObjectRequest(bucketName, key, fileStream, metadata);
s3Client.putObject(request);

What If You Don’t Know the Length?

Sometimes, you can’t know the content length ahead of time (e.g., you’re piping data from another service). In that case:

• Write the stream to a ByteArrayOutputStream first (good for small data)
• Use the S3 Multipart Upload API to stream large files without specifying the total size

Conclusion

Always set the content length when uploading to S3 via streams. It’s a small change that prevents large-scale problems down the road.

By taking care of this up front, you make your service safer, more memory-efficient, and more scalable.

Got questions or dealing with tricky S3 upload scenarios? Drop them in the comments!

Quantum computing stands at the forefront of technological advancement, promising to unlock solutions to some of humanity’s most complex challenges. Charina Chou and Erik Lucero, representing Google Quantum AI, provided a structured exploration divided into facts, fiction, and future prospects. Their insights draw from ongoing research, emphasizing the quantum mechanical principles that govern nature and how they can be harnessed for computational power. By blending scientific rigor with accessible explanations, they aim to demystify this field, encouraging broader participation from developers and innovators alike.

Fundamental Principles of Quantum Mechanics and Computing

At its core, quantum computing leverages the inherent properties of quantum mechanics, which permeate everyday natural phenomena. For instance, fluorescence, photosynthesis, and even the way birds navigate using Earth’s magnetic field all rely on quantum effects. Charina and Erik highlighted superposition, where particles exist in multiple states simultaneously, and entanglement, where particles share information instantaneously regardless of distance. These concepts enable quantum systems to process information in ways classical computers cannot.

Google Quantum AI’s laboratory embodies this inspiration, adorned with art that celebrates nature’s quantum beauty. Erik described the lab as a space where creativity and science intersect, fostering an environment that propels exploration. The motivation to build quantum computers stems from the limitations of classical systems in simulating natural processes accurately. Nobel laureate Richard Feynman articulated this need, stating that to simulate nature effectively, computations must be quantum mechanical.

The team’s thesis posits quantum computers as tools for exponential speedups in specific domains. Quantum simulation, for example, could revolutionize materials science and biology by modeling molecules and materials with unprecedented precision. This is particularly relevant for drug discovery, where understanding molecular interactions at a quantum level could accelerate the development of treatments for diseases like cancer. Erik shared a personal anecdote about a friend’s battle with cancer, underscoring the human stakes involved. Similarly, quantum machine learning promises efficiency in processing quantum data from sensors, potentially requiring exponentially less data than classical methods.

Enriching this, Google’s roadmap includes milestones like demonstrating quantum supremacy in 2019 with the Sycamore processor, which performed a task in 200 seconds that would take classical supercomputers 10,000 years. This achievement, detailed in Nature, validated the potential for quantum systems to outperform classical ones in targeted computations.

Dispelling Myths and Clarifying Realities

Amidst the hype, numerous misconceptions surround quantum computing. Fiction often depicts quantum computers as immediate threats to global encryption or universal problem-solvers. In truth, while they could factor large numbers efficiently—potentially breaking RSA encryption—this requires error-corrected systems not yet realized. Current quantum computers, like Google’s, operate with noisy intermediate-scale quantum (NISQ) devices, limited in scope.

Charina addressed the myth of quantum computers replacing classical ones, clarifying they excel in niche areas like optimization and simulation, not general-purpose tasks. For instance, they won’t speed up video games or everyday computations but could optimize logistics or financial modeling. Erik debunked the idea of instantaneous computations, noting quantum algorithms like Shor’s for factoring provide polynomial speedups, not infinite ones.

A key milestone was Google’s 2023 demonstration of quantum error correction, published in Nature, where increasing qubits reduced overall error rates—a counterintuitive breakthrough. This “below threshold” achievement, using the Willow chip as of 2024, marks progress toward scalable systems. The chip’s ability to perform calculations beyond classical limits in septillion years faster exemplifies this leap.

Fiction also includes overestimations of current capabilities; quantum computers aren’t yet “useful” for real-world applications but are approaching milestones where they could simulate unattainable chemical reactions or design efficient batteries.

Prospects and Collaborative Pathways Ahead

Looking forward, Google Quantum AI envisions applications in fusion energy, fertilizer production, and beyond. The XPRIZE, sponsored with google.org, offers $5 million to incentivize quantum solutions for global issues, open for submissions to mobilize diverse ideas.

Erik emphasized the need for a global workforce, inviting scientists, engineers, artists, and developers to contribute. The roadmap targets a million-qubit system by milestone six, enabling practical utility. Early processors will aid along the way, with ongoing collaborations fostering innovation.

Recent advancements, like the Willow chip’s error reduction, as reported in Nature 2024, position quantum computing for breakthroughs in medicine and energy. Feynman’s quote on the “wonderful problem” encapsulates the challenge and excitement, inviting collective effort to extend human potential.

Links:

• Charina Chou on LinkedIn
• Erik Lucero on LinkedIn

Sri Satish Ambati, visionary founder and CEO of H2O.ai, extolled the emancipatory ethos of communal code at DotAI 2024. Architecting H2O.ai since 2012 to universalize AI—spanning 20,000 organizations and spearheading o2forindia.org’s life-affirming logistics—Ambati views open-source as sovereignty’s salve. His manifesto positioned multi-agent symphonies as the symphony of tomorrow, where LLMs orchestrate collectives, transmuting solitary sparks into societal symphonies.

The Imperative of Inclusive Innovation

Ambati evoked AGI’s communal cradle: mathematics and melody as heirlooms, AI as extension—public patrimony, not proprietorial prize. Open-source’s vanguard—Meta’s LLaMA kin—eclipses enclosures, birthing bespoke brains via synthetic seeds and scaling sagas.

H2O’s odyssey mirrors this: from nascent nets to agentic ensembles, where h2oGPT’s modular mosaics meld models, morphing monoliths into mosaics. Ambati dissected LLM lineages: from encoder sentinels to decoder dynamos, now agent architects—reasoning relays, tool tenders, memory marshals.

This progression, he averred, democratizes dominion: agents as apprentices, apprising actions, auditing anomalies—autonomy amplified, not abdicated.

Orchestrating Agentic Alliances for Societal Surplus

Ambati unveiled h2oGPTe’s polyphonic prowess: document diviners, code conjurers, RAG refiners—each a specialist in symphonic service. Multi-agent marvels emerge: debate dynamos deliberating dilemmas, hierarchical heralds delegating duties, self-reflective sages self-correcting.

He heralded horizontal harmonies—peers polling peers for probabilistic prudence—and vertical vigils, overseers overseeing outputs. Ambati’s canvas: marketing maestros mirroring motifs, scientific scribes sifting syntheses—abundance assured, from temporal treasures to spatial expanses.

Yet, perils persist: viral venoms, martial mirages, disinformation deluges. Ambati’s antidote: AI as altruism’s ally, open-source as oversight’s oracle—fostering forges where innovation inoculates inequities.

In epilogue, Ambati summoned a selfie symphony, a nod to global galvanizers—from Parisian pulses to San Franciscan surges—where communal code kindles collective conquests.

Forging Futures Through Federated Fabrics

Ambati’s coda canvassed consumption’s crest: prompts as prolific progeny, birthing billion-thought tapestries. AI devours dogmas—SaaS supplanted, Nobels nipped—yet nourishes novelty, urging utility’s uplift.

H2O’s horizon: agentic abundances, ethical engines—open-source as equalizer, ensuring enlightenment’s equity.

Links:

• H2O.ai Website
• Sri Satish Ambati on LinkedIn

From the trenches of an early-stage startup, a CTO’s gaze is fixed on the horizon, but the immediate focus must remain sharply on the ground beneath our feet. The siren song of building a perfectly scalable and architecturally pristine system can be deafening, promising a future of effortless growth. However, for most young companies navigating the volatile landscape of product validation, this pursuit can be a perilous detour. The core imperative? **Relentlessly deliver valuable product features to your initial users.**

In these formative months and years, the paramount goal is **validation**. We must rigorously prove that our core offering solves a tangible problem for a discernible audience and, crucially, that they are willing to exchange value (i.e., money) for that solution. This validation is forged through rapid iteration on our fundamental features, the diligent collection and analysis of user feedback, and the agility to pivot our product direction based on those insights. A CTO understands that time spent over-engineering for a distant future is time stolen from this critical validation process.

Dedicating significant and scarce resources to crafting intricate architectures and achieving theoretical hyper-scalability before establishing a solid product-market fit is akin to constructing a multi-lane superhighway leading to a town with a mere handful of inhabitants. The infrastructure might be an impressive feat of engineering, but its utility is severely limited, representing a significant misallocation of precious capital and effort.

The Early-Stage Advantage: Why the Monolith Often Reigns Supreme

From a pragmatic CTO’s standpoint, the often-underappreciated monolithic architecture presents several compelling advantages during a startup’s vulnerable early lifecycle:

The CTO’s Caution: Resisting the Siren Call of Premature Complexity

The pervasive industry discourse surrounding microservices, Kubernetes, and other distributed technologies can exert considerable pressure on a young engineering team to adopt these paradigms prematurely. However, a seasoned CTO recognizes the inherent risks and advocates for a more pragmatic approach in the early stages:

The Tipping Point: When a CTO Strategically Considers Advanced Architectures

A pragmatic CTO understands that the architectural landscape isn’t static. The transition towards more sophisticated architectures becomes a strategic imperative when the startup achieves demonstrable and sustained traction:

The CTO’s Guiding Principle: Feature Focus Now, Scalability When Ready

As a CTO navigating the turbulent waters of an early-stage startup, the guiding principle remains clear: empower the engineering team to build and iterate rapidly on product features using the most straightforward and efficient tools available. For the vast majority of young companies, a well-architected monolith serves this purpose admirably. A CTO will continuously monitor the company’s growth trajectory and performance metrics, strategically considering more complex architectures like microservices and their associated infrastructure *only when the business need becomes unequivocally evident and the financial resources are appropriately aligned*. The unwavering focus must remain on delivering tangible value to users and rigorously validating the core product in the market. Scalability is a future challenge to be embraced when the time is right, not a premature obsession that jeopardizes the crucial initial progress.

Having recently absorbed my esteemed colleague Danish Javed’s insightful piece on Docker Networking (https://www.linkedin.com/pulse/docker-networking-danish-javed-rzgyf) – a truly worthwhile read for anyone navigating the container landscape – I felt compelled to further explore a critical facet: the intricate security considerations surrounding Docker networking. While Danish laid a solid foundation, let’s delve deeper into how we can fortify our containerized environments at the network level.

Beyond the Walls: Understanding Default Docker Network Isolation

As Danish aptly described, Docker’s inherent isolation, primarily achieved through Linux network namespaces, provides a foundational layer of security. Each container operates within its own isolated network stack, preventing direct port conflicts and limiting immediate interference. Think of it as each container having its own virtual network interface card and routing table within the host’s kernel.

However, it’s crucial to recognize that this isolation is a boundary, not an impenetrable fortress. Containers residing on the *same* Docker network (especially the default bridge network) can often communicate freely. This unrestricted lateral movement poses a significant risk. If one container is compromised, an attacker could potentially pivot and gain access to other services within the same network segment.

Architecting for Security: Leveraging Custom Networks for Granular Control

The first crucial step towards enhanced security is strategically utilizing **custom bridge networks**. Instead of relying solely on the default bridge, design your deployments with network segmentation in mind. Group logically related containers that *need* to communicate on dedicated networks.

docker network create frontend-network
docker network create backend-network
docker network create payment-network
        

docker run --name frontend --network frontend-network -p 80:80 frontend-image
docker run --name auth --network backend-network -p 8081:8080 auth-image
docker run --name users --network backend-network -p 8082:8080 users-image
docker run --name payment --network payment-network -p 8083:8080 payment-image
docker network connect frontend-network auth
docker network connect frontend-network users
docker network connect backend-network users
docker network connect payment-network auth
        

The Fine-Grained Firewall: Implementing Network Policies with CNI Plugins

For truly granular control over inter-container traffic, **Docker Network Policies**, facilitated by CNI (Container Network Interface) plugins like Calico, Weave Net, Cilium, and others, are essential. These policies act as a micro-firewall at the container level, allowing you to define precise rules for ingress (incoming) and egress (outgoing) traffic based on labels, network segments, and port protocols.

apiVersion: networking.k8s.io/v1 # (Calico often aligns with Kubernetes NetworkPolicy API)
kind: NetworkPolicy
metadata:
  name: allow-web-to-db
spec:
  podSelector:
    matchLabels:
      app: db
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: web
    ports:
    - protocol: TCP
      port: 5432
  policyTypes:
  - Ingress
        

Essential Best Practices for a Secure Docker Network

Beyond network segmentation and policies, a holistic approach to Docker network security involves several key best practices:

• Apply the Principle of Least Privilege Network Access: Just as you would with user permissions, grant containers only the necessary network connections required for their specific function. Avoid broad, unrestricted access.
• Isolate Sensitive Workloads on Dedicated, Strictly Controlled Networks: Databases, secret management tools, and other critical components should reside on isolated networks with rigorously defined and enforced network policies.
• Internal Port Obfuscation: While exposing standard ports externally might be necessary, consider using non-default ports for internal communication between services on the same network. This adds a minor layer of defense against casual scanning.
• Exercise Extreme Caution with --network host: This mode bypasses all container network isolation, directly exposing the container’s network interfaces on the host. It should only be used in very specific, well-understood scenarios with significant security implications considered. Often, there are better alternatives.
• Implement Regular Network Configuration Audits: Periodically review your Docker network configurations, custom networks, and network policies (if implemented) to ensure they still align with your security posture and haven’t been inadvertently misconfigured.
• Harden Host Firewalls: Regardless of your internal Docker network configurations, ensure your host machine’s firewall (e.g., iptables, ufw) is properly configured to control all inbound and outbound traffic to the host and any exposed container ports.
• Consider Network Segmentation Beyond Docker: For larger and more complex environments, explore network segmentation at the infrastructure level (e.g., using VLANs or security groups in cloud environments) to further isolate groups of Docker hosts or nodes.
• Maintain Up-to-Date Docker Engine and CNI Plugins: Regularly update your Docker engine and any installed CNI plugins to benefit from the latest security patches and feature enhancements. Vulnerabilities in these core components can have significant security implications.
• Implement Robust Network Monitoring and Logging: Monitor network traffic within your Docker environment for suspicious patterns or unauthorized connection attempts. Centralized logging of network events can be invaluable for security analysis and incident response.
• Secure Service Discovery Mechanisms: If you’re using service discovery tools within your Docker environment, ensure they are properly secured to prevent unauthorized registration or discovery of sensitive services.

Conclusion: A Multi-Layered Approach to Docker Network Security

Securing Docker networking is not a one-time configuration but an ongoing process that requires a layered approach. By understanding the nuances of Docker’s default isolation, strategically leveraging custom networks, implementing granular network policies with CNI plugins, and adhering to comprehensive best practices, you can significantly strengthen the security posture of your containerized applications. Don’t underestimate the network as a critical control plane in your container security strategy. Proactive and thoughtful network design is paramount to building resilient and secure container environments.

Overview

This Python script (rss_to_ebook.py) converts RSS or Atom feeds into EPUB format eBooks, allowing you to read your favorite blog posts and news articles offline in your preferred e-reader. The script intelligently handles both RSS 2.0 and Atom feed formats, preserving HTML formatting while creating a clean, readable eBook.

Key Features

• Dual Format Support: Works with both RSS 2.0 and Atom feeds
• Smart Pagination: Automatically handles paginated feeds using multiple detection methods
• Date Range Filtering: Select specific date ranges for content inclusion
• Metadata Preservation: Maintains feed metadata including title, author, and description
• HTML Formatting: Preserves original HTML formatting while cleaning unnecessary elements
• Duplicate Prevention: Automatically detects and removes duplicate entries
• Comprehensive Logging: Detailed progress tracking and error reporting

Technical Details

The script uses several Python libraries:

• feedparser: For parsing RSS and Atom feeds
• ebooklib: For creating EPUB files
• BeautifulSoup: For HTML cleaning and processing
• logging: For detailed operation tracking

Usage

python rss_to_ebook.py <feed_url> [--start-date YYYY-MM-DD] [--end-date YYYY-MM-DD] [--output filename.epub] [--debug]

Parameters:

• feed_url: URL of the RSS or Atom feed (required)
• --start-date: Start date for content inclusion (default: 1 year ago)
• --end-date: End date for content inclusion (default: today)
• --output: Output EPUB filename (default: rss_feed.epub)
• --debug: Enable detailed logging

Example

python rss_to_ebook.py https://example.com/feed --start-date 2024-01-01 --end-date 2024-03-31 --output my_blog.epub

Requirements

• Python 3.x
• Required packages (install via pip):

  pip install feedparser ebooklib beautifulsoup4

How It Works

1. Feed Detection: Automatically identifies feed format (RSS 2.0 or Atom)
2. Content Processing:
   • Extracts entries within specified date range
   • Preserves HTML formatting while cleaning unnecessary elements
   • Handles pagination to get all available content
3. EPUB Creation:
   • Creates chapters from feed entries
   • Maintains original formatting and links
   • Includes table of contents and navigation
   • Preserves feed metadata

Error Handling

• Validates feed format and content
• Handles malformed HTML
• Provides detailed error messages and logging
• Gracefully handles missing or incomplete feed data

Use Cases

• Create eBooks from your favorite blogs
• Archive important news articles
• Generate reading material for offline use
• Create compilations of related content

Gist: GitHub

Here is the script:

[python]
#!/usr/bin/env python3

import feedparser
import argparse
from datetime import datetime, timedelta
from ebooklib import epub
import re
from bs4 import BeautifulSoup
import logging

# Configure logging
logging.basicConfig(
level=logging.INFO,
format=’%(asctime)s – %(levelname)s – %(message)s’,
datefmt=’%Y-%m-%d %H:%M:%S’
)

def clean_html(html_content):
"""Clean HTML content while preserving formatting."""
soup = BeautifulSoup(html_content, ‘html.parser’)

# Remove script and style elements
for script in soup(["script", "style"]):
script.decompose()

# Remove any inline styles
for tag in soup.find_all(True):
if ‘style’ in tag.attrs:
del tag.attrs[‘style’]

# Return the cleaned HTML
return str(soup)

def get_next_feed_page(current_feed, feed_url):
"""Get the next page of the feed using various pagination methods."""
# Method 1: next_page link in feed
if hasattr(current_feed, ‘next_page’):
logging.info(f"Found next_page link: {current_feed.next_page}")
return current_feed.next_page

# Method 2: Atom-style pagination
if hasattr(current_feed.feed, ‘links’):
for link in current_feed.feed.links:
if link.get(‘rel’) == ‘next’:
logging.info(f"Found Atom-style next link: {link.href}")
return link.href

# Method 3: RSS 2.0 pagination (using lastBuildDate)
if hasattr(current_feed.feed, ‘lastBuildDate’):
last_date = current_feed.feed.lastBuildDate
if hasattr(current_feed.entries, ‘last’):
last_entry = current_feed.entries[-1]
if hasattr(last_entry, ‘published_parsed’):
last_entry_date = datetime(*last_entry.published_parsed[:6])
# Try to construct next page URL with date parameter
if ‘?’ in feed_url:
next_url = f"{feed_url}&before={last_entry_date.strftime(‘%Y-%m-%d’)}"
else:
next_url = f"{feed_url}?before={last_entry_date.strftime(‘%Y-%m-%d’)}"
logging.info(f"Constructed date-based next URL: {next_url}")
return next_url

# Method 4: Check for pagination in feed description
if hasattr(current_feed.feed, ‘description’):
desc = current_feed.feed.description
# Look for common pagination patterns in description
next_page_patterns = [
r’next page: (https?://\S+)’,
r’older posts: (https?://\S+)’,
r’page \d+: (https?://\S+)’
]
for pattern in next_page_patterns:
match = re.search(pattern, desc, re.IGNORECASE)
if match:
next_url = match.group(1)
logging.info(f"Found next page URL in description: {next_url}")
return next_url

return None

def get_feed_type(feed):
"""Determine if the feed is RSS 2.0 or Atom format."""
if hasattr(feed, ‘version’) and feed.version.startswith(‘rss’):
return ‘rss’
elif hasattr(feed, ‘version’) and feed.version == ‘atom10’:
return ‘atom’
# Try to detect by checking for Atom-specific elements
elif hasattr(feed.feed, ‘links’) and any(link.get(‘rel’) == ‘self’ for link in feed.feed.links):
return ‘atom’
# Default to RSS if no clear indicators
return ‘rss’

def get_entry_content(entry, feed_type):
"""Get the content of an entry based on feed type."""
if feed_type == ‘atom’:
# Atom format
if hasattr(entry, ‘content’):
return entry.content[0].value if entry.content else ”
elif hasattr(entry, ‘summary’):
return entry.summary
else:
# RSS 2.0 format
if hasattr(entry, ‘content’):
return entry.content[0].value if entry.content else ”
elif hasattr(entry, ‘description’):
return entry.description
return ”

def get_entry_date(entry, feed_type):
"""Get the publication date of an entry based on feed type."""
if feed_type == ‘atom’:
# Atom format uses updated or published
if hasattr(entry, ‘published_parsed’):
return datetime(*entry.published_parsed[:6])
elif hasattr(entry, ‘updated_parsed’):
return datetime(*entry.updated_parsed[:6])
else:
# RSS 2.0 format uses pubDate
if hasattr(entry, ‘published_parsed’):
return datetime(*entry.published_parsed[:6])
return datetime.now()

def get_feed_metadata(feed, feed_type):
"""Extract metadata from feed based on its type."""
metadata = {
‘title’: ”,
‘description’: ”,
‘language’: ‘en’,
‘author’: ‘Unknown’,
‘publisher’: ”,
‘rights’: ”,
‘updated’: ”
}

if feed_type == ‘atom’:
# Atom format metadata
metadata[‘title’] = feed.feed.get(‘title’, ”)
metadata[‘description’] = feed.feed.get(‘subtitle’, ”)
metadata[‘language’] = feed.feed.get(‘language’, ‘en’)
metadata[‘author’] = feed.feed.get(‘author’, ‘Unknown’)
metadata[‘rights’] = feed.feed.get(‘rights’, ”)
metadata[‘updated’] = feed.feed.get(‘updated’, ”)
else:
# RSS 2.0 format metadata
metadata[‘title’] = feed.feed.get(‘title’, ”)
metadata[‘description’] = feed.feed.get(‘description’, ”)
metadata[‘language’] = feed.feed.get(‘language’, ‘en’)
metadata[‘author’] = feed.feed.get(‘author’, ‘Unknown’)
metadata[‘copyright’] = feed.feed.get(‘copyright’, ”)
metadata[‘lastBuildDate’] = feed.feed.get(‘lastBuildDate’, ”)

return metadata

def create_ebook(feed_url, start_date, end_date, output_file):
"""Create an ebook from RSS feed entries within the specified date range."""
logging.info(f"Starting ebook creation from feed: {feed_url}")
logging.info(f"Date range: {start_date.strftime(‘%Y-%m-%d’)} to {end_date.strftime(‘%Y-%m-%d’)}")

# Parse the RSS feed
feed = feedparser.parse(feed_url)

if feed.bozo:
logging.error(f"Error parsing feed: {feed.bozo_exception}")
return False

# Determine feed type
feed_type = get_feed_type(feed)
logging.info(f"Detected feed type: {feed_type}")

logging.info(f"Successfully parsed feed: {feed.feed.get(‘title’, ‘Unknown Feed’)}")

# Create a new EPUB book
book = epub.EpubBook()

# Extract metadata based on feed type
metadata = get_feed_metadata(feed, feed_type)

logging.info(f"Setting metadata for ebook: {metadata[‘title’]}")

# Set basic metadata
book.set_identifier(feed_url) # Use feed URL as unique identifier
book.set_title(metadata[‘title’])
book.set_language(metadata[‘language’])
book.add_author(metadata[‘author’])

# Add additional metadata if available
if metadata[‘description’]:
book.add_metadata(‘DC’, ‘description’, metadata[‘description’])
if metadata[‘publisher’]:
book.add_metadata(‘DC’, ‘publisher’, metadata[‘publisher’])
if metadata[‘rights’]:
book.add_metadata(‘DC’, ‘rights’, metadata[‘rights’])
if metadata[‘updated’]:
book.add_metadata(‘DC’, ‘date’, metadata[‘updated’])

# Add date range to description
date_range_desc = f"Content from {start_date.strftime(‘%Y-%m-%d’)} to {end_date.strftime(‘%Y-%m-%d’)}"
book.add_metadata(‘DC’, ‘description’, f"{metadata[‘description’]}\n\n{date_range_desc}")

# Create table of contents
chapters = []
toc = []

# Process entries within date range
entries_processed = 0
entries_in_range = 0
consecutive_out_of_range = 0
current_page = 1
processed_urls = set() # Track processed URLs to avoid duplicates

logging.info("Starting to process feed entries…")

while True:
logging.info(f"Processing page {current_page} with {len(feed.entries)} entries")

# Process current batch of entries
for entry in feed.entries[entries_processed:]:
entries_processed += 1

# Skip if we’ve already processed this entry
entry_id = entry.get(‘id’, entry.get(‘link’, ”))
if entry_id in processed_urls:
logging.debug(f"Skipping duplicate entry: {entry_id}")
continue
processed_urls.add(entry_id)

# Get entry date based on feed type
entry_date = get_entry_date(entry, feed_type)

if entry_date < start_date:
consecutive_out_of_range += 1
logging.debug(f"Skipping entry from {entry_date.strftime(‘%Y-%m-%d’)} (before start date)")
continue
elif entry_date > end_date:
consecutive_out_of_range += 1
logging.debug(f"Skipping entry from {entry_date.strftime(‘%Y-%m-%d’)} (after end date)")
continue
else:
consecutive_out_of_range = 0
entries_in_range += 1

# Create chapter
title = entry.get(‘title’, ‘Untitled’)
logging.info(f"Adding chapter: {title} ({entry_date.strftime(‘%Y-%m-%d’)})")

# Get content based on feed type
content = get_entry_content(entry, feed_type)

# Clean the content
cleaned_content = clean_html(content)

# Create chapter
chapter = epub.EpubHtml(
title=title,
file_name=f’chapter_{len(chapters)}.xhtml’,
content=f'<h1>{title}</h1>{cleaned_content}’
)

# Add chapter to book
book.add_item(chapter)
chapters.append(chapter)
toc.append(epub.Link(chapter.file_name, title, chapter.id))

# If we have no entries in range or we’ve seen too many consecutive out-of-range entries, stop
if entries_in_range == 0 or consecutive_out_of_range >= 10:
if entries_in_range == 0:
logging.warning("No entries found within the specified date range")
else:
logging.info(f"Stopping after {consecutive_out_of_range} consecutive out-of-range entries")
break

# Try to get more entries if available
next_page_url = get_next_feed_page(feed, feed_url)
if next_page_url:
current_page += 1
logging.info(f"Fetching next page: {next_page_url}")
feed = feedparser.parse(next_page_url)
if not feed.entries:
logging.info("No more entries available")
break
else:
logging.info("No more pages available")
break

if entries_in_range == 0:
logging.error("No entries found within the specified date range")
return False

logging.info(f"Processed {entries_processed} total entries, {entries_in_range} within date range")

# Add table of contents
book.toc = toc

# Add navigation files
book.add_item(epub.EpubNcx())
book.add_item(epub.EpubNav())

# Define CSS style
style = ”’
@namespace epub "http://www.idpf.org/2007/ops";
body {
font-family: Cambria, Liberation Serif, serif;
}
h1 {
text-align: left;
text-transform: uppercase;
font-weight: 200;
}
”’

# Add CSS file
nav_css = epub.EpubItem(
uid="style_nav",
file_name="style/nav.css",
media_type="text/css",
content=style
)
book.add_item(nav_css)

# Create spine
book.spine = [‘nav’] + chapters

# Write the EPUB file
logging.info(f"Writing EPUB file: {output_file}")
epub.write_epub(output_file, book, {})
logging.info("EPUB file created successfully")
return True

def main():
parser = argparse.ArgumentParser(description=’Convert RSS feed to EPUB ebook’)
parser.add_argument(‘feed_url’, help=’URL of the RSS feed’)
parser.add_argument(‘–start-date’, help=’Start date (YYYY-MM-DD)’,
default=(datetime.now() – timedelta(days=365)).strftime(‘%Y-%m-%d’))
parser.add_argument(‘–end-date’, help=’End date (YYYY-MM-DD)’,
default=datetime.now().strftime(‘%Y-%m-%d’))
parser.add_argument(‘–output’, help=’Output EPUB file name’,
default=’rss_feed.epub’)
parser.add_argument(‘–debug’, action=’store_true’, help=’Enable debug logging’)

args = parser.parse_args()

if args.debug:
logging.getLogger().setLevel(logging.DEBUG)

# Parse dates
start_date = datetime.strptime(args.start_date, ‘%Y-%m-%d’)
end_date = datetime.strptime(args.end_date, ‘%Y-%m-%d’)

# Create ebook
if create_ebook(args.feed_url, start_date, end_date, args.output):
logging.info(f"Successfully created ebook: {args.output}")
else:
logging.error("Failed to create ebook")

if __name__ == ‘__main__’:
main()

[/python]

In the torrent of event-driven ecosystems, where streams supplant silos and resilience reigns, Ian Cooper, a polyglot architect and Brighter’s steward, demystifies Kafka for .NET artisans. As London’s #ldnug founder and a messaging maven, Ian unravels Kafka’s enigma—records, offsets, SerDes, schemas—from novice nods to nuanced integrations. His hour, a whirlwind of wisdom and wireframes, equips ensembles to embed Kafka as backbone, blending brokers with .NET’s breadth for robust, reactive realms.

Ian immerses immediately: Kafka, a distributed commit log, chronicles changes for consumption, contrasting queues’ ephemera. Born from LinkedIn’s logging ledger in 2011, it scaled to streams, spawning Connect for conduits and Flink for flows. Ian’s inflection: Kafka as nervous system, not notification nook—durable, disorderly, decentralized.

Unpacking the Pipeline: Kafka’s Primal Primitives

Kafka’s corpus: topics as ledgers, partitioned for parallelism, replicated for redundancy. Producers pen records—key-value payloads with headers—SerDes serializing strings or structs. Consumers cull via offsets, groups coordinating coordination, enabling elastic elasticity.

Ian illuminates inroads: Confluent’s Cloud for coddling, self-hosted for sovereignty. .NET’s ingress: Confluent.Kafka NuGet, crafting IProducer for publishes, IConsumer for pulls. His handler: await producer.ProduceAsync(topic, new Message {Key = key, Value = serialized}).

Schemas safeguard: registries register Avro or Protobuf, embedding IDs for evolution. Ian’s caveat: magic bytes mandate manual marshaling in .NET, yet compatibility curtails chaos.

Forging Flows: From Fundamentals to Flink Frontiers

Fundamentals flourish: idempotent producers preclude duplicates, transactions tether topics. Ian’s .NET nuance: transactions via BeginTransaction, committing confluences. Exactly-once semantics, once Java’s jewel, beckon .NET via Kafka Streams’ kin.

Connect catalyzes: sink sources to SQL, sources streams from files—redpanda’s kin for Kafka-less kinship. Flink forges further: stream processors paralleling data dances, yet .NET’s niche narrows to basics.

Ian’s interlude: brighter bridges, abstracting brokers for seamless swaps—Rabbit to Kafka—sans syntactic shifts.

Safeguarding Streams: Resilience and Realms

Resilience roots in replicas: in-sync sets (ISR) insure idempotence, unclean leader elections avert anarchy. Ian’s imperative: tune retention—time or tally—for traceability, not torrent.

His horizon: Kafka as canvas for CQRS, where commands commit, queries query—event sourcing’s engine.

Links:

• BrighterCommand GitHub
• Ian Cooper on LinkedIn
• Ian Cooper on GitHub

I’ve created a Python script that converts WordPress export files to Blogger/Atom XML format. Here’s how to use it:

The script takes two command-line arguments:

• wordpress_export.xml: Path to your WordPress export XML file
• blogger_export.xml : Path where you want to save the converted Blogger/Atom XML file

To run the script:

python wordpress_to_blogger.py wordpress_export.xml blogger_export.xml

The script performs the following conversions:

• Converts WordPress posts to Atom feed entries
• Preserves post titles, content, publication dates, and authors
• Maintains categories as Atom categories
• Handles post status (published/draft)
• Preserves HTML content formatting
• Converts dates to ISO format required by Atom

The script uses Python’s built-in xml.etree.ElementTree module for XML processing and includes error handling to make it robust.
Some important notes:

• The script only converts posts (not pages or other content types)
• It preserves the HTML content of your posts
• It maintains the original publication dates
• It handles both published and draft posts
• The output is a valid Atom XML feed that Blogger can import

The file:

[python]#!/usr/bin/env python3
import xml.etree.ElementTree as ET
import sys
import argparse
from datetime import datetime
import re

def convert_wordpress_to_blogger(wordpress_file, output_file):
# Parse WordPress XML
tree = ET.parse(wordpress_file)
root = tree.getroot()

# Create Atom feed
atom = ET.Element(‘feed’, {
‘xmlns’: ‘http://www.w3.org/2005/Atom’,
‘xmlns:app’: ‘http://www.w3.org/2007/app’,
‘xmlns:thr’: ‘http://purl.org/syndication/thread/1.0’
})

# Add feed metadata
title = ET.SubElement(atom, ‘title’)
title.text = ‘Blog Posts’

updated = ET.SubElement(atom, ‘updated’)
updated.text = datetime.now().isoformat()

# Process each post
for item in root.findall(‘.//item’):
if item.find(‘wp:post_type’, {‘wp’: ‘http://wordpress.org/export/1.2/’}).text != ‘post’:
continue

entry = ET.SubElement(atom, ‘entry’)

# Title
title = ET.SubElement(entry, ‘title’)
title.text = item.find(‘title’).text

# Content
content = ET.SubElement(entry, ‘content’, {‘type’: ‘html’})
content.text = item.find(‘content:encoded’, {‘content’: ‘http://purl.org/rss/1.0/modules/content/’}).text

# Publication date
pub_date = item.find(‘pubDate’).text
published = ET.SubElement(entry, ‘published’)
published.text = datetime.strptime(pub_date, ‘%a, %d %b %Y %H:%M:%S %z’).isoformat()

# Author
author = ET.SubElement(entry, ‘author’)
name = ET.SubElement(author, ‘name’)
name.text = item.find(‘dc:creator’, {‘dc’: ‘http://purl.org/dc/elements/1.1/’}).text

# Categories
for category in item.findall(‘category’):
category_elem = ET.SubElement(entry, ‘category’, {‘term’: category.text})

# Status
status = item.find(‘wp:status’, {‘wp’: ‘http://wordpress.org/export/1.2/’}).text
if status == ‘publish’:
app_control = ET.SubElement(entry, ‘app:control’, {‘xmlns:app’: ‘http://www.w3.org/2007/app’})
app_draft = ET.SubElement(app_control, ‘app:draft’)
app_draft.text = ‘no’
else:
app_control = ET.SubElement(entry, ‘app:control’, {‘xmlns:app’: ‘http://www.w3.org/2007/app’})
app_draft = ET.SubElement(app_control, ‘app:draft’)
app_draft.text = ‘yes’

# Write the output file
tree = ET.ElementTree(atom)
tree.write(output_file, encoding=’utf-8′, xml_declaration=True)

def main():
parser = argparse.ArgumentParser(description=’Convert WordPress export to Blogger/Atom XML format’)
parser.add_argument(‘wordpress_file’, help=’Path to WordPress export XML file’)
parser.add_argument(‘output_file’, help=’Path to output Blogger/Atom XML file’)

args = parser.parse_args()

try:
convert_wordpress_to_blogger(args.wordpress_file, args.output_file)
print(f"Successfully converted {args.wordpress_file} to {args.output_file}")
except Exception as e:
print(f"Error: {str(e)}")
sys.exit(1)

if __name__ == ‘__main__’:
main()[/python]

Daniel Messe, a seasoned librarian, delivers a passionate call to action against the corporatization of public libraries. Facing challenges like book bans, inflated eBook prices, and restricted access to academic research, Daniel shares his journey as a “shadow librarian,” using quasi-legal methods to ensure equitable access to knowledge. His talk inspires attendees to join the fight for open information in an era of digital gatekeeping.

The Plight of Public Libraries

Daniel opens by highlighting the existential threats to libraries, including censorship and corporate exploitation. He describes how publishers impose exorbitant eBook licensing fees, rendering digital content unaffordable for libraries. Book bans, particularly targeting marginalized voices, further erode access. Daniel’s narrative underscores the library’s role as a public good, now undermined by profit-driven models.

Shadow Librarianship in Action

Drawing from three decades of library work, Daniel recounts his efforts to bypass restrictive systems. By digitizing out-of-print materials and sharing banned books, he ensures access for underserved communities. His methods, while ethically driven, skirt legal boundaries, reflecting a commitment to serving patrons over corporate interests. Daniel’s stories, including providing banned books to struggling youth, resonate deeply.

Empowering Community Action

Daniel encourages attendees to become shadow librarians, emphasizing that anyone can contribute by sharing knowledge. He advocates for scanning and distributing unavailable materials, challenging unconstitutional bans, and supporting patrons in need. His lack of a formal library degree, yet extensive impact, illustrates that passion and action outweigh credentials in this fight.

Building a Knowledge Commons

Concluding, Daniel envisions a future where communities reclaim access to information. He urges collective resistance against corporate control, drawing parallels to hacker ethics of openness and collaboration. By sharing resources and skills, anyone can become a librarian for their community, ensuring knowledge remains a public right rather than a commodity.

Links:

• None

In the world of systems architecture, we’re deeply familiar with the dangers of single points of failure: a server goes down, and suddenly, an entire service collapses. But what about the human side of our operations? What happens when a single employee holds the keys—sometimes literally—to critical infrastructure or institutional knowledge?

As a project manager, you’re not just responsible for timelines and deliverables—you’re also a risk manager. And one of the most insidious risks to any project or company is over-reliance on one individual.

The “Only One Who Knows” Problem

Here are some familiar but risky scenarios:

• The lead engineer who is the only one with access to production.

• The architect who built a legacy system but never documented it.

• The IT admin who’s the sole owner of critical credentials.

• The contractor who manages deployments but stores scripts only on their local machine.

These situations might feel efficient in the short term—“Let her handle it, she knows it best”—but they are dangerous. Because the moment that person is unavailable (sick leave, resignation, burnout, or worse), your entire project or company is exposed.

This isn’t just about contingency; it’s about resilience.

Human Capital Is Capital

As Peter Drucker famously said, “What gets measured gets managed.” But too often, human capital is not measured or managed with the rigor applied to financial or technical assets.

Yet your people—their knowledge, access, habits—are core infrastructure.

Consider the risks:

• Operational disruption if a key team member disappears without handover

• Security vulnerability if credentials are centralized in one individual’s hands

• Knowledge drain when processes live only in someone’s memory

• Compliance risk if proper delegation and documentation are missing

Practical Ways to Mitigate the Risk

As a PM or senior tech manager, you can apply several concrete practices to reduce this risk:

1. 📄 Document Everything

• Maintain centralized and versioned process documentation

• Include architecture diagrams, deployment workflows, emergency protocols

• Use internal wikis or documentation tools like Confluence, Notion, or GitBook

2. 👥 Promote Redundancy Through Collaboration

• Encourage pair programming, shadowing, or “brown bag” sessions

• Rotate team members through different systems to broaden familiarity

3. 🔄 Rotate Access and Responsibilities

• Build redundancy into roles—no one should be a bottleneck

• Use tools like AWS IAM, 1Password, or HashiCorp Vault for shared, audited access

4. 🔎 Test the System Without Them

• Simulate unavailability scenarios. Can the team deploy without X? Can someone else resolve critical incidents?

• This is part of operational resiliency planning

A Real-World Example: HSBC’s Core Vacation Policy

When I worked at HSBC, a global financial institution with high security and compliance standards, they enforced a particularly impactful policy:

👉 Every employee or contractor was required to take at least 1 consecutive week of “core vacation” each year.

The reasons were twofold:

1. ✅ Operational Resilience: To ensure that no person was irreplaceable, and teams could function in their absence.

2. 🚨 Fraud Detection: Continuous presence often masks subtle misuse of systems or privileges. A break allows for behaviors to be reviewed or irregularities to surface.

This policy, common in banking and finance, is a brilliant example of using absence as a testing mechanism—not just for risk, but for trust and transparency.

Building Strong People and Even Stronger Systems

Let’s be clear: this is not about making people “replaceable.”
This is about making systems sustainable and protecting your team from burnout, stress, and unrealistic dependence.

You want to:

• ✅ Respect your team’s contribution

• ✅ Protect them from overexposure

• ✅ Ensure your project or company remains healthy and functional

As the CTO of Basecamp, David Heinemeier Hansson, once said:

“People should be able to take a real vacation without the company collapsing. If they can’t, it’s a leadership failure, not a workforce problem.”

Further Reading and Resources

• 👥 The Human Element of Risk Management (Harvard Business Review)