Posts Tagged ‘SoftwareEngineering’
[DotJs2024] Becoming the Multi-armed Bandit
In the intricate ballet of software stewardship, where intuition waltzes with empiricism, resides the multi-armed bandit—a probabilistic oracle guiding choices amid uncertainty. Ben Halpern, co-founder of Forem and dev.to’s visionary steward, dissected this gem at dotJS 2024. A full-stack polymath blending code with community curation, Ben recounted its infusions across his odyssey—from parody O’Reilly covers viralizing memes to mutton-busting triumphs—framing bandits as bridges between artistic whimsy and scientific rigor, aligning devs with stakeholders in pursuit of optimal paths.
Ben’s prologue evoked dev.to’s genesis: Twitter-era jests birthing a creative agora, bandit logic A/B-testing post formats for engagement zeniths. The archetype—casino levers, pulls maximizing payouts—mirrors dev dilemmas: UI variants, feature rollouts, content cadences. Exploration probes unknowns; exploitation harvests proven yields. Ben advocated epsilon-greedy: baseline exploitation (1-ε pulls best arm), exploratory ventures (ε samples alternatives), ε tuning via Thompson sampling for contextual nuance.
Practical infusions abounded. Load balancing: bandit selects origins, favoring responsive backends. Feature flags: variants vie, metrics crown victors. Smoke tests: endpoint probes, failures demote. ML pipelines: hyperparameter hunts, models ascend via validation. Ben’s dev.to saga: title A/Bs, bandit-orchestrated, surfacing resonant headlines sans bias. Organizational strata: nascent projects revel in exploration—ideation fests yielding prototypes; maturity mandates exploitation—scaling victors, pruning pretenders. This lexicon fosters accord: explorers and scalers, once at odds, synchronize via phases, preempting pivots’ friction.
Caution tempered zeal: bandits thrive on voluminous outcomes, not trivial toggles; overzealous testing paralyzes. As AI cheapens variants—code gen’s bounty—feedback scaffolds intensify, bandits as arbiters ensuring quality amid abundance. Ben’s coda: wield judiciously, blending craft’s flair with datum’s discipline for endeavors audacious yet assured.
Algorithmic Essence and Variants
Ben unpacked epsilon-greedy’s equilibrium: 90% best-arm fealty, 10% novelty nudges; Thompson’s Bayesian ballet contextualizes. UCB (Upper Confidence Bound) optimism tempers regret, ideal for sparse signals—dev.to’s post tweaks, engagement echoes guiding refinements.
Embeddings in Dev Workflows
Balancing clusters bandit-route requests; flags unleash cohorts, telemetry triumphs. ML’s parameter quests, smoke’s sentinel sweeps—all bandit-bolstered. Ben’s ethos: binary pass-fails sideline; array assays exalt, infrastructure for insight paramount.
Strategic Alignment and Prudence
Projects arc: explore’s ideation inferno yields scale’s forge. Ben bridged divides—stakeholder symposia in bandit vernacular—averting misalignment. Overreach warns: grand stakes summon science; mundane mandates art’s alacrity, future’s variant deluge demanding deft discernment.
Links:
[DevoxxGR2025] Engineering for Social Impact
Giorgos Anagnostaki and Kostantinos Petropoulos, from IKnowHealth, delivered a concise 15-minute talk at Devoxx Greece 2025, portraying software engineering as a creative process with profound social impact, particularly in healthcare.
Engineering as Art
Anagnostaki likened software engineering to creating art, blending design and problem-solving to build functional systems from scratch. In healthcare, this creativity carries immense responsibility, as their work at IKnowHealth supports radiology departments. Their platform, built for Greece’s national imaging repository, enables precise diagnoses, like detecting cancer or brain tumors, directly impacting patients’ lives. This human connection fuels their motivation, transforming code into life-saving tools.
The Radiology Platform
Petropoulos detailed their cloud-based platform on Azure, connecting hospitals and citizens. Hospitals send DICOM imaging files and HL7 diagnosis data via VPN, while citizens access their medical history through a portal, eliminating CDs and printed reports. The system supports remote diagnosis and collaboration, allowing radiologists to share anonymized cases for second opinions, enhancing accuracy and speeding up critical decisions, especially in understaffed regions.
Technical Challenges
The platform handles 2.5 petabytes of imaging data annually from over 100 hospitals, requiring robust storage and fast retrieval. High throughput (up to 600 requests per minute per hospital) demands scalable infrastructure. Front-end challenges include rendering thousands of DICOM images without overloading browsers, while GDPR-compliant security ensures data privacy. Integration with national health systems added complexity, but the platform’s impact—illustrated by Anagnostaki’s personal story of his father’s cancer detection—underscores its value.
Links
[DevoxxBE2024] Mayday Mark 2! More Software Lessons From Aviation Disasters by Adele Carpenter
At Devoxx Belgium 2024, Adele Carpenter delivered a gripping follow-up to her earlier talk, diving deeper into the technical and human lessons from aviation disasters and their relevance to software engineering. With a focus on case studies like Air France 447, Copa Airlines 201, and British Midlands 92, Adele explored how system complexity, redundancy, and human factors like cognitive load and habituation can lead to catastrophic failures. Her session, packed with historical context and practical takeaways, highlighted how aviation’s century-long safety evolution offers critical insights for building robust, human-centric software systems.
The Evolution of Aviation Safety
Adele began by tracing the rapid rise of aviation from the Wright Brothers’ 1903 flight to the jet age, catalyzed by two world wars and followed by a 20% annual growth in commercial air traffic by the late 1940s. This rapid adoption led to a peak in crashes during the 1970s, with 230 fatal incidents, primarily due to pilot error, as shown in data from planecrashinfo.com. However, safety has since improved dramatically, with fatalities dropping to one per 10 million passengers by 2019. Key advancements, like Crew Resource Management (CRM) introduced after the 1978 United Airways 173 crash, reduced pilot-error incidents by enhancing cockpit communication. The 1990s and 2000s saw further gains through fly-by-wire technology, automation, and wind shear detection systems, making aviation a remarkable engineering success story.
The Perils of Redundancy and Complexity
Using Air France 447 (2009) as a case study, Adele illustrated how excessive redundancy can overwhelm users. The Airbus A330’s three pitot tubes, feeding airspeed data to multiple Air Data Inertial Reference Units (ADIRUs), failed due to icing, causing the autopilot to disconnect and bombard pilots with alerts. In alternate law, without anti-stall protection, the less-experienced pilot’s nose-up input led to a stall, exacerbated by conflicting control inputs in the dark cockpit. This cascade of failures—compounded by sensory overload and inadequate training—resulted in 228 deaths. Adele drew parallels to software, recounting an downtime incident at Trifork caused by a RabbitMQ cluster sync issue, highlighting how poorly understood redundancy can paralyze systems under pressure.
Deadly UX and Consistency Over Correctness
Copa Airlines 201 (1992) underscored the dangers of inconsistent user interfaces. A faulty captain’s vertical gyro fed bad data, disconnecting the autopilot. The pilots, trained on a simulator where a switch’s “left” position selected auxiliary data, inadvertently set both displays to the faulty gyro due to a reversed switch design in the actual Boeing 737. This “deadly UX” caused the plane to roll out of the sky, killing all aboard. Adele emphasized that consistency in design—over mere correctness—is critical in high-stakes systems, as it aligns with human cognitive limitations, reducing errors under stress.
Human Factors: Assumptions and Irrationality
British Midlands 92 (1989) highlighted how assumptions can derail decision-making. Experienced pilots, new to the 737-400, mistook smoke from a left engine fire for a right engine issue due to a design change in air conditioning systems. Shutting down the wrong engine led to a crash beside a motorway, though 79 of 126 survived. Adele also discussed irrational behavior under stress, citing the Manchester Airport disaster (1984), where 55 died from smoke inhalation during an evacuation. Post-crash recommendations, like strip lighting and wider exits, addressed irrational human behavior in emergencies, offering lessons for software in designing for stressed users.
Habituation and Complacency
Delta Airlines 1141 (1988) illustrated the risks of habituation, where routine dulls vigilance. Pilots, accustomed to the pre-flight checklist, failed to deploy flaps, missing a warning due to a modified takeoff alert system. The crash after takeoff killed 14. Adele likened this to software engineers ignoring frequent alerts, like her colleague Pete with muted notifications. She urged designing systems that account for human tendencies like habituation, ensuring alerts are meaningful and workflows prevent complacency. Her takeaways emphasized understanding users’ cognitive limits, balancing redundancy with simplicity, and prioritizing human-centric design to avoid software disasters.
Links:
[DevoxxFR2014] Git-Deliver: Streamlining Deployment Beyond Java Ecosystems
Lecturer
Arnaud Bétrémieux is a passionate developer with 18 years of experience, including 8 professionally, specializing in open-source technologies, GNU/Linux, and languages like Java, PHP, and Lisp. He works at Key Consulting, providing development, hosting, consulting, and expertise services. Sylvain Veyrié, with nearly a decade in Java platforms, serves as Director of Delivery at Transparency Rights Management, focusing on big data, and has held roles in development, project management, and training at Key Consulting.
Abstract
This article investigates git-deliver, a deployment tool leveraging Git’s integrity guarantees for simple, traceable, and atomic deployments across diverse languages. It dissects the tool’s mechanics, from remote setup to rollback features, and discusses customization via scripts and presets, emphasizing its role in replacing ad-hoc scripts in dynamic language projects.
Core Principles and Setup
Git-deliver emerges as a Bash script extending Git with a “deliver” subcommand, aiming for simplicity, reliability, efficiency, and universality in deployments. Targeting non-Java environments like Node.js, PHP, or Rails, it addresses the pitfalls of custom scripts that introduce risks in traceability and atomicity.
A deployment target equates to a Git remote over SSH. For instance, creating remotes for test and production environments involves commands like git remote add test deliver@test.example.fr:/appli and git remote add prod deliver@example.fr:/appli. Deliveries invoke git deliver <remote> <version>, where version can be a branch, commit SHA, or tag.
On the target server, git-deliver initializes a bare Git repository alongside a “delivered” directory containing clones for each deployment. Each clone includes Git metadata and a working copy checked out to the specified version. Symbolic links, particularly “current,” point to the latest clone, ensuring a fixed path for applications and atomic switches— the link updates instantaneously, avoiding partial states.
Directory names incorporate timestamps and abbreviated SHAs, facilitating quick identification of deployed versions. This structure preserves history, enabling audits and rollbacks.
Information Retrieval and Rollback Mechanisms
To monitor deployments, git-deliver offers a “status” option. Without arguments, it surveys all remotes, reporting the current commit SHA, tag if applicable, deployment timestamp, and deployer. It also verifies integrity, alerting to uncommitted changes that might indicate manual tampering.
Specifying a remote yields a detailed history of all deliveries, including directory identifiers. Additionally, git-deliver auto-tags each deployment in the local repository, annotating with execution logs and optional messages. Pushing these tags to a central repository shares deployment history team-wide.
Rollback supports recovery: git deliver rollback <remote> reverts to the previous version by updating the “current” symlink to the prior clone. For specific versions, provide the directory name. This leverages preserved clones, ensuring exact restoration even if files were altered post-deployment.
Customization and Extensibility
Deployments divide into stages (e.g., init-remote for first-time setup, post-symlink for post-switch actions), allowing user-provided scripts executed at each. For normal deliveries, scripts might install dependencies or migrate databases; for rollbacks, they handle reversals like database adjustments.
To foster reusability, git-deliver introduces “presets”—collections of stage scripts for frameworks like Rails or Flask. Dependencies between presets (e.g., Rails depending on Ruby) enable modular composition. The “init” command copies preset scripts into a .deliver directory at the project root, customizable and versionable via Git.
This extensibility accommodates varied workflows, such as compiling sources on-server for compiled languages, though git-deliver primarily suits interpreted ones.
Broader Impact on Deployment Practices
By harnessing Git’s push mechanics and integrity checks, git-deliver minimizes errors from manual interventions, ensuring deployments are reproducible and auditable. Its atomic nature prevents service disruptions, crucial for production environments.
While not yet supporting distributed deployments natively, scripts can orchestrate multi-server coordination. Future enhancements might incorporate remote groups for parallel pushes.
In production at Key Consulting, git-deliver demonstrates maturity beyond prototyping, offering a lightweight alternative to complex tools, promoting standardized practices across projects.