Posts Tagged ‘Serverless’
[DevoxxUK2024] Devoxx UK Introduces: Aspiring Speakers 2024, Short Talks
The Aspiring Speakers 2024 session at DevoxxUK2024, organized in collaboration with the London Java Community, showcased five emerging talents sharing fresh perspectives on technology and leadership. Rajani Rao explores serverless architectures, Yemurai Rabvukwa bridges chemistry and cybersecurity, Farhath Razzaque delves into AI-driven productivity, Manogna Machiraju tackles imposter syndrome in leadership, and Leena Mooneeram offers strategies for platform team synergy. Each 10-minute talk delivers actionable insights, reflecting the diversity and innovation within the tech community. This session highlights the power of new voices in shaping the future of software development.
Serverless Revolution with Rajani Rao
Rajani Rao, a principal technologist at Viva and founder of the Women Coding Community, presents a compelling case for serverless computing. Using a restaurant analogy—contrasting home cooking (traditional computing) with dining out (serverless)—Rajani illustrates how serverless eliminates infrastructure management, enhances scalability, and optimizes costs. She shares a real-world example of porting a REST API from Windows EC2 instances to AWS Lambda, handling 6 billion monthly requests. This shift, completed in a day, resolved issues like CPU overload and patching failures, freeing the team from maintenance burdens. The result was not only operational efficiency but also a monetized service, boosting revenue and team morale. Rajani advocates starting small with serverless to unlock creativity and improve developer well-being.
Chemistry Meets Cybersecurity with Yemurai Rabvukwa
Yemurai Rabvukwa, a cybersecurity engineer and TikTok content creator under STEM Bab, draws parallels between chemistry and cybersecurity. Her squiggly career path—from studying chemistry in China to pivoting to tech during a COVID-disrupted study abroad—highlights transferable skills like analytical thinking and problem-solving. Yemurai identifies three intersections: pharmaceuticals, healthcare, and energy. In pharmaceuticals, both fields use a prevent-detect-respond framework to safeguard systems and ensure quality. The 2017 WannaCry attack on the NHS underscores a multidisciplinary approach in healthcare, involving stakeholders to restore services. In energy, geopolitical risks and ransomware target renewable sectors, emphasizing cybersecurity’s critical role. Yemurai’s journey inspires leveraging diverse backgrounds to tackle complex tech challenges.
AI-Powered Productivity with Farhath Razzaque
Farhath Razzaque, a freelance full-stack engineer and AI enthusiast, explores how generative AI can transform developer productivity. Quoting DeepMind’s Demis Hassabis, Farhath emphasizes AI’s potential to accelerate innovation. He outlines five levels of AI adoption: zero-shot prompting for quick error resolution, AI apps like Cursor IDE for streamlined coding, prompt engineering for precise outputs, agentic workflows for collaborative AI agents, and custom solutions using frameworks like LangChain. Farhath highlights open-source tools like NoAI Browser and MakeReal, which rival commercial offerings at lower costs. By automating repetitive tasks and leveraging domain expertise, developers can achieve 10x productivity gains, preparing for an AI-driven future.
Overcoming Imposter Syndrome with Manogna Machiraju
Manogna Machiraju, head of engineering at Domestic & General, shares a candid exploration of imposter syndrome in leadership roles. Drawing from her 2017 promotion to engineering manager, Manogna recounts overworking to prove her worth, only to face project failure and team burnout. This prompted reflection on her role’s expectations, realizing she wasn’t meant to code but to enable her team. She advocates building clarity before acting, appreciating team efforts, and embracing tolerable imperfection. Manogna also addresses the challenge of not being the expert in senior roles, encouraging curiosity and authenticity over faking expertise. Her principle—leaning into discomfort with determination—offers a roadmap for navigating leadership doubts.
Platform Happiness with Leena Mooneeram
Leena Mooneeram, a platform engineer at Chainalysis, presents a developer’s guide to platform happiness, emphasizing mutual engagement between engineers and platform teams. Viewing platforms as products, Leena suggests three actions: be an early adopter to shape tools and build relationships, contribute by fixing documentation or small bugs, and question considerately with context and urgency details. These steps enhance platform robustness and reduce friction. For instance, early adopters provide critical feedback, while contributions like PRs for typos streamline workflows. Leena’s mutual engagement model fosters collaboration, ensuring platforms empower engineers to build software joyfully and efficiently.
Links:
[DevoxxPL2022] Accelerating Big Data: Modern Trends Enable Product Analytics • Boris Trofimov
Boris Trofimov, a big data expert from Sigma Software, delivered an insightful presentation at Devoxx Poland 2022, exploring modern trends in big data that enhance product analytics. With experience building high-load systems like the AOL data platform for Verizon Media, Boris provided a comprehensive overview of how data platforms are evolving. His talk covered architectural innovations, data governance, and the shift toward serverless and ELT (Extract, Load, Transform) paradigms, offering actionable insights for developers navigating the complexities of big data.
The Evolving Role of Data Platforms
Boris began by demystifying big data, often misconstrued as a magical solution for business success. He clarified that big data resides within data platforms, which handle ingestion, processing, and analytics. These platforms typically include data sources, ETL (Extract, Transform, Load) pipelines, data lakes, and data warehouses. Boris highlighted the growing visibility of big data beyond its traditional boundaries, with data engineers playing increasingly critical roles. He noted the rise of cross-functional teams, inspired by Martin Fowler’s ideas, where subdomains drive team composition, fostering collaboration between data and backend engineers.
The convergence of big data and backend practices was a key theme. Boris pointed to technologies like Apache Kafka and Spark, which are now shared across both domains, enabling mutual learning. He emphasized that modern data platforms must balance complexity with efficiency, requiring specialized expertise to avoid pitfalls like project failures due to inadequate practices.
Architectural Innovations: From Lambda to Delta
Boris delved into big data architectures, starting with the Lambda architecture, which separates data processing into speed (real-time) and batch layers for high availability. While effective, Lambda’s complexity increases development and maintenance costs. As an alternative, he introduced the Kappa architecture, which simplifies processing by using a single streaming layer, reducing latency but potentially sacrificing availability. Boris then highlighted the emerging Delta architecture, which leverages data lakehouses—hybrid systems combining data lakes and warehouses. Technologies like Snowflake and Databricks support Delta, minimizing data hops and enabling both batch and streaming workloads with a single storage layer.
The Delta architecture’s rise reflects the growing popularity of data lakehouses, which Boris praised for their ability to handle raw, processed, and aggregated data efficiently. By reducing technological complexity, Delta enables faster development and lower maintenance, making it a compelling choice for modern data platforms.
Data Mesh and Governance
Boris introduced data mesh as a response to monolithic data architectures, drawing parallels with domain-driven design. Data mesh advocates for breaking down data platforms into bounded contexts, each owned by a dedicated team responsible for its pipelines and decisions. This approach avoids the pitfalls of monolithic pipelines, such as chaotic dependencies and scalability issues. Boris outlined four “temptations” to avoid: building monolithic pipelines, combining all pipelines into one application, creating chaotic pipeline networks, and mixing domains in data tables. Data mesh, he argued, promotes modularity and ownership, treating data as a product.
Data governance, or “data excellence,” was another critical focus. Boris stressed the importance of practices like data monitoring, quality validation, and retention policies. He advocated for a proactive approach, where engineers address these concerns early to ensure platform reliability and cost-efficiency. By treating data governance as a checklist, teams can mitigate risks and enhance platform maturity.
Serverless and ELT: Simplifying Big Data
Boris highlighted the shift toward serverless technologies and ELT paradigms. Serverless solutions, available across transformation, storage, and analytics tiers, reduce infrastructure management burdens, allowing faster time-to-market. He cited AWS and other cloud providers as enablers, noting that while not always cost-effective, serverless minimizes maintenance efforts. Similarly, ELT—where transformation occurs after loading data into a warehouse—leverages modern databases like Snowflake and BigQuery. Unlike traditional ETL, ELT reduces latency and complexity by using database capabilities for transformations, making it ideal for early-stage projects.
Boris also noted the resurgence of SQL as a domain-specific language across big data tiers, from transformation to governance. By building frameworks that express business logic in SQL, developers can accelerate feature delivery, despite SQL’s perceived limitations. He emphasized that well-designed SQL queries can be powerful, provided engineers avoid poorly structured code.
Productizing Big Data and Business Intelligence
The final trend Boris explored was the productization of big data solutions. He likened this to Intel’s microprocessor revolution, where standardized components accelerated hardware development. Companies like Absorber offer “data platform as a service,” enabling rapid construction of data pipelines through drag-and-drop interfaces. While limited for complex use cases, such solutions cater to organizations seeking quick deployment. Boris also discussed the rise of serverless business intelligence (BI) tools, which support ELT and allow cross-cloud data queries. These tools, like Mode and Tableau, enable self-service analytics, reducing the need for custom platforms in early stages.