Posts Tagged ‘CloudComputing’
[DevoxxPL2022] Challenges Running Planet-Wide Computer: Efficiency • Jacek Bzdak, Beata Strack
Jacek Bzdak and Beata Strack, software engineers at Google Poland, delivered an engaging session at Devoxx Poland 2022, exploring the intricacies of optimizing Google’s planet-scale computing infrastructure. Their talk focused on achieving efficiency in a distributed system spanning global data centers, emphasizing resource utilization, auto-scaling, and operational strategies. By sharing insights from Google’s internal cloud and Autopilot system, Jacek and Beata provided a blueprint for enhancing service performance while navigating the complexities of large-scale computing.
Defining Efficiency in a Global Fleet
Beata opened by framing Google’s data centers as a singular “planet-wide computer,” where efficiency translates to minimizing operational costs—servers, CPU, memory, data centers, and electricity. Key metrics like fleet-wide utilization, CPU/RAM allocation, and growth rate serve as proxies for these costs, though they are imperfect, often masking quality issues like inflated memory usage. Beata stressed that efficiency begins at the service level, where individual jobs must optimize resource consumption, and extends to the fleet through an ecosystem that maximizes resource sharing. This dual approach ensures that savings at the micro level scale globally, a principle applicable even to smaller organizations.
Auto-Scaling: Balancing Utilization and Reliability
Jacek, a member of Google’s Autopilot team, delved into auto-scaling, a critical mechanism for achieving high utilization without compromising reliability. Autopilot’s vertical scaling adjusts resource limits (CPU/memory) for fixed replicas, while horizontal scaling modifies replica counts. Jacek presented data from an Autopilot paper, showing that auto-scaled services maintain memory slack below 20% for median cases, compared to over 60% for manually managed services. Crucially, automation reduces outage risks by dynamically adjusting limits, as demonstrated in a real-world case where Autopilot preempted a memory-induced crash. However, auto-scaling introduces complexity, particularly feedback loops, where overzealous caching or load shedding can destabilize resource allocation, requiring careful integration with application-specific metrics.
Java-Specific Challenges in Auto-Scaling
The talk transitioned to language-specific hurdles, with Jacek highlighting Java’s unique challenges in auto-scaling environments. Just-in-Time (JIT) compilation during application startup spikes CPU usage, complicating horizontal scaling decisions. Memory management poses further issues, as Java’s heap size is static, and out-of-memory errors may be masked by garbage collection (GC) thrashing, where excessive CPU is devoted to GC rather than request handling. To address this, Google sets static heap sizes and auto-scales non-heap memory, though Jacek envisioned a future where Java aligns with other languages, eliminating heap-specific configurations. These insights underscore the need for language-aware auto-scaling strategies in heterogeneous environments.
Operational Strategies for Resource Reclamation
Beata concluded by discussing operational techniques like overcommit and workload colocation to reclaim unused resources. Overcommit leverages the low probability of simultaneous resource spikes across unrelated services, allowing Google to pack more workloads onto machines. Colocating high-priority serving jobs with lower-priority batch workloads enables resource reclamation, with batch tasks evicted when serving jobs demand capacity. A 2015 experiment demonstrated significant machine savings through colocation, a concept influencing Kubernetes’ design. These strategies, combined with auto-scaling, create a robust framework for efficiency, though they demand rigorous isolation to prevent interference between workloads.
Links:
[PHPForumParis2021] Migrating a Bank-as-a-Service to Serverless – Louis Pinsard
Louis Pinsard, an engineering manager at Theodo, captivated the Forum PHP 2021 audience with a detailed recounting of his journey migrating a Bank-as-a-Service platform to a serverless architecture. Having returned to PHP after a hiatus, Louis shared his experience leveraging AWS serverless technologies to enhance scalability and reliability in a high-stakes financial environment. His narrative, rich with practical insights, illuminated the challenges and triumphs of modernizing critical systems. This post explores four key themes: the rationale for serverless, leveraging AWS tools, simplifying with Bref, and addressing migration challenges.
The Rationale for Serverless
Louis Pinsard opened by explaining the motivation behind adopting a serverless architecture for a Bank-as-a-Service platform at Theodo. Traditional server-based systems struggled with scalability and maintenance under the unpredictable demands of financial transactions. Serverless, with its pay-per-use model and automatic scaling, offered a solution to handle variable workloads efficiently. Louis highlighted how this approach reduced infrastructure management overhead, allowing his team to focus on business logic and deliver a robust, cost-effective platform.
Leveraging AWS Tools
A significant portion of Louis’s talk focused on the use of AWS services like Lambda and SQS to build a resilient system. He described how Lambda functions enabled event-driven processing, while SQS managed asynchronous message queues to handle transaction retries seamlessly. By integrating these tools, Louis’s team at Theodo ensured high availability and fault tolerance, critical for financial applications. His practical examples demonstrated how AWS’s native services simplified complex workflows, enhancing the platform’s performance and reliability.
Simplifying with Bref
Louis discussed the role of Bref, a PHP framework for serverless applications, in streamlining the migration process. While initially hesitant due to concerns about complexity, he found Bref to be a lightweight layer over AWS, making it nearly transparent for developers familiar with serverless concepts. Louis emphasized that Bref’s simplicity allowed his team to deploy PHP code efficiently, reducing the learning curve and enabling rapid development without sacrificing robustness, even in a demanding financial context.
Addressing Migration Challenges
Concluding his presentation, Louis addressed the challenges of migrating a legacy system to serverless, including team upskilling and managing dependencies. He shared how his team adopted AWS CloudFormation for infrastructure-as-code, simplifying deployments. Responding to an audience question, Louis noted that Bref’s minimal overhead made it a viable choice over native AWS SDKs for PHP developers. His insights underscored the importance of strategic planning and incremental adoption to ensure a smooth transition, offering valuable lessons for similar projects.
Links:
[KotlinConf2019] Kotless: A Kotlin-Native Approach to Serverless with Vladislav Tankov
Serverless computing has revolutionized how applications are deployed and scaled, but it often comes with its own set of complexities, including managing deployment DSLs like Terraform or CloudFormation. Vladislav Tankov, then a Software Developer at JetBrains, introduced Kotless at KotlinConf 2019 as a Kotlin Serverless Framework designed to simplify this landscape. Kotless aims to eliminate the need for external deployment DSLs by allowing developers to define serverless applications—including REST APIs and event handling—directly within their Kotlin code using familiar annotations. The project can be found on GitHub at github.com/JetBrains/kotless.
Vladislav’s presentation provided an overview of the Kotless Client API, demonstrated its use with a simple example, and delved into the architecture and design concepts behind its code-to-deployment pipeline. The core promise of Kotless is to make serverless computations easily understandable for anyone familiar with event-based architectures, particularly those comfortable with JAX-RS-like annotations.
Simplifying Serverless Deployment with Kotlin Annotations
The primary innovation of Kotless, as highlighted by Vladislav Tankov, is its ability to interpret Kotlin code and annotations to automatically generate the necessary deployment configurations for cloud providers like AWS (initially). Instead of writing separate configuration files in YAML or other DSLs, developers can define their serverless functions, API gateways, permissions, and scheduled events using Kotlin annotations directly on their functions and classes.
For example, creating a REST API endpoint could be as simple as annotating a Kotlin function with @Get("/mypath"). Kotless then parses these annotations during the build process and generates the required infrastructure definitions, deploys the lambdas, and configures the API Gateway. This approach significantly reduces boilerplate and the cognitive load associated with learning and maintaining separate infrastructure-as-code tools. Vladislav emphasized that a developer only needs familiarity with these annotations to create and deploy a serverless REST API application.
Architecture and Code-to-Deployment Pipeline
Vladislav Tankov provided insights into the inner workings of Kotless, explaining its architecture and the pipeline that transforms Kotlin code into a deployed serverless application. This process generally involves:
1. Annotation Processing: During compilation, Kotless processes the special annotations in the Kotlin code to understand the desired serverless architecture (e.g., API routes, event triggers, scheduled tasks).
2. Terraform Generation (Initially): Kotless then generates the necessary infrastructure-as-code configurations (initially using Terraform as a backend for AWS) based on these processed annotations. This includes defining Lambda functions, API Gateway resources, IAM roles, and event source mappings.
3. Deployment: Kotless handles the deployment of these generated configurations and the application code to the target cloud provider.
He also touched upon optimizations built into Kotless, such as “outer warming” of lambdas to reduce cold starts and optimizing lambdas by size. This focus on performance and ease of use is central to Kotless’s philosophy. The framework aims to abstract away the underlying complexities of serverless platforms, allowing developers to concentrate on their application logic.
Future Directions and Multiplatform Aspirations
Looking ahead, Vladislav Tankov discussed the future roadmap for Kotless, including ambitious plans for supporting Kotlin Multiplatform Projects (MPP). This would allow developers to choose different runtimes for their lambdas—JVM, JavaScript, or even Kotlin/Native—depending on the task and performance requirements. Supporting JavaScript lambdas, for example, could open up compatibility with platforms like Google Cloud Platform more broadly, which at the time had better support for JavaScript runtimes than JVM for serverless functions.
Other planned enhancements included extended event handling for custom events on AWS and other platforms, and continued work on performance optimizations. The vision for Kotless was to provide a comprehensive and flexible serverless solution for Kotlin developers, empowering them to build efficient and scalable cloud-native applications with minimal friction. Vladislav encouraged attendees to try Kotless and contribute to its development, positioning it as a community-driven effort to improve the Kotlin serverless experience.
Links:
[DevoxxFR2013] Security for Enterprises in a Cloudy and Mobile World
Lecturer
Ludovic Poitou serves as Product Manager at ForgeRock, overseeing directory products, and holds the position of General Manager for ForgeRock France. With a background in open-source Java and LDAP, he previously worked at Sun Microsystems as a developer and architect for directory solutions, later engaging in community management.
Abstract
Ludovic Poitou examines evolving enterprise security demands amid mobile proliferation, social networks, and cloud computing. Centering on identity management, he analyzes ForgeRock’s Open Identity Stack—an open-source Java solution—detailing standards like OAuth, OpenID Connect, and SCIM. The discussion evaluates impacts on information systems infrastructure and application architecture, advocating adaptive strategies for secure access in hybrid environments.
Shifting Paradigms: Mobile, Cloud, and Social Influences on Security
Poitou identifies three transformative trends reshaping information security: ubiquitous mobile devices, pervasive social platforms, and cloud services adoption. These necessitate reevaluating traditional perimeters, as data flows beyond firewalls to diverse endpoints.
Mobile introduces BYOD challenges—personal devices accessing corporate resources—demanding granular controls. Cloud shifts storage and processing externally, requiring federated trust. Social networks amplify identity federation needs for seamless yet secure interactions.
At the core lies identity management: provisioning, authentication, authorization, and storage across lifecycles. ForgeRock, emerging post-Sun acquisition, builds on open-source projects like OpenDJ (LDAP server) to deliver comprehensive solutions.
Core Components of Open Identity Stack: Directory, Access, and Federation
ForgeRock’s stack comprises OpenDJ for LDAP-based storage, OpenAM for access management, and OpenIDM for identity administration. OpenDJ handles scalable directories; OpenAM manages SSO, federation; OpenIDM orchestrates provisioning.
Poitou highlights Java foundations enabling portability. Development centers in Grenoble support global operations.
This modular approach allows tailored deployments, integrating with existing systems while supporting modern protocols.
Emerging Standards: OAuth, OpenID Connect, and SCIM for Interoperability
Addressing federation, Poitou details OAuth 2.0 for delegated authorization—clients obtain tokens without credentials. Variants include authorization code for web, implicit for browsers.
OpenID Connect layers identity atop OAuth, providing ID tokens (JWT) with user claims. This enables authenticated APIs, profile sharing.
SCIM standardizes user/group provisioning via REST, simplifying cloud integrations. Poitou contrasts with LDAP’s genericity, noting SCIM’s user-centric focus.
Code illustration (conceptual OAuth flow):
// Client requests token
HttpResponse response = client.execute(new HttpPost("token_endpoint"));
// Server validates, issues JWT
JWTClaimsSet claims = new JWTClaimsSet.Builder()
.subject(userId)
.build();
SignedJWT signedJWT = new SignedJWT(header, claims);
These standards facilitate secure, standardized exchanges.
Architectural Implications: Token-Based Authorization and Device Management
Traditional session cookies falter in mobile/cloud; tokens prevail. Applications validate JWTs statelessly, reducing server load.
Poitou discusses administrative token generation—pre-authorizing apps/devices without logins. OpenAM supports this for seamless access.
Infrastructure evolves: decouple authentication from apps via gateways. Hybrid models blend on-premise directories with cloud federation.
Challenges include token revocation, scope management. Solutions involve introspection endpoints, short-lived tokens.
Practical Deployment and Future Considerations
ForgeRock’s stack deploys flexibly—on-premise, cloud, hybrid. OpenDJ scales horizontally; OpenAM clusters for high availability.
Poitou stresses user-centric policies: dynamic authorizations based on context (location, device).
Emerging: UMA for resource owner control. Standards mature via IETF, OpenID Foundation.
Enterprises must adapt architectures for agility, ensuring compliance amid fluidity.
Links:
[DevoxxFR2013] Regular or Decaffeinated? Java’s Future in the Cloud
Lecturer
Alexis Moussine-Pouchkine, a veteran of Sun Microsystems, currently serves as a Developer Relations lead at Google in Paris, assisting developers in achieving success. With over a decade at Sun and nearly two years at Oracle, he brings extensive experience in Java ecosystems and cloud technologies.
Abstract
Alexis Moussine-Pouchkine’s presentation examines Java’s evolution and its potential trajectory in cloud computing. Reflecting on historical shifts in technology, he critiques current limitations and advocates for advancements like multi-tenancy and resource management to ensure Java’s relevance. Through industry examples and forward-looking analysis, the talk underscores the need for adaptation to maintain Java’s position amid resource rationalization and emerging paradigms.
Java’s Maturation and the Cloud Imperative
Moussine-Pouchkine opens by recounting his transition from Sun Microsystems to Oracle and then Google, highlighting how each company has shaped computing history. At Sun, innovation abounded but market fit was inconsistent; Oracle emphasized acquisitions over novelty, straining community ties; Google prioritizes engineers, fostering rapid development.
He likens Java’s current state to emerging from adolescence, facing challenges in cloud environments where resource optimization is paramount. Drawing from his engineering school days with strict quotas on compilation and connection time, Alexis contrasts this with Java’s initial promise of freedom and flexibility. Early experiences with Linux provided boundless experimentation, mirroring Java’s liberating potential in 1997.
The speaker invokes historical predictions: IBM’s CEO allegedly foresaw a market for only five computers in 1943, possibly prescient regarding cloud providers. Bill Gates’ 640K memory quip and Greg Papadopoulos’ 2003 vision of five to seven massive global computers underscore consolidation trends. Papadopoulos envisioned entities like Google, eBay, Salesforce, Microsoft, Amazon, and a Chinese cloud, a perspective less radical today given web evolution.
Java’s centrality in tomorrow’s cloud is questioned. While present in many offerings, most implementations remain prototypes, circumventing Java’s constraints. The cloud demands shared resources and concentration of expertise, yet Java’s future here is uncertain, risking obsolescence like COBOL.
Challenges and Necessary Evolutions for Java in Multi-Tenant Environments
A core issue is Java’s adaptation to multi-tenancy, where multiple applications share a JVM without interference. Current JVMs lack robust isolation, leading to inefficiencies in cloud settings. Moussine-Pouchkine notes Java’s success in Android and Chrome, where processes are segregated, but enterprise demands shared instances for cost savings.
He critiques the stalled JSR-284 for resource management, essential for quotas and usage-based billing. Without these, Java lags in cloud viability. Examples like Google’s App Engine illustrate Java’s limitations: no threads, file system restrictions, and 30-second request limits, forcing workarounds.
Commercial solutions emerge: Waratek’s hypervisor on HotSpot, IBM’s J9 VM, and SAP’s container enable multi-tenancy. Yet, quotas remain crucial for responsible computing, akin to not overindulging at a buffet to ensure sustainability.
Java 9 priorities include modularity (Jigsaw), potentially aiding resource management. Cloud Foundry’s varying memory allocations by language highlight Java’s inefficiencies. Moussine-Pouchkine urges a “slider” for JVM scaling, from minimal to robust, without API fractures.
The community, pioneers in agile practices, continuous integration, and dependency management, must embrace modularity and quotas. Java 7 introduced dynamic languages; Java 8 tackles multicore with lambdas. Recent Oracle slides affirm multi-tenancy and resource management in Java 9 and beyond.
Implications for Sustainable and Credible Informatics
Moussine-Pouchkine advocates responsible informatics: quotas foster predictability, countering perceptions of IT as imprecise and costly. Developers, like artisans, must steward tools and design thoughtfully. Over-reliance on libraries (90% bloat) signals accumulated technical debt.
Quotas enhance credibility, enabling commitments and superior delivery. Java’s adaptive history positions it well, provided the community envisions it “caffeinated” – vibrant and adult – rather than “decaffeinated” and stagnant.
In essence, Java must address multi-tenancy and resources to thrive in consolidated clouds, avoiding the fate of outdated technologies.