Heirloom Carbon Overcomes Data Loss and Boosts Efficiency with AWS
Cloud303 Helps Integrate OpenFOAM and Ansys Fluent For Atmospheric Carbon Level Reduction Research CompanyMigration Modernization Disaster Recovery
Heirloom Carbon is a company dedicated to developing technology to reduce atmospheric carbon levels worldwide. By developing scalable direct air capture (DAC) technology, paired with engineering geological carbon storage, Heirloom Carbon offers a path to removing ambient CO2 at the gigatonne scale. They have developed a solution that utilizes Earth-abundant minerals, namely calcium carbonate (CaCO3 ), and by powering the process with renewable energy they are able to capture carbon dioxide (CO2 ) directly from air while maintaining a carbon negative environmental impact.
Heirloom Carbon, a company dedicated to reducing carbon emissions in the atmosphere through sustainable means, recently received a large grant from the US government to support its mission. The company has successfully developed innovative solutions to reduce carbon emissions in various industries, including agriculture, energy, and transportation.
The air that loomed over Heirloom Carbon was filled with dread towards the tail-end of 2022 when their worst fears were realized. A significant portion of the company’s data, vital for fluid flow simulation and heat transfer engineering, was lost during a standard upgrade procedure performed by their prior managed service provider, and all hope was lost when they discovered the crucial data could not be recovered since no backup policy or disaster recover plan existed.
As Heirloom Carbon scrambled to recover from its data loss, the clock ticked away, threatening to set back its efforts to reduce carbon emissions. The company had moved quickly to adopt cloud technology, but its lack of experience with AWS had left them vulnerable to the shortcomings of its previous partner. In the face of this crisis, Heirloom Carbon needed a partner who could not only provide technical expertise but also guide them towards a more secure and scalable infrastructure.
Why Heirloom Carbon Chose AWS?
AWS was ready to step in and help address the issues and challenges Heirloom faced. Heirloom Carbon could access a wealth of tools and services that could support its mission to reduce carbon emissions while also providing enterprise-grade infrastructure and cost savings. By partnering with AWS and Cloud303, Heirloom Carbon saw hope in recovering from its setback and building a more secure and efficient cloud infrastructure to support its critical workload.
Why Heirloom Carbon Chose Cloud303?
Heirloom Carbon chose Cloud303 as their partner due to their unrivaled expertise and experience in HPC and life sciences workloads, robust infrastructure, exceptional support, and a strong track record in the life sciences industry. Previously, Heirloom Carbon partnered with other cloud providers but encountered limitations as their organization scaled and internal operations, including security, were streamlined. Cloud303's commitment to excellence has allowed Heirloom Carbon to focus on their core mission of advancing carbon capture and storage technologies, confident in the knowledge that their computational and data processing needs were in capable hands.
Principal Solutions Architect/Life Sciences Team Lead
EC2 VPC Amazon FSx for Lustre AWS ParallelCluster Simple Storage Service (S3)
For Heirloom Carbon, the use of OpenFOAM and Ansys Fluent is critical to their goal of reducing carbon dioxide (CO2) in the atmosphere. These software packages enable Heirloom Carbon to simulate the performance of different carbon capture technologies and evaluate their effectiveness in reducing atmospheric carbon levels. Through simulations Heirloom Carbon is able to develop more effective carbon capture technologies to deploy at scale, reducing the carbon footprint of industrial processes.
OpenFOAM and Ansys Fluent are widely used computational fluid dynamics (CFD) software packages commonly used to simulate fluid flow and heat transfer in engineering and environmental applications. By deploying these software packages on AWS, it allows researchers and engineers at Heirloom Carbon to model and analyze complex systems difficult to study using traditional experimental methods.
Slurm is a workload manager that is used to manage and schedule jobs on a compute cluster and is particularly beneficial when used in conjunction with OpenFOAM and Ansys Fluent jobs. Slurm allows efficient compute resource management for jobs that are computationally intensive, and requiring significant memory and processing power. Cloud303 engineers implemented Slurm using Amazon EC2 instances to optimize and automate the allocation of AWS resources as needed. This ensures that jobs are processed quickly and efficiently. Additionally, Slurm provides a fair share scheduling algorithm, ensuring that all users get a fair share of compute resources. It prevents any one user from monopolizing resources which could cause delays for others. Ultimately, these improvements result in faster job turnaround times and improved overall efficiency.
The Slurm head node was deployed in a private subnet alongside the compute nodes in a cluster placement group to optimize performance. AWS ParallelClusters were used to provision and manage cluster compute resources such as the Slurm head node, compute queues, and auto-scaling infrastructure.
ParallelCluster is an open-source cluster management tool for deploying and managing High-Performance Computing (HPC) clusters on AWS. AWS ParallelCluster uses a simple text file to model and provision all the resources needed for the HPC application(s) in an automated, secure manner. It supports multiple instance types, job submission queues, and job schedulers like Slurm.
Cloud303 leveraged Amazon FsX for Lustre - a high-performing file system - to ensure Heirloom Carbon’s simulations were executed in a timely and efficient manner while maintaining durability and availability of data. This is a critical aspect of the architectural design: OpenFOAM jobs often involve large and complex simulations that generate a significant amount of data.
Slurm was also configured to manage the powering of nodes when jobs were submitted and shutting off nodes when jobs were complete. An Ansys Fluent license server was hosted on Heirloom Carbon’s corporate network. IP whitelisting controlled access, thus eliminating the need for a site-to-site VPN.
The Slurm head node had Paraview installed to visualize the solution and/or mesh. NICE DCV was used to create Remote Desktop sessions into the SLURM Head Node to use the Paraview GUI. Amazon Simple Storage Service (s3) was used to store job inputs and outputs. All S3 objects utilized a lifecycle policy to transfer infrequently used data into cheaper storage classes. A CLI script was written to submit jobs. This CLI uploaded the required inputs to the S3 bucket before initiating the job.
Cloud303 also provided a notification system to notify users when jobs reached certain key states. This was necessary because of the long duration of OpenFOAM and Ansys Fluent jobs. Cloud303 provided Slurm accounting capability to collect accounting information for every job and job step executed, cluster resource utilization monitoring, multiple users per cluster support with NICE DCV session isolation, and restricted access to all compute resources via OpenVPN, durable log storage, and all using AWS Best Practices. Cloud303 also conducted a Well-Architected Review and provided a Success Criteria Report to Heirloom Carbon.
Cloud303's solution allowed Heirloom Carbon to develop a scalable, adaptable application on AWS. No matter where the future takes this environmental research company, the infrastructure Cloud303 built on AWS will be able to handle any processing tasks researchers throw at it. On the slim chance that another data disaster does occur, Heirloom Carbon has peace of mind with a robust disaster recovery solution in place to protect and recover their data.
"MAP" Migration Acceleration Program 2.0 Large