Customer Challenge

Future Super faced several challenges as it expanded its investor base and funds under management, prompting a reassessment of its hosting infrastructure on the Google Cloud Platform (GCP).

With most of its workloads running on a Kubernetes Cluster, Future Super realised that Kubernetes was not the optimal solution for their business needs. The complexity and steep learning curve associated with Kubernetes deployment and maintenance proved challenging for the team. Managing Kubernetes, especially without prior experience, required significant effort and resources. Moreover, the Google Kubernetes Engine (GKE)-specific complexities further compounded the learning curve, making it even more daunting for the team to effectively manage the platform.

Additionally, the operational overhead of managing a Kubernetes cluster posed a significant burden. The regular updates, patches, and maintenance tasks required for cluster management consumed valuable resources and detracted from the team's ability to focus on strategic initiatives. The desire for a more streamlined and hands-off approach to infrastructure management became increasingly apparent as the team sought to optimise efficiency and scalability.

The challenges with Kubernetes configuration and maintenance also impacted cluster stability, leading to disruptions and challenges in maintaining service reliability. To mitigate these issues, the team resorted to scheduling problematic jobs during off-peak hours, albeit at the cost of added complexity and inconvenience.

Furthermore, Future Super encountered frustrations with other infrastructure inefficiencies, such as the manual management of databases, which resulted in convoluted processes for updates and maintenance.

The limitations of the GCP setup, including the lack of dedicated development environments beyond Staging and Production, further hindered development and QA efforts. This resulted in some staff resorting to running local development environments with mock data, disrupting deployment processes and impacting overall productivity.

In response to these challenges, Future Super wanted to rearchitect and replatform its infrastructure. By taking proactive action to address these issues, Future Super aimed to streamline operations, enhance scalability, and ensure seamless customer experiences as the company continued to grow, while remaining carbon neutral.

Fitzroy IT - The Perfect Partner

Fitzroy IT prides itself on its breadth and depth of expertise and its ability to be nimble and adapt to our customers' changing needs. As an AWS Advanced Consulting and Well-Architected Partner specialising in Microsoft Windows Migrations, Fitzroy IT assisted Future Super with migrating the Windows workloads of an acquired superannuation fund. This first-hand experience showed Future Super that Fitzroy IT was competent and efficient, with a team wholly committed to achieving quality outcomes.

Fitzroy IT’s experience with the Well-Architected framework, and their ability to undertake regular right-sizing reviews also provides Future Super with the confidence to know that their compute layer is sustainable and maintains a minimal environmental footprint.

What Future Super really valued was Fitzroy IT’s experience with technologies beyond just core AWS infrastructure. With our team of developers and systems administrators, when Future Super needed assistance with modernising their Application Layer, they simply had to turn to their AWS Partner to provide additional assistance.

Why AWS

Future Super's decision to adopt AWS company-wide was influenced by the need to rearchitect the platform to accommodate multiple funds. They had exposure to AWS when it acquired another superannuation fund and migrated its infrastructure to AWS. This showed them several things that were key to their decision to adopt AWS company-wide:

1. AWS provides tools like AWS Organisations and AWS Resource Groups for centralised management of resources, simplifying administrative tasks, monitoring, and governance.
2. AWS's robust security features enable isolation between different tenants or projects, ensuring security through Virtual Private Clouds (VPCs), Security Groups, and Network Access Control Lists (NACLs).
3. AWS had an extensive set of services that made system architecture and implementation quick and easy.
4. All services had simple, granular control interfaces, including simplified management via the AWS console.
5. There was a lot of appeal in being able to “start afresh”, building the infrastructure from the ground up, using best practice and Infrastructure-as-Code tooling. This approach also enabled significant modernisation to occur.
6. The AWS Account team were able to support both Future Super and Fitzroy IT with SA support and financial incentives.
7. AWS are targeting to be powered by 100% renewable energy by 2030 (and are on-track for 2025).
8. The AWS Customer Carbon footprint tool enables Future Super to accurately measure the emissions associated with their use of AWS.
9. AWS' Carbon Emission Factor for its ap-southeast-2 data centre was lower than GCP’s australia-southeast1.

Source: https://www.climatiq.io/blog/cloud-computing-amazon-google-microsoft-helping-companies-go-green

Solution

Fitzroy IT was initially engaged to work solely on infrastructure design and implementation. Planning began, and it soon became apparent that the advanced orchestration features of Kubernetes were excessive for Future Super and that the AWS Elastic Container Service (ECS) was the ideal solution. It supported the Docker images being used by Future Super, so there would be a degree of familiarity. The rollout began with a standard, containerised environment featuring AWS’s core tools for secure networking and a managed ECS Cluster: VPC, ECS, ECR, ELB, S3, EFS and RDS. AWS SSM is used for secure remote access, and CloudWatch provides logging. This multi-account setup was deployed and managed using Infrastructure-as-Code tools, centrally managed by AWS Control Tower, with compliance being enforced with AWS Config and inter-account networking being handled by AWS Transit Gateway.

During discussions with Future Super’s application development team, it quickly became apparent that there was an opportunity to perform substantial modernisation during the re-platforming; modernisation that would radically improve the security, performance and usability of the software as a whole. Fitzroy IT’s engagement was expanded to include modernisation and testing of the application layer. This work included updating the operating systems and software used on all Docker images and establishing new processes for building and deploying the Docker images, utilising AWS native tools, such as AWS Inspector.

Security improvements were also factored into Future Super’s different software services. Previously, in GCP, services had communicated securely with each other using encrypted network traffic that traversed the public Internet. In AWS, it was possible to refactor the services so that all cross-service communication remained within the private AWS network.

A range of tools and services were configured to improve and manage Future Super’s cloud security posture, ensuring it follows best practices and provides automated remediation. Authentication was a core component of Future Super’s solution and was built around least privilege, zero trust access. Role-based access controls (RBAC) were implemented, allowing easier management of access into the different environments.

The modernisation process also led to the adoption of several other AWS services to provide additional functionality and improved performance. AWS API Gateway and Lambda were utilised for secure, public-facing APIs, while AWS Transfer Family provided SFTP services. Elastic File System (EFS) was utilised extensively, and Route53 resolvers allowed EFS to be shared across accounts, provided centralised storage, and optimised costs. Amazon MemoryDB for Redis was utilised to provide a performance boost.

Finally, Future Super and Fitzroy IT worked together to implement an extensive Disaster Recovery plan with AWS Backup at its core.