Safeguarding New Financial Apps: AWS-Based Backup and Recovery Plan

Name and Sector of Client:

• Client Name: A leading company from the financial sector Financial Company – Neowise
• Sector: Financial Services and Banking

Services Include:

• Personal and commercial banking services
• Loan and EMI solutions

Primary Work:

• Financial research and market analysis
• Risk management and credit assessment
• Management of Customer Financial Records and Transaction Histories
• Ensuring Regulatory Compliance and Reporting

Problem Faced by Client:

A prominent financial services company has successfully migrated its production workloads from on-premises infrastructure to AWS. Following the successful completion of User Acceptance Testing (UAT), the organization is now preparing to launch ten newly developed applications within their AWS production environment. These applications manage a total of 56 TB of financial data and require a reliable, compliant backup and restore strategy to ensure data protection and meet regulatory standards.

The 56 TB of data comprises various critical data types, including customer information (such as personal details, financial profiles, loan histories, and customer service records), transactional data (including financial transactions, trading activity, investment portfolio changes, and payment operations), and regulatory and compliance-related data (such as anti-money laundering records, Know Your Customer (KYC) documentation, and audit logs). Additionally, it includes market and economic intelligence, such as market research, risk evaluations, and economic forecasts.

This data is distributed across a total of eight AWS resources: three EC2 instances, which collectively store 2 TB, and five RDS databases, each holding 10 TB, contributing to the overall 56 TB footprint.

The Applications Overview:

• Product Recommendation Engine: Enhances sales by suggesting complementary products based on customer behavior and preferences. Leverages real-time data to support effective cross-selling strategies.
  • Data Criticality: High to Medium
  • Backup Frequency: Every hour
  • Retention Period: 12 hours
  • RTO: 2 hours
  • RPO: 1 hour

• Stock Control System: Monitors inventory quantities, order fulfillment, and stock transfers to ensure efficient warehouse operations and optimal inventory levels. Real-time updates help avoid stockouts and excess stock.

• Data Criticality: High to Medium
• Backup Frequency: Every hour
• Retention Period: 12 hours
• RTO: 2 hours
• RPO: 1 hour

• Commission Payment Processor: Manages financial disbursements related to agent commissions and payments. Ensures accuracy and timeliness in payment processing.

• Data Criticality: High to Medium
• Backup Frequency: Every hour
• Retention Period: 12 hours
• RTO: 2 hours
• RPO: 1 hour

• Customer Support Tracker: Handles customer service requests and monitors resolution timelines to improve customer satisfaction. Enables streamlined management of customer inquiries and issues.

• Data Criticality: High to Medium 
• Backup Frequency: Every hour 
• Retention Period: 24 hours 
• RTO: 3 hours 
• RPO: 1 hours 

• Vendor Support Manager: Coordinates vendor communications and manages support activities to ensure smooth vendor relationships and timely issue resolution.

• Data Criticality: High to Medium 
• Backup Frequency: Every hour 
• Retention Period: 24 hours 
• RTO: 3 hours 
• RPO: 1 hours

A leading company from the financial sector Financial Company is seeking AWS-native solutions to support the production deployment and protection of these new applications while ensuring full compliance with financial regulations regarding backup and restore procedures.

Challenges for Backups and Restore:

• Limitation on taking Custom Backups: The current backup infrastructure struggles with scalability, limiting the number of backups that can be efficiently managed for compute and database services.
• Limited Retention Period: The retention period for snapshot-based backups is restricted, which affects the efficiency and reliability of the backup process.
• Maintenance Complexity: Manually managing backups across diverse data types is complex and resource-intensive, driving up operational costs.
• Time-Consuming Restores: Restoring 56 TB of data is time-intensive, leading to potential extended downtime and impacting business operations.

Proposed Solution & Architecture:

• AWS Storage Solutions:

• Amazon S3 (Simple Storage Service): Implement S3 for scalable object storage, providing a robust solution for backing up large volumes of financial data. S3’s scalability ensures that the backup system can grow alongside the client’s data needs.
• Amazon EBS (Elastic Block Store): Utilize EBS for block-level storage, offering reliable and secure for backup solution. EBS provides high-performance storage that is for handling transactional and regulatory data.
• Amazon RDS (Relational Database Service): Automate database backups using RDS with the PostgreSQL engine, with using AWS Backup. This ensures reliable and consistent backups of the client’s database systems for financial transactions and records.

• Automated Backup Management:

• AWS Backup: As part of the automated backup management solution, AWS Backup provides a centralized and automated backup service that supports various AWS services, including RDS. It offers centralized, policy-based management and compliance features, reducing complexity and ensuring regulatory adherence. 
• Lambda Functions: Deploy Lambda functions to complement AWS Backup by automating the backup process across multiple EBS volumes and virtual machines. This automation enhances flexibility and control, providing custom automation and event-driven backup processes, and ensuring that all data is consistently and securely backed up.

Reasons for Not Using AWS Backup for EC2 Backups:

1. Selective File-Level Backups: AWS Backup typically operates at the volume or instance level, but FinTech Innovations requires the ability to back up specific files or directories more frequently, which is not directly supported by AWS Backup.
2. AMI Creation Limitation: AWS Backup does not directly create an Amazon Machine Image (AMI) from a backup. Instead, it offers the option to “Create an Image from the latest restore point” during a restore operation, which then creates an AMI. This approach might not align with FinTech Innovations’ need for direct and immediate AMI creation.
3. Granular Scheduling: AWS Backup supports daily, weekly, or monthly backup schedules, but FinTech Innovations requires more frequent backups, such as every 45 minutes, for certain data. AWS Backup’s minimum offering for some resources is 1-hour intervals, which may not meet these specific needs.

• Data Protection and Security:

• Key Management Service (KMS): Implement KMS to encrypt all data stored in EBS volumes and S3, ensuring that financial data remains secure. This adds an additional layer of protection, safeguarding against unauthorized access.
• Cross-Region Replication: Set up cross-region replication for backups, ensuring that data is stored in multiple geographical locations. This redundancy increases data availability and resilience against regional failures or disasters.

• Efficient Data Restoration:

• Secrets Manager Integration: Used AWS Secrets Manager to securely manage access credentials during the restoration process, ensuring that only authorized personnel can restore data.
• Primary Region Restore: Restores data from S3 to Lambda functions for handling regular image updates in an auto-scaling group. This process ensures that the latest images are quickly and efficiently distributed across instances, maintaining optimal performance and availability in the primary region.
• Secondary Region Restore: Manages data restoration from S3 through Lambda functions to various components including Load Balancer (LB), Auto Scaling Group (ASG), EC2 instances, Route 53, Web Application Firewall (WAF), and NAT gateway. This comprehensive approach ensures that all necessary services and infrastructure are promptly and accurately reconfigured, ensuring continuity and resilience in the secondary region.
• Regulatory Compliance:

• Ensured that all backup and restoration processes adhere to financial regulations and internal policies. Implement necessary checks and balances to maintain data security and accuracy throughout the backup and restoration lifecycle.
• IAM and Bucket Policies for enhanced security and control in accessibility to avoid any vulnerability that may arise. 

General Architecture:

Architecture-1: (EC2 Application Backup and Restoration Architecture)

Architecture-2: (RDS Data Backup and Restoration Architecture)

Architecture-3: (S3-Life-Cycle-Policy)

Outcomes:

• Increased Data Scalability and Storage Efficiency:
  • Amazon S3:

The solution will leverage Amazon S3 to efficiently manage 56 TB of financial data, with seamless scalability to accommodate growing data volumes. This eliminates previous storage limitations and ensures high-performance, cost-effective data management.

• Enhanced Data Security:
  • AWS Key Management Service (KMS) & Cross-Region Replication:

All data will be encrypted using AWS KMS and securely replicated across multiple AWS regions. This robust security framework significantly mitigates the risk of data breaches and provides strong protection against regional failures.

• Operational Efficiency through Automation:
  • AWS Lambda:

The integration of AWS Lambda will automate backup and restore operations, reducing the need for manual intervention. This leads to more efficient resource utilization and ensures timely and consistent execution of critical tasks. 

• Reliable and Fast Data Restoration:
  • Automated Restore Processes & AWS Secrets Manager:

Automated restore processes and secure credential management via AWS Secrets Manager will ensure quick and accurate data restoration, minimize downtime and enhance business continuity during disaster recovery scenarios.

• Increased Data Resilience and Availability:
  • Cross-Region Replication:

Replicating data across several AWS regions will increase its availability and resilience, providing continuous access to data even in the event of a regional outage.

• Compliance with Internal Policies and Best Practices:
  • AWS Compliance Tools:

The solution will ensure adherence to internal data governance policies and best practices, reducing the risk of non-compliance and aligning with audit requirements.