Client Overview and Sector
The client is one of India’s largest real estate business agencies, operating extensively across the country.
Infrastructure Overview
The organization’s existing IT infrastructure is divided into two segments:
On-Premises Environment
The client maintains three large-scale, monolithic, multi-tier applications hosted on their on-premises servers.
These applications are strictly for internal use, handling sensitive business-critical data. The applications are structured with dedicated web, application, and database tiers.
Applications and Their Functions
- Vendor Management Portal
- Facilitates financial tracking for construction material vendors.
- Manages invoices, purchase requirements, and vendor interactions.
- Construction Management Application
- Supports project scheduling, cost tracking, and resource allocation.
- Handles site management, contract documentation, and contractor assignments.
- Enterprise Resource Planning (ERP) for Employees
- Manages employee attendance, payroll, and HR operations.
On-Premises Database Distribution
The organization’s on-premises databases store significant volumes of business-critical data:
- Vendor Management Database: Holds vendor details, purchase records, and financial transactions, with a data volume of approximately 1.8 TB.
- Construction Management Database: Maintains project data, site information, and cost tracking records, totaling around 6.5 TB.
- ERP Database: Stores employee records, HR data, and payroll details, with a size of 5.7 TB.
This structured on-premises setup forms the backbone of their operational and financial management systems.
AWS Infrastructure Overview
A separate user-centric, multi-tier microservices-based application environment is hosted on an Amazon Elastic Kubernetes Service (EKS) cluster. Given the need for high scalability and dynamic resource allocation, AWS was chosen as the preferred cloud provider. The on-premises data contains critical business records, including:
- User registration details (buyers and sellers) along with their identification documents.
- Property listings with associated documents.
- Tender invoices and architectural plans designed by in-house architects.
- Land records, power of attorney documents, and legal agreements.
As these datasets are highly sensitive, they are handled with strict security measures to comply with Indian regulatory standards such as PDPB, DPDP, and RERA.
AWS EKS Cluster Configuration
- Worker Nodes: 20 nodes across 4 node groups.
- Microservices: Over 50 microservices running in pods to handle different functionalities.
Applications Hosted on AWS EKS
- Property Listings and Management
- Enables customers to browse available properties.
- Provides location details, pricing (both for sale and rent), and property types (commercial/private).
- Customer Feedback Portal
- Allows users to submit feedback, raise queries, and track support requests.
- Interior Design and Modification Services
- Showcases design options for home and office interiors.
- Enables customers to submit service requests.
- Investment Growth Analysis
- Provides insights into real estate market trends and year-over-year (YoY) investment opportunities.
- Sales Inquiry System
- Offers real-time chat support for customers interested in property purchases and investment guidance.
Data Volume in AWS
- The total data stored within the AWS-hosted applications is approximately 4.8 TB.
Challenges Faced by the Client
Inadequate Disaster Recovery Configuration
- The on-premises infrastructure lacked a robust disaster recovery (DR) setup, leading to frequent failures and occasional data loss due to the limitations of their local DR approach.
- Similarly, the AWS EKS-based application did not have a well-defined DR strategy, raising concerns about potential disruptions in the future.
Business Continuity Risks
- All critical business data was housed in a single data center, creating a major operational risk.
- Any downtime or failure in this centralized system could severely impact business operations, potentially resulting in financial losses and reduced customer confidence.
Heightened Data Security Risks
- Storing sensitive business data in a single location increases the risk of security breaches or system failures.
- A single-point failure could compromise confidential business records, financial transactions, and customer information, leading to operational disruptions and reputational damage.
Solution Implementation
Overview of the Solution
ANcrewglobal, leveraging its cloud infrastructure expertise, developed a comprehensive backup and disaster recovery (DR) strategy for the client. This solution integrated AWS S3 storage services with Veeam Backup & Restore, enabling efficient DR for on-premises application servers. Additionally, Amazon S3 Glacier was utilized for long-term archival of legacy snapshots, ensuring business continuity and data protection against potential disruptions.
Network Connectivity Between On-Premises and AWS
- The on-premises network was securely connected to AWS Virtual Private Cloud (VPC) using Site-to-Site VPN, facilitating seamless data transfer and traffic routing.
- Existing on-premises DNS records were replaced with Amazon Route 53. In case of a disaster, Route 53 dynamically redirects traffic to the AWS environment while continuing to serve from the primary on-premises infrastructure during normal operations.
- Veeam Backup & Replication was configured to transfer snapshots to Amazon S3 as Amazon Machine Images (AMIs) through the VPN connection.
- The on-premises load balancer (private IP) was associated with Route 53, ensuring intelligent traffic routing based on infrastructure availability.
Veeam Backup & Restore Licensing
- The client acquired 9 Veeam licenses to enable backup and replication across multiple on-premises servers hosting critical applications.
On-Premises Disaster Recovery Setup with Veeam
- A Veeam Backup & Replication agent was deployed on an on-premises machine to manage backup operations.
- Incremental backups were scheduled, with periodic synthetic full backups to optimize storage and recovery speed.
- These backups were stored in an Amazon S3 bucket for redundancy.
- In the event of a disaster, the backup job triggers infrastructure provisioning in AWS, restoring the environment within the agreed Recovery Time Objective (RTO) of 6 hours and Recovery Point Objective (RPO) of 6 hours.
EKS Cluster Disaster Recovery Strategy
- Amazon RDS Read Replica was created in the DR region, ensuring database availability in case of failure. The secondary region’s read-replica endpoint was preconfigured in the application.
- Two separate Git branches (Main & DR) were set up, with parallel CI/CD pipelines in both regions to maintain an active/active architecture.
- Amazon Route 53 was integrated with AWS Global Accelerator, which continuously monitors the health of Application Load Balancers (ALBs) across both regions. If a failure is detected, traffic is automatically redirected to the DR region.
- In case of a disaster, the read replica is promoted to primary, allowing full read/write operations in the DR region.
- The Service Level Agreement (SLA) defined an RTO of 8 hours and RPO of 12 hours for the EKS environment, which was successfully validated through multiple DR drills.
Architecture:
- On-Premises DR Architecture:
- EKS DR Setup
Security Measures Implemented
- Secure S3 Storage: Snapshots stored in Amazon S3 were encrypted using AWS Key Management Service (KMS) along with Server-Side Encryption (SSE) to enhance data security.
- Least Privilege Access: A strict bucket policy was enforced, ensuring least privilege access to prevent unauthorized access and mitigate security vulnerabilities.
Storage and Cost Optimization
- Lifecycle Management Policies were implemented to automate storage tier transitions within Amazon S3, optimizing costs by moving infrequently accessed files to lower-cost storage classes.
Quarterly Disaster Recovery (DR) Drills
- Regular DR Testing: The team conducts quarterly DR drills to assess the effectiveness of the implemented solution.
- SLA Compliance Reports: After each drill, a detailed DR report is generated, highlighting SLA achievements and system recovery performance.
Services Utilized
AWS Services:
- Storage & Compute: Amazon S3, Amazon EC2
- Database: Amazon RDS
- Container Management: Amazon EKS, Amazon ECR
- Networking & Traffic Management: Route 53, AWS Global Accelerator, ALB
- CI/CD & Automation: AWS CodePipeline, AWS CodeBuild
Third-Party Services:
- Veeam Backup & Replication for backup and disaster recovery management.
Quantified Outcomes & Business Impact
- Reduced Downtime: Disaster recovery implementation minimized operational disruptions by 95%.
- Enhanced Data Resilience: Cross-Region Replication (CRR) ensured 99.999999999% durability, securing data across multiple AWS regions.
- Optimized Storage Costs: Incremental backups stored as AMIs resulted in up to 50% savings compared to traditional full backups.
- Accelerated Recovery: Leveraging Amazon S3 Standard for AMIs improved recovery time objectives (RTO) by 80%.
- Comprehensive Data Protection: The dual backup strategy covered both system-level and granular data protection, ensuring full business continuity.
Regulatory Compliance: The disaster recovery framework and security controls enabled the client to meet regulatory requirements, including PDPB, DPDP, and RERA.