Name and Sector of Client:

• Sector: Retail

Background:

Fable Street is a leading E-commerce platform, the first large-format female fashion retail store, revolutionized the shopping experience for fashion enthusiasts by offering a comprehensive range of products. With over 130 stores across India and a significant online presence, they built a strong reputation for providing the latest technology, personalized service, and a seamless shopping experience.

However, as consumer demands surged and digital transformation became imperative, they faced challenges such as scaling its infrastructure to handle increased traffic, ensuring high availability, and optimizing operational costs. To address these issues, as a cloud vendor for them we used multiple other AWS services leveraging its scalability, reliability, and cost-efficiency to enhance customer experiences and drive business growth.

Overview of Challenges in Companies Existing Architecture:

The company's existing architecture, primarily based on AWS EC2 instances, presented several challenges that led to the decision to transition its applications to Amazon ECS. While the initial focus was on cost optimization, a detailed review of the AWS account revealed key issues:

• Fault Management: The existing setup lacked robust fault tolerance mechanisms, making it challenging to isolate and recover failures. Manual intervention was often required for managing outages, increasing downtime, and operational overheads.
• Chaos Management: The architecture struggled to handle unpredictable traffic spikes, leading to inconsistent application performance. Without automated scaling and load distribution, the system faced difficulties in maintaining high availability during peak loads.
• Security: The security practices were fragmented, with limited visibility into the overall posture. Managing updates, patches, and access controls across multiple EC2 instances was error-prone and time-consuming, exposing the system to potential vulnerabilities.
• Scalability: The EC2-based setup lacked seamless scalability. Adding or removing resources required manual provisioning and configuration, resulting in inefficiencies and delays in meeting dynamic business demands.
• Resource Optimization: Overprovisioning of EC2 instances to handle peak traffic resulted in underutilized resources during non-peak times, leading to unnecessary costs without delivering optimal performance.
• Operational Complexity: Deploying and managing applications across multiple EC2 instances introduced complexities in versioning, monitoring, and orchestration, making it harder to ensure consistent deployments.
• Monitoring and Observability: Limited monitoring capabilities made it difficult to gain actionable insights into application behavior, hindering proactive issue detection and resolution.
• Feature Update Complexity: The architecture lacked streamlined mechanisms for rolling out new features and updates. Updating applications across multiple EC2 instances requires significant manual effort, often resulting in deployment delays, inconsistent configurations, and potential downtime during updates.

Transitioning to Amazon ECS enabled the company to address these challenges effectively by leveraging containerized applications, automated scaling, improved fault isolation, enhanced security features, and centralized management for a more resilient, cost-effective, and scalable architecture.

Architecture Diagram of Proposed Solution –

The Applications Overview:

The FableStreet e-commerce platform is a modular system comprising multiple microservices, each responsible for a core business function. This service-oriented architecture ensures scalability, maintainability, and a seamless user experience.

Product Catalogue Service (FableStreet-prod-product-svc)

Overview:

The Product Catalogue Service acts as the central repository for all product-related data, supporting browsing, searching, and filtering of items in the store.

Functionality:

• Maintains product details such as name, description, price, availability, and images.
• Supports category-based browsing, search, and filter features.
• Integrates with inventory systems to reflect real-time stock levels.
• Exposes REST APIs for use by front-end (UI) and other backend services like Cart and Order.

Purpose:

To ensure comprehensive, real-time access to product information and enable a user-friendly product discovery experience.

User Profile Service (FableStreet-prod-user-svc)

Overview:

The User Service manages user information and preferences to enable personalization and secure access.

Functionality:

• Handles user registration, authentication, and authorization.
• Stores profile information such as contact details and delivery addresses.
• Tracks user behavior and purchase history to personalize experiences.
• Provides secure endpoints for profile updates and account management.

Purpose:

To deliver a personalized and secure shopping journey, building trust and customer loyalty.

Cart Service (FableStreet-prod-cart-svc)

Overview:

The Cart Service allows users to manage products they intend to purchase.

Functionality:

• Supports adding, updating, and removing items from the cart.
• Calculates dynamic totals including discounts and taxes.
• Ensures cart persistence across user sessions.
• Validates product availability and pricing in real-time through integration with the Product Catalogue Service.

Purpose:

To offer a seamless and intuitive pre-checkout experience, improving conversion rates.

Order Management Service (FableStreet-prod-order-svc)

Overview:

The Order Service handles the lifecycle of customer orders from placement to delivery and returns.

Functionality:

• Manages order creation, confirmation, and status tracking.
• Sends real-time updates on order progress.
• Coordinates with the Payment Service for transactions and refund processing.
• Supports return and exchange workflows.

Purpose:

To ensure smooth and transparent post-purchase operations and customer satisfaction.

Payment Gateway Service (FableStreet-prod-payment-svc)

Overview:

The Payment Service manages all aspects of secure payment processing.

Functionality:

• Enables multiple payment methods: cards, wallets, UPI, and net banking.
• Provides encryption, tokenization, and PCI DSS compliance for secure transactions.
• Handles transaction success/failure states and integrates with order and refund workflows.

Purpose:

To facilitate secure, reliable, and fast payment experiences for users.

UI Frontend (FableStreet-prod-ui)

Overview:

This is the front-end application that interacts with users and renders the e-commerce interface.

Functionality:

• Built using TypeScript and modern frontend frameworks.
• Connects to backend APIs for product listing, cart updates, user authentication, and checkout.
• Provides responsive and user-friendly interfaces for browsing and transactions.

Purpose:

To deliver a smooth and visually engaging customer experience on web platforms.

Shared Libraries (FableStreet-prod-shared)

Overview:

The Shared module hosts reusable components and utilities across services.

Functionality:

• Common DTOs, error handling utilities, logging, and middlewares.
• Ensures code reusability and consistent standards across microservices.

Purpose:

To promote maintainability, reduce redundancy, and enhance code quality across the platform.

Benefits of Proposed ECS Architecture:

The proposed architecture leverages a robust combination of AWS services to address the challenges faced in the existing EC2-based architecture. The design emphasizes fault management, chaos management, scalability, security, resource optimization, operational simplicity, and monitoring. Below is a summary of the proposed architecture and how the various services contribute to solving the identified issues:

1. Fault Management and Chaos Management:

1. AWS Fault Injection Simulator (FIS):

• Helps test the system's resilience by introducing controlled chaos experiments, such as simulating ECS task failures.
• Validates fault tolerance mechanisms and ensures the application gracefully handles outages and recovers automatically.

How It Helped:

• FIS introduced simulated failures (e.g., ECS task interruptions, network throttling) to evaluate the system's fault tolerance.
• The tests validated the application's ability to handle disruptions gracefully without affecting user experience, ensuring recovery mechanisms (like auto-scaling or task restarts) functioned as expected.
• FIS ensured the application architecture was resilient under various failure scenarios.

2. AWS Fargate for ECS:

• Eliminates the need to manage underlying EC2 instances, providing fault isolation at the container level.
• Automatically provisions resources for containers, reducing downtime during failures.

How It Helped:

• Containers running on Fargate were isolated from underlying infrastructure failures, reducing system-wide impact.
• It replaces failed containers, automatically reducing downtime.
• Customers could focus on application improvements instead of infrastructure maintenance.

2. Security:

1. AWS Web Application Firewall (WAF):

• Protects against common web threats like SQL injection, cross-site scripting, and DDoS attacks.

How It Helped:

• WAF blocked malicious traffic, such as SQL injections and DDoS attacks, before it reached the application (e.g., excessive requests from a single IP or user agent). It enforces limits on the number of requests allowed per IP or session over a specific timeframe. Also restricted traffic for specific geographical regions.
• Tailored WAF rules protected application endpoints against specific vulnerabilities.
• Helped meet security standards required for sensitive applications (e.g., payment systems).

2. AWS Certificate Manager (ACM):

• Manages TLS 1.2/1.3 certificates to secure communication between users and applications.

How It Helped:

• ACM handled the issuance, renewal, and deployment of TLS certificates without manual intervention.
• Secured data in transit using TLS 1.2/1.3, protecting sensitive user and payment data.
• Integrated easily with ALB and ECS for secure application access.

Illustrative Diagram of Same:

Below is the first time the page is accessed we will sign up as a new user.

After signing in to the ALB, it redirects you to the landing page of the application, which is shown in the diagram below.

1. Container Orchestration and Compute

AWS ECS with Fargate:

• Service: Container orchestration platform using AWS Fargate for serverless container management
• Configuration:
  • 5 microservices deployed: UI, User, Cart, Product, Order
  • Each service allocated 256 CPU units and 512MB memory
  • Fargate eliminates the need to manage underlying EC2 instances
• Benefits:
  • Pay-per-use pricing model reduces costs
  • Automatic scaling based on demand
  • No server management overhead
  • Built-in high availability across multiple AZs

Elastic Load Balancer (ALB):

• Service: Application Load Balancer for traffic distribution
• Configuration: HTTPS listener with SSL certificate, health checks enabled
• Benefits:
  • Distributes traffic across healthy ECS tasks
  • Automatic failover to healthy instances
  • SSL termination and certificate management
  • Path-based routing for different services

2. Database and Data Storage

Amazon RDS PostgreSQL:

• Service: Managed PostgreSQL database
• Configuration:
  • Instance: db.t3.micro (free tier eligible)
  • Storage: 20GB GP3 storage
  • Multi-AZ deployment for high availability
  • Automated backups enabled
• Benefits:
  • Managed database service reduces operational overhead
  • Automatic backups and point-in-time recovery
  • Built-in security with encryption at rest

Amazon ElastiCache Redis:

• Service: Managed Redis cluster for caching
• Configuration:
  • Node type: cache.t2.micro (free tier eligible)
  • Single node cluster for development
  • In-memory caching for session management and cart data
• Benefits:
  • High-performance caching layer
  • Reduces database load
  • Managed service with automatic failover

Amazon DocumentDB:

• Service: MongoDB-compatible managed database
• Configuration:
  • Instance class: db.t4g.medium
  • Cluster for product catalog and order management
  • JSON document storage for flexible schema
• Benefits:
  • Native MongoDB compatibility
  • Automatic scaling and backups
  • Built-in security and encryption

Amazon OpenSearch Service:

• Service: Managed search and analytics engine
• Configuration:
  • Single node deployment for development
  • Full-text search capabilities for product catalog
• Benefits:
  • Advanced search functionality
  • Real-time analytics and monitoring
  • Managed service with automatic updates

3. Networking and Security

Amazon VPC:

• Service: Virtual Private Cloud for network isolation
• Configuration:
  • Public subnets for ALB
  • Private subnets for ECS tasks and databases
  • NAT Gateway for outbound internet access
• Benefits:
  • Network isolation and security
  • Controlled internet access
  • Custom routing and security groups

Security Groups:

• Configuration:
  • ALB Security Group: Allows HTTP/HTTPS from internet
  • ECS Tasks Security Group: Allows traffic from ALB only
  • Database Security Groups: Restrict access to ECS tasks
• Benefits:
  • Network-level security controls
  • Principle of least privilege
  • Defense in depth

AWS Certificate Manager (ACM):

• Service: SSL/TLS certificate management
• Configuration: Wildcard certificate for domain
• Benefits:
  • Free SSL certificates
  • Automatic renewal
  • Centralized certificate management

4. CI/CD Pipeline

AWS CodePipeline:

• Service: Continuous integration and deployment
• Configuration:
  • V2 pipeline with GitHub integration
  • Automated build and deployment for all services
  • CodeStar Connection for GitHub integration
• Benefits:
  • Automated deployment workflows
  • Consistent deployment process
  • Integration with GitHub repositories

AWS CodeBuild:

• Service: Build service for container images
• Configuration:
  • Multi-stage Docker builds
  • ECR image pushing
  • Build artifacts management
• Benefits:
  • Automated container image building
  • Integration with ECR
  • Scalable build infrastructure

Amazon ECR:

• Service: Container registry
• Configuration:
  • Private repositories for each service
  • Image scanning and vulnerability detection
• Benefits:
  • Secure container image storage
  • Integration with ECS
  • Image lifecycle management

5. Monitoring and Observability

AWS CloudWatch:

• Service: Monitoring and logging
• Configuration:
  • ECS service metrics
  • Application logs collection
  • Custom dashboards
• Benefits:
  • Centralized monitoring
  • Real-time metrics and logs
  • Automated alerting

6. IAM Roles and Policies

Service Roles:

• ECS Task Execution Role: Allows ECS to pull images and write logs
• CodePipeline Role: Permissions for pipeline operations
• CodeBuild Role: Permissions for building and pushing images
• Configuration: Liberal permissions for development environment
• Benefits:
  • Secure access control
  • Principle of least privilege (configurable)
  • Centralized permission management

Cost Optimization Strategies

• Fargate: Pay only for resources used
• Auto Scaling: Scale down during low traffic
• Storage: GP3 storage for better price/performance
• Networking: Single NAT Gateway to reduce costs

Security Implementation

1. Network Security

• VPC Isolation: All resources in private subnets
• Security Groups: Restrictive access controls
• SSL/TLS: End-to-end encryption

2. Data Security

• Encryption at Rest: All databases encrypted
• Encryption in Transit: HTTPS for all communications
• IAM: Role-based access control

3. Application Security

• Container Security: ECR image scanning
• Secret Management: Environment variables for sensitive data
• Monitoring: CloudWatch for security events

Scalability Features

1. Horizontal Scaling

• ECS Auto Scaling: Based on CPU/memory utilization
• ALB: Distributes traffic across multiple instances
• Database: Read replicas for scaling (production)

2. Vertical Scaling

• Fargate: Easy resource adjustment
• RDS: Instance class upgrades
• ElastiCache: Node type scaling

3. Load Distribution

• ALB: Health checks and failover
• Target Groups: Service-specific routing
• Path-based Routing: Different services on same ALB

Operational Benefits

1. Managed Services

• Reduced Overhead: AWS manages infrastructure
• Automatic Updates: Security patches and updates
• High Availability: Multi-AZ deployments

2. Automation

• CI/CD Pipeline: Automated deployments
• Infrastructure as Code: Terraform for reproducible deployments
• Monitoring: Automated alerting and logging

3. Developer Experience

• Container-based: Consistent development environment
• GitHub Integration: Direct code deployment

Why This Architecture is Optimal

1. Cost Efficiency

• Serverless: Pay only for resources used
• Free Tier: Maximize free tier benefits
• Auto Scaling: Scale down during low usage

2. Scalability

• Microservices: Independent scaling of services
• Container-based: Easy deployment and scaling
• Managed Services: AWS handles scaling complexity

3. Security

• Defense in Depth: Multiple security layers
• Network Isolation: VPC and security groups
• Encryption: Data encrypted at rest and in transit

4. Operational Excellence

• Infrastructure as Code: Reproducible deployments
• CI/CD: Automated deployment pipeline
• Monitoring: Comprehensive observability

Why was ECS the most optimum solution:

• Containerization Advantage: ECS leveraged containerized workloads, enabling better resource isolation, scaling, and deployment consistency compared to EC2 instances.
• Managed Infrastructure: By using Fargate, ECS abstracted away infrastructure management, allowing the company to focus on application development.
• Seamless Integration: ECS integrated seamlessly with AWS services like WAF, ACM, IAM, CloudWatch, and X-Ray, addressing security, observability, and scaling challenges effectively.
• Cost Efficiency: ECS's pay-as-you-go pricing model and auto-scaling capabilities optimized resource utilization, reducing unnecessary costs.
• Operational Simplicity: ECS's container-first approach simplified deployments, scaling, and fault management, resolving the complexities faced in the EC2 architecture.

By transitioning to ECS, the customer achieved a modern, resilient, and cost-effective architecture tailored to their specific needs, solving all challenges identified in their legacy EC2-based setup.

Disaster Recovery (DR) Setup –

To ensure business continuity and protect against data loss, we prioritized Disaster Recovery (DR) as a critical component of their ECS architecture. This strategy focuses on ensuring zero downtime even in the event of unforeseen incidents or failures.

Implemented Plan:

Multi-Region Architecture –

• To provide high availability and minimize the risk of service disruption, ECS services have been deployed across two AWS regions: Mumbai (primary region) and Hyderabad (secondary region). This multi-region setup ensures that if one region experiences an issue, they can seamlessly failover to another region, thereby maintaining uninterrupted service for users.
• To facilitate traffic routing between regions, Amazon Route 53 is leveraged, utilizing health checks to monitor the status of services. In the event of a failure in the primary region, Route 53's failover routing capabilities automatically redirect traffic to the secondary region, ensuring that service remains operational without requiring manual intervention.

ECR Replication for Container Availability –

• To further enhance the resiliency of the ECS infrastructure, Amazon Elastic Container Registry (ECR) has been configured with cross-region replication. This ensures that container images stored in ECR are available in the secondary region, allowing ECS to launch containers from the replicated images in case of a failure in the primary region.
• This replication ensures that containerized applications are always ready to be deployed, even in a disaster scenario.

Database Replication –

• To safeguard data and ensure resilience, we have implemented cross-region read replicas for their Aurora PostgreSQL database. This approach enables asynchronous replication of the database from the primary region (Mumbai) to the secondary region (Hyderabad).
• The cross-region read replica ensures that data is continuously replicated, providing data protection, read scalability and disaster recovery readiness.

Testing and Validation –

• Ensuring the effectiveness of the DR strategy is vital. They conducted periodic (every 6 months) DR drills to test and validate the failover capabilities of the architecture. This ensures that the system is prepared for real-world scenarios, with all components functioning as expected during a failure.

The goal of this DR strategy was to minimize downtime and protect against data loss in the event of an unforeseen incident. By implementing this multi-region architecture, ECR replication, database snapshotting, and failover testing ensures that both its customers' trust and its own business operations are safeguarded, even during major incidents.

CodePipeline Security Enhancements

As e-commerce platform scales its operations, ensuring the security and integrity of deployments becomes paramount. To fortify the CodePipeline and reduce the risk of unauthorized or faulty deployments, several key security measures were implemented.

1. Least-Privilege IAM Roles for Pipeline Stages –

To ensure the principle of least privilege, each stage of the pipeline was assigned specific IAM roles. These roles provide only the minimal set of permissions required for execution, limiting access to resources and preventing unnecessary exposure of sensitive infrastructure. This approach significantly reduces the attack surface and ensures that only authorized entities can perform actions within the pipeline.

2. Encryption of Build Artifacts Using AWS KMS –

To ensure the confidentiality and integrity of build artifacts during pipeline operations, we encrypted the artifacts using AWS Key Management Service (KMS). This encryption protects sensitive data stored in Amazon S3, both at rest and in transit. By utilizing a multi-region KMS key, we maintained control over encryption keys and enhanced the security of its CI/CD processes across regions.

3. Manual Approval Stages for Production Deployments –

To add an additional layer of protection against unauthorized or accidental changes, manual approval stages were introduced before production deployments. This ensures that no changes are automatically pushed to production without explicit authorization. The approval process acts as a safeguard, allowing teams to carefully review changes before they go live and ensuring that deployments align with business and security requirements.

4. Third-Party Vulnerability Scanning with Snyk and ECR –

Recognizing the importance of proactive security, we integrated third-party tools like Snyk and enabled ECR vulnerability scanning within the pipeline. These tools automatically scanned container images for vulnerabilities, allowing to detect and mitigate security issues early in the development process. This proactive approach helps identify vulnerabilities before they make their way into production, minimizing the potential for exploitation.

Together, these measures have significantly enhanced the reliability of the CodePipeline. By securing the pipeline, introducing manual approvals, and leveraging automated security scanning, we have minimized operational risks while fostering greater confidence in deployment integrity.

Terraform Versioning and Security –

To bolster cloud infrastructure and streamline deployments, we centralized all Terraform configurations into a dedicated version-controlled repository within the primary AWS account where the application is present. This change allows for a more organized and secure approach to managing infrastructure as code, with a focus on versioning, access control, and security.

Git Repository as the Central Repository for Terraform Configurations

Rather than relying on manual infrastructure management, we opted to store all Terraform configurations in a dedicated Git repository within the primary AWS account where the application is hosted. This approach allows for centralized access and management of infrastructure configurations, leveraging the scalability and durability of Git version control to securely store and organize them.

The Git repository is configured with versioning enabled, allowing the team to maintain the latest configuration versions while preserving previous iterations. This ensures that changes are easily tracked, and previous versions can be rolled back if necessary, facilitating robust change management.

Cross-Account Roles for Infrastructure Deployment

In this architecture, the primary account, which hosts the application, stores the Terraform configurations in a Git repository used for infrastructure deployment. In contrast, the secondary account holds another S3 bucket that contains files required by the application code.

To ensure secure and controlled access to the necessary resources in the secondary account, cross-account IAM roles were configured. These roles enabled the primary account to access the required files stored in the secondary account's S3 bucket, ensuring that the application in the primary account can seamlessly use these resources during its operation.

This setup promotes a clear segregation of resources between the accounts, maintaining strong security practices by adhering to the principle of least privilege. Only authorized roles from the primary account are allowed access to the specific resources in the secondary account, ensuring that resources are managed securely and efficiently across the two accounts.

Lifecycle Management for Terraform Configurations

To ensure the optimal management of Terraform configurations and files, lifecycle policies were implemented in the S3 bucket of both accounts. This policy automatically eliminates outdated configurations, files and images that are no longer required, preventing unnecessary clutter and reducing the potential for errors due to using deprecated configurations. The policy ensures that only configurations in active use are retained, improving operational efficiency.

Object Lock with Governance Policy for Enhanced Security

As part of enhancing data integrity, Object Lock with a Governance policy was applied to the S3 bucket. This ensures that no users, except for the root user, can delete configurations stored in the bucket, providing a safeguard against accidental or unauthorized deletions. With Object Lock in place, we ensure the integrity and availability of Terraform configurations, especially critical for compliance and auditing purposes.

Automated Infrastructure Creation Using Terraform

Once the configurations are securely stored and managed in the Git repository, they are used to create infrastructure in the primary account via Terraform deployments. By leveraging the central repository in Git ensures that the latest, validated versions of configurations are consistently used across deployments. This approach simplifies infrastructure automation, ensuring that deployments remain repeatable, reliable, and scalable.

Security and Compliance Monitoring

To strengthen the security and ensure compliance with organizational policies, we have implemented a comprehensive monitoring strategy using AWS Config and AWS CloudTrail. These services are essential for maintaining infrastructure security and meeting compliance requirements.

AWS Config continuously monitors and evaluates the configurations of AWS resources, automatically detecting any deviations from predefined security best practices. For example, it can flag overly permissive IAM roles, unsecured S3 buckets, and non-compliant security group settings. If any resource fails to comply with security policies, AWS Config can trigger notifications, helping the team to respond swiftly and rectify any issues.

AWS CloudTrail logs every API call made within the AWS environment, providing a comprehensive trail of activity. This includes tracking all Terraform deployments, such as infrastructure deployments, updates, and deletions. CloudTrail's detailed logs enable real-time visibility into resource management and configuration changes, ensuring that any unauthorized or suspicious activities are promptly detected.

Together, these services provide a robust mechanism for auditing, reporting, and ensuring that security policies are enforced. They also allow for proactive risk management, helping to identify and mitigate potential vulnerabilities before they become threats.

Outcomes and Achievements:

By transitioning to Amazon ECS, e-commerce platform successfully addressed key challenges in its legacy EC2-based architecture. The new setup improved fault tolerance and resilience through AWS Fargate and Fault Injection Simulator, enabling seamless recovery and handling of unpredictable traffic surges. Security was significantly enhanced with WAF, Cognito, and IAM policies, while scalability and cost-efficiency were achieved through Fargate's dynamic resource allocation. Operational complexity was reduced with automated deployments using CodePipeline, and monitoring capabilities were strengthened with CloudWatch and X-Ray. The Disaster Recovery strategy, including multi-region architecture, ECR replication, and database replication, ensured minimal downtime and data protection. These changes led to a more secure, scalable, and cost-optimized infrastructure, improving overall operational efficiency and user experience.