College Course Curriculum

Designing Software for Scalability & High Concurrency

A rigorous exploration of how systems remain fast, correct, and available as load scales from thousands to millions of concurrent users.

Duration

16 Weeks

Modules

8 Modules

Level

Upper-div UG / Graduate

Format

Lecture + Lab + Capstone

This course equips students to design, build, and operate software systems that remain fast, correct, and available as load scales. Modules progress from concurrency fundamentals through distributed patterns, data storage, messaging, performance engineering, and cloud-native deployment.

Quizzes 20%Labs 30%Project 30%Final 20%

Overview

Course Structure at a Glance

ModuleTitleWeeksLevel
M1Foundations of Scalable Systems1–2Introductory
M2Concurrency Fundamentals & Threading Models3–4Foundational
M3Distributed System Design Patterns5–6Intermediate
M4Data Storage at Scale7–8Intermediate
M5Messaging, Streaming & Event-Driven Architecture9–10Inter–Advanced
M6Performance Engineering & Load Testing11–12Advanced
M7Cloud-Native Scalability & Production Readiness13–14Advanced
M8Capstone — End-to-End System Design15–16Synthesis
M1

Week 1–2 · Introductory

Foundations of Scalable Systems

IntroductoryWeeks 1–2
Establish the conceptual foundation for scalability and high concurrency. Students explore the core properties that distinguish scalable systems from fragile ones, and learn to reason about trade-offs using established frameworks.

Learning Objectives

  • →Define scalability and distinguish horizontal from vertical scaling
  • →Explain the CAP theorem and its practical implications
  • →Identify bottlenecks using Amdahl's Law and the Universal Scalability Law
  • →Describe the 8 Fallacies of Distributed Computing
  • →Read and produce basic system architecture diagrams

Topics & Content

Scalability vs. Performance vs. Reliability

Core definitions; latency, throughput, availability SLAs

Horizontal vs. Vertical Scaling

Trade-offs, cost curves, elasticity

CAP Theorem & PACELC

Consistency, Availability, Partition tolerance; practical design choices

Amdahl's Law & USL

Predicting speedup limits; coherency and contention penalties

Fallacies of Distributed Computing

Network reliability, latency, bandwidth, topology assumptions

Activities & Labs

  • Case study analysis: How Netflix moved from monolith to microservices
  • Whiteboard exercise: sketch a naive vs. scalable architecture for a URL shortener
  • Problem set: CAP theorem classification of 5 real systems
Assessment: Short-answer quiz on CAP theorem + 1-page system design sketch with rationale
M2

Week 3–4 · Foundational

Concurrency Fundamentals & Threading Models

FoundationalWeeks 3–4
A rigorous treatment of concurrency primitives—threads, locks, atomic operations, and memory models—alongside modern alternatives such as async/await and the actor model.

Learning Objectives

  • →Explain the difference between concurrency and parallelism
  • →Identify race conditions, deadlocks, livelocks, and starvation in code
  • →Apply mutexes, semaphores, read-write locks, and condition variables correctly
  • →Describe the Java Memory Model and the C++ memory ordering model
  • →Compare thread-per-request, event-loop, and async/await patterns

Topics & Content

Threads, Processes, Fibers, Coroutines

OS scheduling; user-space threading; M:N models

Synchronization Primitives

Mutex, RWLock, Semaphore, Condition Variable, Barrier

Memory Models & Visibility

Happens-before, volatile, atomic, memory fences

Lock-Free & Wait-Free Data Structures

CAS loops, ABA problem, hazard pointers

Async I/O & Event Loops

epoll/kqueue, Node.js event loop, async/await, structured concurrency

Actor Model & CSP

Erlang/Elixir actors, Go channels, Akka

Activities & Labs

  • Lab: Reproduce and fix a race condition in a provided bank-account simulation
  • Lab: Implement a bounded blocking queue using condition variables
  • Discussion: Review a post-mortem where a deadlock caused production outage
Assessment: Coding assignment: concurrent producer-consumer pipeline with correctness proof
M3

Week 5–6 · Intermediate

Distributed System Design Patterns

IntermediateWeeks 5–6
Students learn canonical patterns for building distributed systems that remain correct and available under partial failure.

Learning Objectives

  • →Apply the Saga pattern for distributed transactions without two-phase commit
  • →Design idempotent APIs and explain exactly-once semantics
  • →Compare leader-follower, multi-leader, and leaderless replication
  • →Implement circuit breakers, bulkheads, and timeouts to contain failures
  • →Explain consensus algorithms such as Raft and Paxos at a conceptual level

Topics & Content

Replication & Consensus

Raft log replication, leader election, linearizability

Distributed Transactions

2PC problems, Saga pattern, outbox pattern

Idempotency & Exactly-Once

Idempotency keys, deduplication windows, at-least-once + idempotency

Failure Isolation Patterns

Circuit breaker, bulkhead, timeout, retry with exponential backoff

Service Discovery & Load Balancing

Client-side vs server-side, consistent hashing, least-connections

Observability

OpenTelemetry tracing, structured logging, SLOs and error budgets

Activities & Labs

  • Design review: critique an architecture diagram for a payment processing service
  • Lab: Implement a circuit breaker and inject faults
  • Reading: Designing Data-Intensive Applications chapters 5–9
Assessment: Group project milestone 1: distributed system design document with trade-off analysis
M4

Week 7–8 · Intermediate

Data Storage at Scale

IntermediateWeeks 7–8
Explores how storage engines, database internals, and caching tiers enable high-throughput, low-latency data access.

Learning Objectives

  • →Explain B-tree and LSM-tree storage engines and their I/O trade-offs
  • →Design a sharding strategy for relational and NoSQL databases
  • →Apply read and write caching patterns
  • →Describe how indexes affect query performance
  • →Reason about isolation levels and when to relax consistency

Topics & Content

Storage Engine Internals

B-tree, LSM-tree, copy-on-write

Sharding & Partitioning

Range, hash, directory-based; hotspot avoidance

Replication Lag & Read Replicas

Read-your-writes, monotonic reads, bounded staleness

Caching Strategies

Cache-aside, write-through, write-behind, eviction policies

NoSQL Data Modeling

Denormalization, wide-column design, document modelling

Database Isolation Levels

Read uncommitted to serializable; performance vs correctness

Activities & Labs

  • Benchmark PostgreSQL vs Cassandra for a time-series workload
  • Design exercise: shard a social media followers table
  • Measure cache hit rates and implement early expiration
Assessment: Technical report: storage architecture recommendation with benchmarks
M5

Week 9–10 · Intermediate–Advanced

Messaging, Streaming & Event-Driven Architecture

Inter–AdvancedWeeks 9–10
Covers asynchronous communication as a first-class architectural concern, including queues, streams, and event-driven operations at scale.

Learning Objectives

  • →Distinguish message queues, event streams, and pub/sub systems
  • →Explain at-most-once, at-least-once, and exactly-once delivery semantics
  • →Design a Kafka topic partition strategy
  • →Apply event sourcing and CQRS
  • →Handle consumer groups, offsets, rebalancing, and backpressure

Topics & Content

Message Queues vs Event Streams

RabbitMQ, SQS, Kafka, Kinesis; ordering guarantees

Kafka Deep Dive

Partition leadership, ISR, consumer groups, log compaction

Delivery Semantics

Producer acks, idempotent producers, transactional producers

Event Sourcing & CQRS

Append-only event log, projections, eventual consistency

Backpressure & Flow Control

Reactive Streams, bounded queues

Schema Evolution

Avro, Protobuf, schema registry, compatibility

Activities & Labs

  • Build a Kafka producer-consumer pipeline and measure latency
  • Design an event-driven order-processing system
  • Debate: when event sourcing helps and when it hurts
Assessment: Lab report and design diagram for event-driven order fulfilment with failure analysis
M6

Week 11–12 · Advanced

Performance Engineering & Load Testing

AdvancedWeeks 11–12
Teaches a systematic methodology for measuring, profiling, and improving concurrent and distributed system performance.

Learning Objectives

  • →Design statistically valid load tests using Little's Law
  • →Identify CPU, memory, I/O, and lock contention bottlenecks
  • →Interpret flame graphs, latency percentiles, and throughput curves
  • →Apply USE and RED methods
  • →Conduct capacity planning from first principles

Topics & Content

Load Testing Methodology

Little's Law, workload models, coordinated omission

Profiling Tools

perf, async-profiler, pprof, py-spy, eBPF

Flame Graphs & Latency

p50/p95/p99/p999 and HDR Histogram

USE & RED Methods

Utilisation, Saturation, Errors; Rate, Errors, Duration

JVM & GC Tuning

GC pause analysis, heap sizing, ZGC/Shenandoah

Capacity Planning

Demand forecasting, headroom, cost modelling

Activities & Labs

  • Load test a sample microservice with Gatling
  • Generate and interpret flame graphs
  • Capacity plan a Black Friday traffic spike
Assessment: Performance audit: profile a service, identify bottlenecks, propose and validate fixes
M7

Week 13–14 · Advanced

Cloud-Native Scalability & Production Readiness

AdvancedWeeks 13–14
Integrates earlier modules into cloud-native deployment patterns using Kubernetes, serverless, managed cloud services, and production readiness methods.

Learning Objectives

  • →Configure Kubernetes autoscaling using custom metrics
  • →Compare serverless, container, and VM deployment trade-offs
  • →Design multi-region active-active architecture
  • →Apply chaos engineering to validate resilience
  • →Build a runbook and on-call checklist

Topics & Content

Kubernetes Scaling Internals

HPA, VPA, KEDA, cluster autoscaler

Serverless & FaaS at Scale

Cold starts, concurrency limits, provisioned concurrency

Multi-Region Architecture

Active-active, active-passive, global load balancing

Chaos Engineering

GameDay design, fault injection patterns

Security at Scale

mTLS, zero trust, secrets, rate limiting, DDoS

Production Readiness Review

Runbooks, on-call, post-mortem culture

Activities & Labs

  • Deploy a service to Kubernetes and configure HPA
  • Inject network latency and node failure; measure MTTR
  • Architecture review of group project designs
Assessment: Production readiness review document for the group project system
M8

Week 15–16 · Advanced / Synthesis

Capstone — End-to-End System Design

SynthesisWeeks 15–16
Students synthesize all prior learning in a team capstone project, designing and defending a complete scalable system architecture.

Learning Objectives

  • →Produce a complete system design document covering scalability dimensions
  • →Justify architectural decisions with quantitative trade-off analysis
  • →Demonstrate concurrency, storage, messaging, and observability choices
  • →Present and defend design decisions
  • →Reflect on limitations and future evolution

Topics & Content

Ride-sharing dispatch system

Scenario option

Streaming video CDN

Scenario option

Global chat application

Scenario option

Financial exchange matching engine

Scenario option

Distributed ML training pipeline

Scenario option

Activities & Labs

  • Design document — 8–12 pages
  • Architecture diagrams with component interaction
  • Capacity estimate spreadsheet
  • 20-minute presentation and Q&A
  • Peer evaluation
Assessment: Design document + live presentation + peer evaluation