HTTP and HTTPS in HLD - Scalability & System Analysis

Scalability Analysis - HTTP and HTTPS

Growth Table: HTTP and HTTPS Scaling

Users	Traffic Characteristics	Infrastructure Impact	Security Considerations
100 users	Low requests per second (RPS), mostly HTTP or HTTPS	Single server can handle all traffic	HTTPS overhead negligible, simple SSL cert management
10,000 users	Moderate RPS, mostly HTTPS for security	Need load balancer, multiple servers, SSL termination	SSL handshake CPU cost noticeable, use session resumption
1,000,000 users	High RPS, mostly HTTPS, many concurrent connections	Multiple load balancers, SSL offloading hardware or CDN, caching layers	Use TLS session tickets, OCSP stapling, strong cipher suites
100,000,000 users	Very high RPS, global distribution, HTTPS everywhere	Global CDN with SSL termination, edge caching, distributed load balancers	Automated cert management, strict security policies, DDoS protection

First Bottleneck

The first bottleneck is the SSL/TLS handshake CPU cost on the web servers or load balancers. HTTPS requires encryption and decryption which uses more CPU than HTTP. At moderate to high traffic, this CPU overhead limits how many connections a server can handle.

Scaling Solutions

• SSL Termination at Load Balancer or CDN: Offload encryption work from app servers.
• Use Session Resumption: TLS session tickets or session IDs reduce handshake cost.
• Horizontal Scaling: Add more servers behind load balancers to handle more connections.
• Global CDN: Cache static content and terminate HTTPS close to users to reduce latency and load.
• Hardware Accelerators: Use dedicated SSL/TLS hardware or optimized libraries for encryption.

Back-of-Envelope Cost Analysis

• At 10,000 users, assuming 1 request per second per user -> 10,000 RPS.
• Single server handles ~5,000 concurrent HTTPS connections due to CPU limits.
• Bandwidth: 1 Gbps network supports ~125 MB/s; HTTPS adds ~10-20% overhead.
• Storage for logs and certificates is minimal compared to traffic volume.
• SSL handshake CPU cost can be 10x higher than plain HTTP request processing.

Interview Tip

Start by explaining the difference between HTTP and HTTPS in terms of security and overhead. Then discuss how HTTPS adds CPU and latency costs due to encryption. Identify the SSL handshake as the first bottleneck. Propose offloading SSL to load balancers or CDNs and using session resumption. Finally, mention horizontal scaling and global CDNs for large scale.

Self Check Question

Your web server handles 1000 HTTPS requests per second. Traffic grows 10x to 10,000 RPS. What is your first action and why?

Answer: Offload SSL termination to a load balancer or CDN to reduce CPU load on the web server, because SSL handshakes consume significant CPU and limit concurrency.

Key Result

HTTPS adds CPU overhead due to encryption, making SSL/TLS handshake the first bottleneck as traffic grows. Offloading SSL termination and using CDNs are key to scaling securely.