Understanding System Design: Scaling, Load Balancing, and Consistent Hashing

Throughput can be defined as the total volume/input a system can process given a certain time period.

A system is resilient when it has zero single point of failures. For example, if a teacher calls in sick, a substitute will teach the class.

Load balancing is the process of distributing the load between two systems. For instance, imagine a single server handling a large amount of requests. Having a load balancer can distribute requests evenly between servers to ensure smooth experiences.

Vertical scaling involves individually increasing the amount of computing power for each resource.

Pros:

• Fast Communication (due to large single resource)
• High data consistency

Cons:

• Single Point of Failure
• No load balancing
• Hard to scale (limited by hardware)

Horizontal scaling increases the total computing power by adding more resources/computers.

Pros:

• System is more resilient
• Effective load balancing
• Easily scalable

Cons:

• Slow communication
• Low consistency with data

Consistent Hashing is the concept of creating a hashing function which maps to each server to implement load balancing. Each key maps an associated value to a hash ring containing nodes, and each value is assigned to the first node it encounters following a clockwise direction.

In this example, if Node 1 fails, Harsh, Kamal, and Ram will all be served by Node 2. This may overload Node 2, but a workaround is to have virtual nodes scattered around the ring.