Load balancing for AI services in Prompt Engineering / GenAI - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is load balancing in AI services?

Load balancing is the process of distributing incoming AI service requests evenly across multiple servers or resources to ensure no single server is overwhelmed, improving speed and reliability.

beginner

Why is load balancing important for AI services?

It helps keep AI services fast and available by preventing any one server from getting too busy, which can cause delays or crashes.

intermediate

Name two common load balancing methods used in AI services.

Round Robin (requests go to servers in order) and Least Connections (requests go to the server with the fewest active connections).

intermediate

How does load balancing improve fault tolerance in AI services?

If one server fails, load balancers redirect requests to other healthy servers, keeping the AI service running smoothly.

intermediate

What role does health checking play in load balancing for AI services?

Health checks monitor servers to ensure they are working well; load balancers use this info to avoid sending requests to servers that are down or slow.

What does load balancing do in AI services?

A Deletes old AI data

B Increases the size of AI models

C Reduces the number of AI service users

D Distributes requests evenly across servers

Load balancing spreads requests evenly to keep AI services fast and reliable.

Which load balancing method sends requests to the server with the fewest active connections?

A Least Connections

B Weighted Distribution

C Random Selection

D Round Robin

Least Connections sends requests to the server with the fewest active connections to balance load.

How does load balancing help if one AI server crashes?

A It redirects requests to other servers

B It shuts down all servers

C It deletes user data

D It slows down the service

Load balancers redirect requests to healthy servers to keep the service running.

What is a health check in load balancing?

A A method to speed up AI predictions

B A way to train AI models

C A test to see if servers are working well

D A tool to increase server storage

Health checks monitor server status to avoid sending requests to bad servers.

Which of these is NOT a benefit of load balancing for AI services?

A Better reliability

B Increased server crashes

C Improved speed

D Fault tolerance

Load balancing reduces crashes by spreading load, not increasing them.

Explain in your own words why load balancing is important for AI services.

Think about what happens if one server gets too many requests.

Describe two common methods of load balancing and how they decide where to send requests.

One method cycles through servers, the other checks how busy servers are.