Skip to main content

Load Balancer

A Load Balancer creates a shared entry point for your application and distributes requests among several instances. For example, if your website runs on three instances, users send requests only to the Load Balancer's address, and the Load Balancer delivers each request to one of the healthy instances.

Use a Load Balancer when a website, API, or internal service runs on several instances and you want to:

  • have a shared entry address,
  • split the request load among several destinations,
  • stop sending traffic to a destination that fails its health check,
  • create an internal access point for private services without making each instance public.

How It Works​

The Load Balancer receives traffic through a Listener. The Listener specifies the protocol and port the Load Balancer waits for requests on. Each Listener has a Server Pool that contains the destination instances, called Members. The Server Pool specifies which Member port the request is sent to and with which algorithm. For example, you can send incoming HTTP requests on port 80 to port 8080 of the instances.

Each Load Balancer has a virtual address (VIP) in the private network. If you create an External Load Balancer, the Elastic IP attached to this address is the public entry point, and requests from the internet reach it. In Internal mode, the entry point is the Private VIP itself, and requests must reach it through allowed paths inside the VPC.

The Health Check periodically checks the health of Members and doesn't run again for each request. When a request reaches the Listener, the Load Balancer only considers healthy Members, and the selected algorithm chooses one of them. The request is then sent over the private network to the Member's IP and port, and the response returns to the requester through the Load Balancer. Members don't need a public IP to receive traffic from the Load Balancer.

Before You Start​

To create a Load Balancer, prepare the following:

  • A VPC in the region you want,
  • A Subnet for the Load Balancer and Members,
  • At least one ready instance in the same Subnet, with the application running on its private IP and destination port,
  • Network rules that allow the Load Balancer's traffic to reach the instances' destination port,
  • A decision on whether the entry point is public or private,
  • A Health Check path that gives a reliable, repeatable response, such as /health.

Before creating the Listener, check from one of the instances that the application responds on the private address and the port you want, and that the Health Check response succeeds only when the application is ready to receive requests. Also make sure the operating system firewall and VPC security rules don't block the Load Balancer's traffic.

Creating a Load Balancer​

In the Cloud Server panel, go to Load Balancer and click Create Load Balancer. You set the deployment location, size, network, Listener, Members, and Health Check on this page.

Choosing the AZ​

In the AZ section, select the data center where you want the Load Balancer created. The Load Balancer and the destination instances must be in the same Subnet and zone, so choose the zone based on where your application's instances are.

Choosing the Load Balancer Size​

In the Load Balancer Size section, choose the capacity that fits your service. Each option shows the following information:

  • Concurrent Connection

    The capacity for concurrent connections, meaning the number of connections that stay open at the same time.

  • SSL Connections Per Second

    The capacity for establishing SSL connections per second.

  • Price

    The amount shown for the selected option. The cost of related resources, such as an Elastic IP, is calculated separately.

The right capacity depends on your application's usage pattern. For example, long-lived connections increase the number of concurrent connections, so don't choose the size based only on the number of visitors.

Setting the Access Type and Network​

In the Network Access section, specify who should have access to the Load Balancer:

  • External

    For a website, public API, or service that must be reachable from the internet. The entry point of this type is the Elastic IP attached to the Load Balancer. Members can have only private addresses, and a separate public IP isn't needed for each one.

  • Internal

    For communication between services or private layers inside the VPC network. The entry point of this type is the Private VIP, and it isn't designed to receive public traffic from the internet. Users must reach it from inside the VPC through an allowed network path.

Then, in the VPC and Subnet section, first select the VPC and then the Subnet you want. Members must be in the same VPC and Subnet.

Choosing External doesn't make the destination instances public. Traffic is received at the public entry point and reaches the Members over the private network.

Note that Internal doesn't mean the resources are free, the application is automatically secure, or it's accessible from any network; you still need to set up the network path and access rules correctly.

Setting Up the Listener​

In the Listener section, specify the incoming protocol and port. At least one Listener must be defined.

  • Protocol

    Choose the incoming protocol that fits your application from the panel's options.

  • Port

    The incoming port must be a number between 1 and 65535.

For example, to receive HTTP traffic on port 80, select the HTTP protocol and port 80. The port number alone doesn't determine the protocol or encryption of the connection. For example, entering port 443 alone doesn't enable HTTPS.

The following are shown in Advanced Settings:

  • Client Data Timeout

    The wait time for data on the user side. The unit shown is milliseconds.

  • Member Data Timeout

    The wait time for data on the destination side. The unit shown is milliseconds.

  • Maximum Connections

    The connection limit for the Listener. This value doesn't increase the Load Balancer's selected capacity.

If the panel shows a duplicate error for the incoming port, edit the existing Listener or choose another port.

Setting Up the Server Pool​

In the Server Pool section, specify the application's port on the destination instances and the traffic distribution algorithm.

  • Destination port

    The port the application listens on on the Members. This port can differ from the Listener's incoming port, for example, incoming port 80 and destination port 8080. Enter the same port the application is running on.

  • Algorithm

    How the next Member to receive a request is chosen.

  • Session Persistence

    If the application must always send requests from the same source IP to the same Member, review the Session Persistence based on source IP option and enable it if needed.

Choose the algorithm from these options:

  • Round Robin

    Selects Members in rotation. Suitable for Members with roughly similar capacity and short, similar requests.

  • Least Connections

    Selects the Member with fewer active connections. Suitable for requests whose connection durations vary.

  • Source IP

    Selects the Member based on the source IP. This mapping may change if the set of Members changes. Suitable for scenarios that depend on the source IP.

Note that users behind NAT may share an IP and be directed to the same Member. No algorithm, not even Source IP or Session Persistence, is on its own a substitute for proper session design, shared storage, or application state management. If your application relies on an instance's local state, review its session behavior before choosing the algorithm.

Selecting the Destination Instances​

In the Instance section, select at least one instance for the Server Pool. Only instances in the selected VPC and Subnet are shown in the list. Find the instance you want in the search box and check its name, private IPv4 address, and weight. If the list is empty, check the selected VPC and Subnet and whether instances exist in them.

Add Members with the private IP and the application's actual port. A Member whose port the application isn't running on isn't considered healthy, even if the instance is on.

Weight is a Member's relative share in traffic distribution. If the instances have similar capacity, start with equal weights. A higher weight doesn't increase the instance's own capacity.

Setting Up the Health Check​

The Health Check answers the question: "Is this Member ready to receive traffic right now?" This check doesn't show that the whole application is correct, that the data is accurate, or that business dependencies are healthy.

Set the following fields in the Health Check section:

  • Protocol

    How the Member is checked. Match the check type to the application's needs. For example, responding to ping only shows network reachability and doesn't prove the web application is ready.

  • Maximum failed attempts

    The number of consecutive failed checks before the Member is considered unhealthy.

  • Check interval (seconds)

    The time between health checks. A shorter interval increases the number of monitoring requests.

  • Timeout (seconds)

    The maximum wait time for each check's response. Match this value to the application's normal response time.

  • Maximum retries

    The number of consecutive successful checks before the Member is considered healthy again.

Note that the Timeout and check interval in this section are in seconds. Don't confuse them with the Listener timeouts, which are shown in milliseconds.

For an HTTP check, set up a lightweight, specific endpoint in the application, such as /health. This endpoint should:

  • respond without doing heavy work,
  • return a successful response only when the application is ready to receive traffic,
  • not depend on unnecessary dependencies, such as a long-running report,
  • have a predictable response and status code.

Note that if you point the Health Check at a page built for normal users, or one that always returns success, the Load Balancer may consider the Member healthy while the application's real path has a problem.

After creation, each Member's status is one of these values:

  • Healthy

    The latest required checks succeeded, and the Member is eligible to receive traffic.

  • Unhealthy

    The checks failed, and the Load Balancer may remove the Member from distribution.

  • Checking

    Not enough checks have been done yet to reach a conclusion.

The status doesn't change immediately after the application is fixed, because the check interval and number of attempts affect detection time.

If a Member is unhealthy, first check its private IP and port. Then test the application from inside the same network and on the same port, check the VPC security rules and the operating system firewall, and match the Health Check address and port to the application. Finally, check the endpoint's status code and response time.

If all Members are healthy but the application has errors, the Health Check may only be checking that a connection is established or that one path responds. Check the application's status, its dependencies, user sessions, the database, and the service logs separately.

Naming and Confirming​

In the Load Balancer Name section, enter a specific, descriptive name, for example, web-prod-lb for the website entry point in the production environment.

Then review the summary of settings in the left column. This summary includes the AZ, size and access type, VPC and Subnet, Listener protocol and port, destination port, algorithm and selected Members, Health Check settings, name, and the amount shown. Finally, click Create Load Balancer and wait for the operation to complete.

If the Load Balancer isn't created, check the selected VPC, Subnet, and AZ, and make sure related resources or your account's resource limits aren't preventing creation. If you chose External, also check the Elastic IP's status and its compatibility with the VPC.

Testing the Service​

After creation completes, check the status of the Load Balancer and Listener in the panel and make sure at least one Member is healthy. Then, from a network that should have access to the service, send a few light, consecutive requests to the entry address and Listener port. Use the Elastic IP for External and the Private VIP for Internal. Check the application's response and the instances' logs to make sure traffic reaches the Members.

If Source IP or Session Persistence is enabled, or you have persistent connections, all requests may reach one Member. An unequal number of requests alone isn't a sign of a problem. If possible, take a test Member out of service for a short time and watch the Health Check's behavior. Don't do this test on a production service without a rollback plan.

A successful response from the Load Balancer alone doesn't show that all Members work correctly. Check each Member's status, the application logs, and the network path separately.

If you can't reach an External Load Balancer from outside, check the Elastic IP attached to it and match the Listener port with the port you use in the request. Also check the inbound rules and the network gateway path, and make sure at least one Member is healthy.

If you can't reach an Internal Load Balancer from an internal service, make sure you send the request from an allowed network inside the VPC and use the Private VIP and Listener port correctly. Also check the path and access rules between the source and the Load Balancer's Subnet. A service not being public doesn't mean it's automatically accessible from every Subnet or connected network.

Tutorial Example: An Internal Service with Two Destinations​

Suppose an internal service runs on two instances with the addresses 10.20.1.10 and 10.20.1.11 in the Subnet 10.20.1.0/24 and one data center, and the application responds on port 8080. For this service, create an Internal Load Balancer, set the Listener to the HTTP protocol and port 80, set the destination port to 8080, and choose the Round Robin algorithm. For the Health Check, use the HTTP protocol on port 8080. If the panel shows path and response code fields, enter the path /health and the response 200.

The values in this example are made up. Take the real addresses of your resources from the panel.

First, from an allowed source inside the VPC, check the application's response on both instances. Then create the Load Balancer and wait for the monitoring result. To test the entry point, send a request with a browser or HTTP tool to http://<PRIVATE_VIP>/, replacing <PRIVATE_VIP> with the actual assigned address.

Note that HTTP in this example is only for testing in a test network. Don't test sensitive information or user logins over an unencrypted connection.

Deleting a Load Balancer​

Before deleting a Load Balancer, move the VIP's consumers to another address and check the related Listeners and Members. Also check the effect of deletion on DNS, dependent services, and network rules. If an Elastic IP is attached to the Load Balancer, manage its attachment using the panel's options. Then confirm the deletion and follow the resource's status until the operation finishes.

Note that while dependencies exist, the resource may not be directly deletable. Treat a deletion error as a sign that you need to check dependencies, not as permission to force-delete other resources.

Limitations​

Arvancloud manages the Load Balancer infrastructure. Running the application, configuring the network, and keeping Members ready are your responsibility. The Health Check helps with routing decisions, but it doesn't guarantee that the application's logic is correct or that it can recover from a complete data center failure. If all Members are unhealthy, the Health Check alone doesn't restore the service.

The Load Balancer's limitations are:

  • A Load Balancer is created in only one zone, and the Load Balancer and its Members must be in the same Subnet.
  • Global load balancing, automatic routing between zones, and load balancing across multiple Subnets are not supported.
  • Each Listener has only one Server Pool.
  • The Load Balancer doesn't create your instance or application automatically.
  • The Load Balancer isn't a substitute for application scaling, backups, disaster recovery design, or fixing application errors.