Spring Cloud Ribbon -Client side load balancing

Spring Cloud Ribbon - Client side load balancing

Traditionally we have server side load balance which typically dispatch the load to intended server.


HTTP,TCP,UDP                                           Application1
      Traffic                                                ⬈
                                                            ⬈           
→→→→→→→→ Load Balancer ➤➤➤➤ Application2
                                                          ⬊ 
                                                                ⬊         
                                                                     Application3 

implemented in software(Apache,Nginx,HA Proxy) or Hardware(NSX,BigIP..)

Client Side Load Balancer

client side load balancer selects which server to call based on some criteria 

They reside in the application as inbuilt component and bundled along with the application, so we don’t have to deploy them in separate servers.

these are part of client software

server can also employ its load balancer

Based on some criteria
part of client software

why client side load balancer needed?

No More Single Point Of Failure In A Cluster: In a cluster fronted by a single load balancer there is always risk if the cluster-nodes becoming unreachable if the load balancer goes down. 
With client side load balancing, since the server side load balancers are not required anymore, this limitation goes away. 

Removal of Bandwidth Bottleneck: The server side load balancers can easily become pain points for high throughput systems. Given that the load balancers will have a finite number of threads to process the requests and a finite amount of bandwidth (dictated by the NIC card of the VM / hardware). This means that there is always an upper limit to horizontal scalability with server side load balancers. With client side load balancers, this concern goes away as all inter service communications are peer-to-peer. 

Example: a given cluster is backed by 4 application nodes, each of them having 100 threads configured in server (or servlet container like tomcat, jetty, etc. ) to serve requests. That makes a total of 400 threads. The load balancer fronting this cluster also has 400 threads. Makes sense. Just when things were predictable and stable, the company ran an advertising campaign on YouTube and now they are expecting double the traffic. Which means the application cluster should now have double the number of nodes. Easy. The devops team spins up 4 new nodes - which makes a total of 8 nodes in the cluster. So now they have 800 (8x100) threads to server the request, but the load balancer still has 400 threads. Simple. Let us increase the number of request processing threads on the load balancer. Looks trivial on the surface. However, each request processing thread requires a certain amount of memory and RAM. Does this require a larger VM? If not yet, it won't be long before the VM hits that ceiling.

Auto-discovery:
The discovery is totally transparent and all it needs is configuring the IP/hostnames of the registry servers in the application's configuration files.

Some may be slower than others:performance
Some may be further away from others : regions

In this diagram here we have application running in a region ,i expect application should connect in same region and give faster performance but somehow the server is not available ,if client side load balencer is aware of other region it can rout call to other region.

            Spring Cloud Ntflix Ribbon

Ribbon - Another part of Netflix OSS family
-client side load balancer
-Automatically integrates with service discovery 
-Build in failure resiliency(Hystrix)
-it support things like caching and batching
-multiple protocols(HTTP,TCP,UDP)

Spring cloud provides easy api wrapper for using ribbon


Key Concept in Ribbon

List of Servers
Filtered List of Servers
Load Balancer
Ping

1.List of Servers

Determine what the list of possible servers are for a given service(client)
-static- populated via configuration
-Dyanamic - populated via service discovery (Eureka)

Spring cloud default - use Eureka when presents on the classpath

Example of static server lists

application.yml ("stores" and "products" example of client-ids)
stores:
  ribbon:
     listOfServers: store1.com,store2.com
products:
  ribbon:
    listOfServers: ProductServer1.com,productserver2.com

2. Filtered List of Servers

criteria by which you wish to limit the total list
Spring cloud default- filter server in the same zone

3. Ping
  Use to test server whether up or down
  Spring cloud default - delegate to Eureka to detarmine if server is up or down
4. Load Balancer:

•   load balancer is the actual component which routs the calls to filtered list of servers
• Server stratagies are available but they usually differ to a rule component to make the actual decisions
• spring cloud default: zonAwareLoadBalancer

5. Rule- makes decision whether to call or not

    Spring cloud default - ZoneAvoidanceRule



                                           Feign Client
➤Feign is another part of netflix open source library.it is easly usable within spring cloud
➤it allows you to write calls to rest services without implementation code
➤spring cloud provide easy wrapper for using feign.