Weblog entry #35 for lee

Load-balancing Apache sessions with custom cookies
Posted by lee on Thu 29 May 2008 at 15:09
Tags: ,

If you're already familiar with using apache as a reverse proxy between frontend and backend servers then using mod_proxy_balancer to load-balance sites across multiple backend servers is straight forward.

Well, it's straight forward unless you need to track sessions - for example on a site with a shopping cart. Fortunately, you can use the "stickysession" parameter to keep sessions tied to specific backend servers. This uses the value of a session tracking cookie to chose the correct backend server (or "worker") to use.

What the Apache documentation fails to make clear is that the session tracking value has to have a specific format: it must end with the suffix of a period followed by the backend server's route identifier.

If your cookie isn't in the right format, and you're not in the position to monkey with your app's session management code, you'll probably going to want to force a cookie to be set via the apache config as suggested by Mark Round. This solution works fine, provided you can individually modify the config on each server - but the servers I'm using have identical configs managed centrally using a revision control system.

To get around this, I'm using a RewriteMap to do a table lookup on the server IP address to get the correct route identifier. When a session starts being tracked for stateful purposes the next request gets a balancer cookie. The following code appears in the configuration for the backend server for example.com:

RewriteEngine On
RewriteMap      routemap txt:/path/to/routemap
RewriteCond     %{HTTP_COOKIE} (existing_session_cookie) [NC]
RewriteRule     .* - [CO=balanceid:route.${routemap:%{SERVER_ADDR}|error}:%{HTTP_HOST}]

Note that this approach only starts working from the first request containing a session cookie. If this can cause a loss of state information (for example: login credentials) you'll need to either utilise an app-level cookie, or remove the RewriteCond and track all sessions.

Replace "%{HTTP_HOST}" with ".example.com" if the session needs to be sticky across subdomains.

The routemap file is maintained centrally and consists of simple IP address and route pairs:

10.16.4.4       www1
10.16.4.6       www2
10.16.4.7       www3
10.12.2.13      www0

Then the following is added to the apache config on the reverse-proxy:

<IfModule !proxy_balancer_module>
ProxyPass / http://backend.example.com/
</IfModule>
<IfModule proxy_balancer_module>
ProxyPass /balancer-manager !
ProxyPass / balancer://backend.example.com/ stickysession=balanceid nofailover=On
<Proxy balancer://backend.example.com>
  BalancerMember http://10.16.4.4:80 route=www1 loadfactor=60
  BalancerMember http://10.16.4.6:80 route=www2 loadfactor=60
  BalancerMember http://10.16.4.7:80 route=www3 loadfactor=60
  BalancerMember http://10.12.2.13:80 route=www0 loadfactor=20
  ProxySet lbmethod=byrequests
</Proxy>
<IfModule status_module>
<Location /balancer-manager>
  SetHandler balancer-manager
  Order Deny,Allow
  Deny from all
  Allow from 10.8.2.
</Location> 
</IfModule>
</IfModule>

Note that the extended parameters toe BalanceMember will only be read on Apache's startup (or restart); reload will have no effect, even for a fresh entry.

Also note that, if you're caching static content on the reverse proxy, you'll probably want to prevent the cookies from being cached. If you need to keep other cookies for some reason, you'll need to modify the RewriteCond to avoid adding balance cookies to cached content.

CacheIgnoreHeaders Set-Cookie

 

User Login

Username:

Password:

[ Advanced Login ]

Register Account

Quick Site Search