Re: [load balancing] Outbound HTTP throttling

From: Kelly Vista <kvista7 [izzat]>
Date: Fri May 08 2009 - 16:12:53 EDT

Excellent feedback, thanks!

First, thanks for pointing out the flaw in my post. I didn't mention that I
would strongly prefer some sort of queue-ing. Basically, I imagine that a
device would have some max number of connections it could queue (assuming
the client network timeout didn't kick in). Even better would be a "high
availability" mode where multiple LBs, linked together, share a common
queue. This ensures high availability and scalability, although I don't
know if the replication aspect of throttling is feasible. Returning
connection reset or something else that would require a retry would be an
unfortunate reality. So I'd prefer a queue.

We actually tried to work with F5 (we were comparing them against NetScaler)
but the whole process was pretty slow, unfortunately, and also our IT folks
were more familiar with NetScaler. Among other problems was that the F5
pre-sales folks didn't know where we would be able to get a rule from. They
suggested the big F5 repository online, but we couldn't find one very easily
and we didn't want to spend money learning the intricacies of rule writing
only to find out what we were doing was wrong. Another problem was that we
didn't know what the CPU impact of rule execution might be.

If you can point me to a rule (or write it), I would be incredibly
indebted. Based on this, I would recommend to management that we pursue

-- kv

On Fri, May 8, 2009 at 12:09 PM, Kirk Bauer <> wrote:

> This would be very easy to do with iRules on a BIG-IP. If you have a
> chance to try this out on a BIG-IP, let me know and Iíll write a rule to
> take care of it for you.
> The way it would work is you direct the outbound traffic through a virtual
> server (i.e. vip) on the BIG-IP and have a relatively straight-forward iRule
> that would, for example, check the Host header and keep track of the
> connections/sec for each separate destination host.
> The only question would be: what do you want to do once the threshold is
> exceeded? Do you want to delay the request until later, or do you want to
> return some sort of retry status code to the server, etc?
> *From:* [] *On Behalf
> Of *Kelly Vista
> *Sent:* Friday, May 08, 2009 11:30 AM
> *To:*
> *Subject:* [load balancing] Outbound HTTP throttling
> Hi -
> At our company, we make use of a lot of "web APIs" that exist. For
> example, Google has a search API, Netflix has an API, etc. Using these
> APIs, we can create a lot of "mashups" or richly integrated data sets.
> However, we would like to be a "good Internet citizen" and obey the rate
> limiting requirements of the sites we hit. For example, many companies have
> rate limiting policies like "no more than 5 requests per second".
> Unfortunately, since we have many machines that - for several different
> applications - hit these sites, we need to make sure that our *aggregate*
> use is no more than 5 (or whatever) requests per second.
> I'm writing to this list because I believe there may be load balancers with
> L4 or L7 capabilities that allow us to do this and I was looking for
> suggestions. We are in the process of testing out Citrix NetScaler for this
> and, although it seems somewhat capable, it's been very tricky to configure.
> To re-iterate the complexity of our particular problem, it has to do with
> rate limiting outbound traffic based on number of connections/requests per
> second.
> If anyone has any insight into solutions that do this (h/w or s/w), I would
> appreciate it.
> For the record, I know there are software solutions like like Apache
> mod_throttle and mod_cband, Linux iptables, Squid, Vista Server QoS, etc.
> I'm open to any of these provided it's straightfoward to manage. I would
> prefer the reliability of solid state, but my first priority is any kind of
> solution, reliability can be an "optimization".
> Thanks for any and all help,
> --kv
> _______________________________________________
> lb-l mailing list
> Searchable Archive:
> Load Balancing Digest
> Load Balancing Wiki

lb-l mailing list
Searchable Archive: Load Balancing Digest Load Balancing Wiki
Received on Fri May 8 16:12:50 2009

This archive was generated by hypermail 2.1.8 : Fri May 08 2009 - 16:12:51 EDT