Re: [load balancing] Verisign and Load Balancers

From: Alex Samonte (
Date: Tue Feb 20 2001 - 14:01:39 EST

  • Next message: Alex Samonte: "Re: [load balancing] Cascading switches off of a Foundry switch"

    On Fri, Feb 16, 2001 at 09:27:37PM -0600, KJ & JC Salchow wrote:
    > ----- Original Message -----
    > From: "Alex Samonte" <>
    > To: <>
    > Sent: Friday, February 16, 2001 6:27 PM
    > Subject: Re: [load balancing] Verisign and Load Balancers
    > > The new 600 card had to go to a SMP box (finally working SMP in F5's
    > > BSD code!). But you're still going to max out your cpu's
    > > and impact normal HTTP performance.
    > Not entirely true, the 600 card (and multiple cards) will run on the v3.x
    > which does not have SMP. And you're not going over the CPU's performance -
    > see below.

    This is contrary to our testing.

    > > This is a recent code fix they did. Unfortunately, the F5 itself can't
    > handle
    > > much more than 200 conn/s anyways. The rainbow cards do not do the bulk
    > > encryption/decryption of packets. That's left for the general purpose
    > > CPU to do. At 200 conn/s the F5's cpu is pegged at 100% utilized.
    > There are Rainbow cards which do bulk encryption, the reason most people
    > don't utilize them is because the bulk encryption is generally RC4 which is,
    > by design, built to be fast and efficient as a software implementation -
    > whereas the RSA encryption used in the asymetric key exchange is fast and
    > efficient in straight hardware. If you look at the stages of SSL, the bulk
    > encryption is not difficult or processor intensive - it's essentially a
    > onetime pad cipher - which equates to simple substitution cipher with a
    > unique key at least as long as the message to avoid repitition. This is not
    > tough - programmatically or processor wise. That's what makes PGP such a
    > great system - it uses the RSA for the transmission of the onetime pad
    > cipher and then uses that to perform bacis substitution. Yes, I know this
    > is a little simplified, but its accurate enough to argue the impact of bulk
    > encryption on the processor. If it weren't for the asymetric key exchange -
    > the part everyone is off loading - SSL would have almost no impact on the
    > machine.

    It is not difficult no. It consitutes less than 10% of the actual work
    to do, but at 200 conn/s it adds up. We've tested about 3 or 4 different
    vendors and they all suffer from the same cpu problems when you reach
    large numbers of conn/s because the cpu does the bulk encryption and

    > As to handling more than 200 conn/s - I've never seen an F5 box that I've
    > worked on or supported at more than 54% processor utilized no matter what
    > they were doing. Most of the real work on the BIG-IP revolves around
    > memory, not processor - and when compared to the other "general purpose
    > processors" I'd much rather have the F5's Pentium III than the ones used by

    You're right, keeping the state tables is a function of memory, but searching
    them and doing the packet processing is something that is done by the CPU.
    Even as a small cpu amount it still takes up CPU power. And when it's all
    being sucked up doing the bulk crypto the result is latency per packet
    goes way up.

    We've done lots of testing and this correlation is proven again and again.

    If Eric cares to speak up even F5 admitted to us this was a problem. One
    they hoped would be taken care of by SMP (or just putting bigger procs in

    > > And you also want it to handle all your non encrypted load balancing too.
    > > Not much CPU left for that.
    > And this again depends on what else you are doing. Is all your
    > non-encrypted traffic doing header parsing? That will eat up more CPU than
    > handling the SSL handshakes and bulk encryption - but again - the F5 has the
    > right processor for the job which is why their throughput doesn't degrade at
    > Layer 7 like most other products. RSA is off-loaded and bulk encryption is
    > easy - heavy header parsing or extremely complex rule-sets can and will eat
    > up the processor quickly - but this is an administrative issue as well. If
    > you write your rules correctly and plan ahead, you shouldn't need overly
    > complex rules.

    I prefer doing a second pass through the LB to do the non encrypted header
    parsing. Most LB's support this. it's more of an architecture than
    a technology.

    > No this I can agree with. Even with the F5 box there is a limit to the
    > amount of SSL you can handle from a pure NEW connections per second rating
    > on the Rainbow cards and by having them directly inline with the rest of
    > your site does present some scalability issues. That's why many people use
    > stand alone SSL (which F5 has as well) being load balanced calling back to a
    > load balanced VIP (I think in another message you demonstrated this in the
    > "one-arm" fashion - I would use a three interface BIG-IP to avoid sending
    > traffic accross the internal interface as many times - the same thing I do
    > for server-side cahces). In fact, in another life we actually had different
    > paths for encrypted and non-encrypted traffic. This served to keep us
    > mindful of what was on the wire as well.

    F5 doesn't normally recommend the 1 arm. They normally talk about using
    it like the firewall sandwitch (which of course sells more boxes). But
    since SSL is a small %age of traffic, doing a double pass with that
    smaller amount of traffic I don't see as a problem.

    > On the other hand - how many people are doing that many NEW SSL sessions per
    > second? This is another of those sticky litle requirement issues that seems
    > to crop up. You do NOT need to support 1000 NEW SSL session/second just
    > because you have 1000 users using SSL. This is about as bad as the people
    > who use load balancing for their Internet servers and swear they need Gig
    > throughput - even though they only have a T1 to the Internet.

    Absolutely correct. Not everyone needs that many conn/sec. It's more of
    a big penis thing. BUT There are some sites that do need that, even to
    the tune of 1000 conn/sec.

    Let's take a step to the side and look into the future. I'm going to be
    writing a white paper on this soon (so dont steal my topic...heh) about how
    if the SSL accelerators do 3 key things, and really become ubiquitous, that
    there wouldn't be any reason not to convert whole sites to SSL (as long as
    they do these 3 key things). If that happens, the need for much higher
    performing SSL accelerators will be there. There are several companies
    out there today who are moving in this direction.


    This archive was generated by hypermail 2b30 : Tue Feb 20 2001 - 14:06:21 EST