Re: [load balancing] Re : Layer4 and Layer7 TPS

From: Kenneth Salchow <k.salchow [izzat] f5.com>
Date: Fri Nov 20 2009 - 15:31:07 EST

As always Tony, I bow to your supreme greatness!! :)

One thing to note also about measuring connections per second of any kind.
Many of the tests used in many reports have a zero payload. They simply
measure setup and tear-down, not the impact of real traffic. Remember that
if you are actually transmitting data it takes additional time and resources
to process that data--especially if any reassembly is required as well as
maintain state tables. Therefore, most of the metrics Tony mentions are
truly 'best-case'.

This gets me back to the "taste the pudding" comment. The size of the
payload can drastically impact the performance as well. Real-world testing
of the application is really the only reliable metric. Unfortunately, even
if you do a PoC, you may not be able to generate sufficient real-world
traffic to get an accurate picture of production performance.

KJ (Ken) Salchow, Jr. | Manager, Technical Marketing
D 651.423.1133
M 612.868.1258
P 206.272.5555
F 206.272.5555
www.f5.com

-----Original Message-----
From: lb-l-bounces@vegan.net [mailto:lb-l-bounces@vegan.net] On Behalf Of
Tony Bourke
Sent: Friday, November 20, 2009 1:17 PM
To: Load Balancing Mailing List
Subject: Re: [load balancing] Re : Layer4 and Layer7 TPS

It depends on how you define "TPS", also. While there's no ultimate
authority on this (really? None? Well then, I declare my self supreme
authority! You are all witnesses to my greatness!) TPS is generally
accepted to be a measure of SSL/TLS performance. Specifically TLS is
the measure of *new* SSL sessions initiated (as opposed to re-using
existing SSL sessions, which can save quite a bit of resources).
Sometimes TLS is used to reference the number of HTTP requests processed
as well, but usually you see it with SSL/TLS only.

SSL/TLS is a wrapper for a generic TCP connection, and with the HTTP
protocol you can run multiple (tens, hundreds, perhaps thousands) of
HTTP requests through a single TCP connection. So TPS and "HTTP
requests" per second are very different.

Generally, you don't see a 1:1 ratio in terms HTTP requests per SSL
connection. The whole point of HTTP 1.1 was to allow multiple HTTP
requests per TCP connection (and thus SSL/TLS connection) so as to be
more efficient. But the ratio you do see in the real world could vary
between 1:1 to 1:1000 or more, depending on your site. It's usually the
SSL ASIC (or in some cases, just your general processor) that determines
how many new SSL connection a device can handle. Traffic with a 10:1
ratio would mean that a device with a 10,000 TPS SSL chip could handle
100,000 HTTPS requests per second.

The performance numbers can be confusing, and it's gotten several people
in trouble. Not long ago a guy claimed to have bested F5 performance
with off-the-shelf hardware and no SSL ASICs (He got his terms mixed up,
he was comparing apples to oranges).

So here's a few of the primary ADC/load balancer metrics defined:

Connection rate: The rate of TCP connections per second a device can
handle. This could be either while the load balancer operates in Layer
4 mode or Layer 7 mode.
Request rate: The rate of Layer 7 requests, such as HTTP requests, that
a device can handle. Multiples of these requests will probably be in a
single TCP connection, so this number would likely be higher than the
connection rate.
L4 requests per second: Could arguably be defined as Layer 7 protocol
requests transiting through an ADC operating in L4 mode (the load
balancer is only aware of the TCP connection, not what goes on inside),
or just a simple TCP connection rate.
L7 requests per second: Could arguably be defined as Layer 7 protocol
request rate transiting through the ADC while operating in L7 mode, or
just a simple TCP connection rate as well.
TPS: The rate of *new* SSL sessions. A new SSL session requires an
asymmetric encryption operation, which is what really gets the CPU and
why most devices use an ASIC instead.

Hope that helps,

Tony

ADC expert wrote:
> @Kenneth, I am not F5 employee.
>
> I am validating the existing ADC products for L4-L7 TPS. I really
> appreciate F5 publishing their results.
>
> Coming to the discussion, I believe and agree that hardware plays its
> role to show up better performance. The recent press release from
> Radware claims 340K TPS for its Alteon on their On demand hardware.
> Compared to what Alteon 3408 used to have, its a tremendous
> improvement. That proves software plays its role in performance. At
> the same time, it confuses when I see performance results of one ADC
> vendor to be L7 -300K TPS that claims to have hardware excellence.
>
> Thanks,
>
>
> On Fri, Nov 20, 2009 at 12:41 AM, Kenneth Salchow <k.salchow@f5.com
> <mailto:k.salchow@f5.com>> wrote:
>
> While I might not be quite so exuberant in my response, I will
> suggest the Surya has a couple valid points.
>
>
>
> First, however, let me say that, to the best of my knowledge, that
> post was NOT made by an F5 employee. If it is discovered that it
> was, it was inappropriate and it will be rectified. IF, however,
> it truly is simply one of our customers who found some value in
> our testing, I’m happy to see their excitement and enthusiasm. I,
> personally, believe we do a pretty fair job of presenting the
> facts; we publish the results regardless of whether we come out on
> top or not and we provide complete transparency as to the
> configurations of all devices, testing equipment and methodology.
> Given the report, the time and the equipment, anyone should be
> able to produce the same results. I truly believe it is as
> unbiased as it can be.
>
>
>
> That being said, I agree with Surya the no matter how unbiased a
> performance report may or may not be, the true test is how the
> device performs in YOUR environment with YOUR applications and
> YOUR business processes. As they say, the proof of the pudding is
> in the tasting; make sure you get a true taste. If possible, a
> PoC is something we suggest all our prospective customers do.
> Anyone who has been a part of this list for any time knows that I
> am a strong proponent of that philosophy. In the end, that’s all
> that matters.
>
>
>
> I wish the ADC Expert good luck (Bon Chance, I believe?) in their
> search for the best product for their needs. If you have any
> specific questions, I’m sure everyone in the list will be happy to
> give their perspective and input.
>
>
>
>
>
>
>
> *KJ (Ken) Salchow, Jr.* | Manager, Technical Marketing
>
> *D 651.423.1133*
>
>
>
> *M 612.868.1258*
>
>
>
> *P 206.272.5555*
>
>
>
> *F 206.272.5555*
>
>
>
> *www.f5.com <http://www.f5.com/>*
>
>
>
>
>
> *From:* lb-l-bounces@vegan.net <mailto:lb-l-bounces@vegan.net>
> [mailto:lb-l-bounces@vegan.net <mailto:lb-l-bounces@vegan.net>]
> *On Behalf Of *Surya ARBY
> *Sent:* Thursday, November 19, 2009 12:43 PM
> *To:* Load Balancing Mailing List
> *Subject:* [load balancing] Re : Layer4 and Layer7 TPS
>
>
>
> Hello.
>
> To me this kind of "performance reports" or "competitive analysis"
> are just a bunch of bullshit.
>
> You'll find a lot of other papers on Citrix / Cisco / any vendor
> web site with a lot of case studies stating they are the best
> vendor with the best appliance and that the end customer is the
> happiest customer in the world with his solution :)
>
> Make a POC and setup your own load testing :)
>
> The real L4/L7 performance in a specific configuration varies
> according to the internal architecture of the product you test.
> Cisco ACE module distributes the load between two NPs according to
> the result of a hash function to process load balanced flows, Ace
> appliance relies on a single Cavium Octeon chipset for the data
> plane, Citrix uses a purely software based appliance, F5 and
> Foundry use another architectures (barrel processors in foundry
> boxes are distributed like NPs in cisco ace module)...
>
> When you perform load testing it's very important to understand
> how the box you test really works internally, because results (and
> comparison) will change with the choosen scenario that's why the
> "performance report" you mention is necessarily biaised :)
>
> regards,
>
> Surya
>
> --- En date de : *Jeu 19.11.09, ADC expert /<adcxprt@gmail.com
> <mailto:adcxprt@gmail.com>>/* a écrit :
>
>
> De: ADC expert <adcxprt@gmail.com <mailto:adcxprt@gmail.com>>
> Objet: [load balancing] Layer4 and Layer7 TPS
> À: lb-l@vegan.net <mailto:lb-l@vegan.net>
> Date: Jeudi 19 Novembre 2009, 19h09
>
> Hi Experts,
> I was looking for a ADC device that can give best L4 and L7
> TPS. After considerable search, I got few things to get clarified
> before selecting the best.
> F5 has extensive performance report in its website.
> http://www.f5.com/pdf/reports/f5-performance-report.pdf
> I wonder why other vendors did not report in such detail. Even
> if, there is no free access.
> I appreciate if some one sends me across cisco, citrix and
> other vendors performance details.
>
> -Thanks.
>
>
>
> -----La pièce jointe associée suit-----
>
> _______________________________________________
> lb-l mailing list
> lb-l [izzat] vegan.net <http://mc/compose?to=lb-l@vegan.net>
> http://vegan.net/mailman/listinfo/lb-l
> Searchable Archive: http://vegan.net/lb/archive
> http://lbdigest.com Load Balancing Digest
> http://lbwiki.com Load Balancing Wiki
>
>
>
>
> _______________________________________________
> lb-l mailing list
> lb-l@vegan.net <mailto:lb-l@vegan.net>
> http://vegan.net/mailman/listinfo/lb-l
> Searchable Archive: http://vegan.net/lb/archive
> http://lbdigest.com Load Balancing Digest
> http://lbwiki.com Load Balancing Wiki
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> lb-l mailing list
> lb-l@vegan.net
> http://vegan.net/mailman/listinfo/lb-l
> Searchable Archive: http://vegan.net/lb/archive
> http://lbdigest.com Load Balancing Digest
> http://lbwiki.com Load Balancing Wiki
>

_______________________________________________
lb-l mailing list
lb-l@vegan.net
http://vegan.net/mailman/listinfo/lb-l
Searchable Archive: http://vegan.net/lb/archive
http://lbdigest.com Load Balancing Digest
http://lbwiki.com Load Balancing Wiki

_______________________________________________
lb-l mailing list
lb-l@vegan.net
http://vegan.net/mailman/listinfo/lb-l
Searchable Archive: http://vegan.net/lb/archive
http://lbdigest.com Load Balancing Digest
http://lbwiki.com Load Balancing Wiki

Received on Fri Nov 20 15:31:26 2009

This archive was generated by hypermail 2.1.8 : Fri Nov 20 2009 - 15:31:27 EST