============================ Swamp 1.1.0, README document ============================ This document is a "general discussion of what swamp is, how to use it, and why it exists". For other types of information see; INSTALL: Instructions on configuring, compiling, and "install"ing swamp. LICENSE: Rule and regs about redistribution of swamp, or incorporation into other projects or commercial products/distributions. CHANGES: A list summarising changes in swamp over time/versions, and the names of (and thanks to) those who made them happen. FAQ: A list of "Frequently Asked Questions" about swamp. FYI: Swamp is written by Geoff Thorpe (swamp@geoffthorpe.net), as is this dinky little document. The major contributors to swamp thus far are; Steve Robb Please see CHANGES for more info on "who?", "when?", and "what exactly?". ======== Sections ======== Section I: Quick rough-guide to using swamp Section II: Command-line reference guide Section III: Hints about using swamp =========================================== Section I: Quick rough-guide to using swamp =========================================== If you do not already have "swamp" installed and accessible on your system (run "swamp --help" to see) and you think it is supposed to be already installed (eg. you found this README somewhere right??), contact your sysadmin. If you think it probably isn't installed and you would either like to install it on the system or make a local build for your own purposes, read the INSTALL document. Swamp, the utility ------------------ "swamp" compiles as a self-contained executable, or as an almost self-contained executable depending only on OpenSSL shared-libraries, depending on what kind of OpenSSL installation you built against. Additionally, swamp requires no config files and provides no man/info pages, so "swamp" is the only file you are interested in and what you see here is probably all the help you get (for now). To test that the "swamp" program is installed and usable, you can ask it to display a usage message as in the very next subsection (because if shared-libraries it depends on are not present, or if the program was incorrectly linked or installed, it will fail even to do this). Swamp, the point of it all -------------------------- SSL/TLS is a cryptographic protocol for negotiating "sessions" that provide a framing layer through which you can send/receive data with a peer. That framing layer can provide both encryption (data privacy), authentication ("identity" verification), and integrity (tamper-protection). To learn more about SSL/TLS, consult any of the many good (and highly detailed) references on the subject, in particular Eric Rescorla's book does a pretty decent job of it; "SSL and TLS, Designing and Buidling Secure Systems" (ISBN:0-201-61598-3). The point of this document is not to discuss what it is or how it works but simply to acknowledge and address two points; (i) SSL/TLS can very easily become (and often is) a huge part of the processing overhead of an application, especially server applications. The cryptographic primitives used in the protocol (most noticably the session negotiation) can consume a lot of processing time and resources, and the result is that an application running at (or above) its SSL/TLS processing capacity can drag a system to its knees. (ii) Servers running SSL/TLS are therefore at the mercy of the demands placed on them and we may wish to know; (i) How much traffic a given server can sustain if we attempt to saturate it to capacity. (ii) If a given simulation of "client use" is thrown at the server, how does it behave. The two items aren't quite exactly the same thing, but are very much related. Swamp simulates SSL/TLS client(s) so that you can test the speed, capacity, throughput, and behaviour of an SSL/TLS server. By far and away, the most common cases of SSL/TLS servers are secure webservers - these serve "HTTPS" pages, which is a glorified term for an extremely mundane thing; if you connect to a secure webserver and use the SSL/TLS protocol for managing the traffic over that connection, the "cleartext" you send and receive through that SSL/TLS stream is essentially the same "cleartext" you'd normally send and receive to a webserver directly without any SSL/TLS encryption. That cleartext format your client and server speak (be it inside SSL/TLS or direct) is otherwise known as "HTTP". :-) The basics ---------- When using swamp for the first time, or trying to understand something you've not used before (or have since forgotten) you will probably want to display swamp's usage message. You can do this as follows; [bob@wherever ~] swamp --help That should display all the command-line options "swamp" supports and the most rudimentary descriptions of what they mean. When beginning with "swamp", you will at least probably want to understand the following switches because they are the main ones you'll need to start using "swamp"; -connect : (the IP or hostname of the SSL server and the port number the server is listening on, eg. 443 for https) -num (how many connections to hit the SSL server with) -update (how often to display statistics, 10 is good) If the SSL/TLS "server" you want to target with "swamp" is a secure web server (ie. you're wanting to test HTTPS), you may also wish to learn about; -request (the query string sent to the SSL server) -expect (how many bytes to "wait" for in a reply before closing and sending the next request. -1 means let the server close) All the available switches are detailed in the next section. To learn about the above "important" switches, scan through the next section and find the corresponding explanations. ======================================== Section II: Command-line reference guide ======================================== The following command-line options are supported by swamp and their usage and interpretation are described in approximative terms. The source code to swamp and your own common sense should always be at the ready should you need to "seek clarification". Failing that, check the FAQ. These command-line options are broken into subsections depending on which "subject" they relate to; - Fundamental parameters affecting "swamp" operation - Parameters affecting "swamp"'s output/logging - Parameters affecting data transfer (inside the SSL/TLS layer) - SSL/TLS settings "swamp" uses in communication with the server - Distributing swamp requests across multiple servers (advanced) Fundamental parameters affecting "swamp" operation -------------------------------------------------- -connect : [Mandatory, no default value] This indicates the address of the SSL/TLS server you wish "swamp" to test. can be a dotted-numerical IP address, eg. 207.46.197.100, or a hostname (eg. www.whatever.dom) that swamp will attempt to resolve to an address using your systems default name-resolution (eg. DNS). is the TCP port that the connection should specify - the server application listens on a particular port (or more than one) so you need the right combination of both the address and the port to ensure you are communicating with the application or service you want. Eg. the default port for secure webservers to listen on is "443". You can specify "-connect" more than once and then each 'client' swamp is simulating (see "-num") will send requests to each server in turn (unless this behaviour is modified with the "-distribute" switch). -num [Optional, default value is "5"] Using "asynchronous I/O", swamp is able to simulate multiple clients at a time. In reality, swamp will sleep waiting for network activity on *any* of the simulated clients, then will quickly sweep across all of them processing any traffic that has arrived, and generating any new traffic as a result, then go back to sleep. This means "swamp" does not run in a truly parallel manner (and even if it did, it would still be artificial in some senses unless you are running a system with multiple CPUs). You can use the "-num" switch to control how many "simultaneous" clients it should simulate. This is useful to help work out a server's "parallelism" (ie. how well it works when handling many requests at a time rather than how quickly it can process one request after another). Also, over slow network connections, some servers may appear "slow" simply due to latency of communications, but by increasing "num" there is more chance a server can be kept busy even when some clients are waiting for requests to arrive at the server or responses to arrive back at the client. -count [Optional, default value is "0"] By default, "swamp" continues to run until you stop it, it crashes (I'd like to know if that happens of course), or your system itself shuts it down (or loses power, etc). Using "-count" you can specify the number of completed request/response operations swamp should wait for before stopping voluntarily. This total is counted across all "clients" since swamp was started (ie. when "-num" is greater than 1). The default value is 0, which means swamp won't voluntarily stop based on the number of completed operations. Note that sometimes the limit is reached when multiple requests complete inside the same 'network loop'. This can result in the final line of output indicating slightly more than 'n' completed requests, this is quite normal. -time [Optional, default value is "0"] As with "-count", "-time" provides a way to have "swamp" stop voluntarily. In this case, it is based on the number of seconds elapsed since swamp was started. The default value is 0, which means swamp doesn't stop based on any such timing considerations. -h, -?, or --help [Optional, no default value] If any of these equivalent switches is supplied, swamp will not perform any testing but will simply output the list of command-line options it supports in a standard sort of "usage" message and then exit. All other switches occuring after one of these will be ignored, as will any switches swamp had already parsed before finding one of these. Parameters affecting "swamp"'s output/logging --------------------------------------------- -update [Optional, default value is "0"] If a value greater than 0 is specified to it, this switch will cause statistics to be output to 'stdout' indicating the performance swamp is observing from the server it is communicating with. Two sample lines of output (where we'd specified "-update 10") might look like; 10 seconds since starting, 1607 successful, 0 failed, resumes(+965,-0) 160.70 ops/sec 20 seconds since starting, 3347 successful, 0 failed, resumes(+2008,-0) 174.00 ops/sec the "successful" and "failed" counts are totals (rather than per-10s interval) and report how many complete SSL/TLS+request/response operations completed successfully or failed for some reason. The resumes(+xx,-xx) values indicate (respectively) how many resume attempts that were attempted by the client were successful and how many "rolled-back" to full session negotiations. See the "-session" documentation for more info on this. The final "speed" counter indicates what the average operation speed was over that 10 second interval. However, this average speed is calculated with sub-second precision as the true interval between the updates may vary slightly either side of 'n' seconds. If this switch is not specified, no logging to stdout is performed at all. -nologo [Optional] If this switch is supplied, the "swamp" version and copyright notice will not be output at the beginning of swamp's execution. This can be useful when using "swamp" in a script that attempts to capture output and automatically process it. -csv [Optional, default is no 'csv' output] By providing the path to an output file, this switch can cause "swamp" to generate statistics into a "comma-separated-values" file. These files are readily importable to most spreadsheet/database programs so can help in performing more sophisticated analysis of swamp's performance output (ie. pretty graphs). Every second, a new line of output is generated in the csv file and is indexed (first column) by the absolute time on swamp's host machine. For example, this makes it easier to use multiple invocations of swamp (perhaps even on multiple machines) and be able to collate their results together based on that time value. Two sample lines of output in the "csv" file could look like; Wed Dec 19 17:01:20 2001,130,0,80,0 Wed Dec 19 17:01:21 2001,300,0,180,0 The first value is the absolute time, the second and third values are the total (cumulative) counts of successful operations and failed operations, and the last two values are the hits/misses counts for session resume attempts (see the info on "-session" for details). Parameters affecting data transfer (inside the SSL/TLS layer) ------------------------------------------------------------- -request [Optional, default is "GET /\r\n"] When swamp has connected to the SSL/TLS server and performed the SSL/TLS handshake to establish a session for carrying data between the client and server, it will send a fixed string of "request" data to the server through the SSL/TLS layer and wait for a configurable-length reply (or for the server to close the connection) before starting a new operation. In the case of secure web-servers, the request data should be an appropriate "http" request. This is why the default is what it is - it will ask for the default home page on the web-server the same way an equivalent "https://hostname" browser-request *could*, but not necessarily how it *would*. In the case that the server application uses some other protocol (eg. if SSL/TLS is being used to provide secured remote login facilities) it may be necessary to configure request data appropriate for that protocol (eg. a test username and password) to properly observe the server behaviour you are looking for (eg. processing an error due to a malformed request may take much more *or* much less time than the "normal" operation you are trying to test). At the moment, the only "escaping" that is possible in specifying that request string is with the '\r', '\n', and '\\' characters. All other escape sequences are reduced to literals (for now). -expect [Optional, default is 20] As with "-request", this affects the request/response cycle that the client and server perform using SSL/TLS *after* the handshake has established an SSL/TLS session. This switch allows the user to control how many bytes it should receive from the server through the SSL/TLS layer (ie. the "cleartext" we get after decrypting the SSL/TLS wrapping). If the value is negative (eg. "-expect -1"), then swamp will simply wait for the server to close the SSL/TLS session and the connection, it will continue quietly reading any data the server sends to it until that happens. Otherwise, when "n" bytes of response have been received from the server, swamp will regard this as "a success" and will itself close the SSL/TLS session and the connection with the server. SSL/TLS settings "swamp" uses in communication with the server -------------------------------------------------------------- -CAfile [Optional, default is to search the following paths and otherwise do without; ./cacert.pem /usr/bin/cacert.pem /usr/local/bin/cacert.pem /etc/cacert.pem] SSL/TLS's "handshake" mechanisms involves the server providing a certificate to the client indicating who it is. To distinguish between a certificate from "visa.com" saying it is "visa" and a certificate from "shonky thiefs" saying it is "visa", the certificate is usually digitally signed by another certificate. Sometimes (eg. on test servers using test certificates) this is not the case in which case there is no way to truly authenticate the server. Sometimes, the certificate that did the signing is also provided to the client and in fact can itself be signed by another certificate (ie. you can speak of a "certificate chain"). The principle is simpler than this indicates; if the chain of signatures of the certificate(s) leads back to a certificate that you *do* trust, you can assume from the signatures that the party whose certificate you trust has authenticated who that certificate belongs to, to some degree. These "trusted" certificates are usually pre-installed on user systems (eg. in web browsers) and can be added to your system manually if you're sure of the intergrity of the certificates you wish to "trust". These are called "CA" certificates (Certificate Authority). When "swamp" completes a handshake with a server where the chain of certificate signatures did *not* lead to any certificate that "swamp" implicitly trusts, it will display one single warning (it will surpress that warning for all following handshakes on the assumption they'll all fail the same way and otherwise make too much noise). If you have access to the CA cert in some way or other that signed the server certificate (many applications include OpenSSL itself include a "CA" file containing a list of common third-party CA certificates from Verisign, Thawte, etc), you can specify it using the "-CAfile" parameter. If the handshake results in the server certificate being satisfied with the server's authenticity because of this, no warning will be displayed. -cert [Optional, default is to use nothing] SSL/TLS is most commonly used with server-authentication - ie. (looking at the "-CAfile" comments) the server will send its certificate(-chain) and the client can try to verify its signature(s) against the list of "CA" certificates it trusts. Strange, mostly unsupported, modes of SSL/TLS can operate without any authentication at all - assuming the only threat you're worried about are "passive attacks" (you know who you're talking to and all you're worried about are people listening in who are unable to actually change anything). At the opposite end, it is also possible for the server to also insist (or "ask") the the client pass a certificate so that it can authenticate the client to its satisfaction. Whether the client actually sends a certificate is up to it, and the server can decide whether to accept no certificate if it wants. This gives "dual-authentication", both peers can attempt to authenticate each other by looking at (and checking) the certificates they provide. The "-cert" switch allows swamp to be configured to have a client certificate ready if the server asks for one. Otherwise it will not pass a client cert even if the server wants one (and starts rejecting requests if one is not provided). However, in SSL/TLS the server "requests" that the client send a certificate by indicating the list of CA certificates the server will use to authenticate it. This is one way of not only asking for a client-certificate, but also specifying which 'type' of certificate it wants in case the client has a choice. If swamp (or most other OpenSSL-based applications) has a client cert that is *not* signed by one of those CA certs the server trusts, it will not even bother sending it. This is a common gotcha amongst users - they wonder why the client doesn't send the certificate even though they've specified it. The problem in these cases is that the server hadn't indicated that it would trust the CA certificate that signed the client certificate in question. -cipher [Optional, default is to let OpenSSL choose] SSL/TLS, when performing the handshake between client and server, will select a "cipher-suite" and a session (keys and other "state" info) in which to conduct the encryption of the streamed traffic. Only then is the underlying data actually carried through the SSL/TLS layer between the two parties (ie. during the handshake, all traffic is specific to the handshake and nothing that each party tries to send to each other is carried in that traffic). These cipher-suites are certain combinations of cryptographic algorithms, key-sizes, and message-authentication schemes. Eg. the "RC4-SHA" cipher suite uses the "RC4" algorithm for symmetric encryption and the "SHA" hash algorithm for providing "checksums" of the payload data. At the more complicated extreme, "EDH-DSS-DES-CBC3-SHA" will actually use a temporary (ephemeral) DH key-exchange to negotiate the SSL/TLS session, which will then use the "DES" algorithm (in "CBC3" mode) for encryption and the "SHA" hash algorithm for message authentication. The "ephemeral" suites are arguably the most secure because they provide a curious cryptographic property called "perfect forward secrecy" (see a reference for detailed explanation). Essentially this means that even if your certificate's corresponding private key is ever exposed or broken by an attacker, they will still not be able to break all the past communications you have ever had because those ephemeral keys are temporary, never transmitted over the network, and destroyed after use. However, very few SSL/TLS clients (ie. browsers) actually support these cipher suites, and these cipher suites can also be *dramatically* slower. If you notice that swamp is suggesting your SSL/TLS server is slower than you think it is, it's possible that both swamp *and* the server support these ephemeral cipher suites and so are using them. This is actually a feature, it diagnoses what your server prefers to use when given the option. If this performance is too slow (or not necessary), consider disabling ephemeral cipher suites on the server. If you wish to leave the server as-is, but benchmark the server using different cipher-suites (eg. the ones used most commonly by web browsers), use the "-cipher" switch to provide an OpenSSL-compatible cipher-string that controls the list of available (and rejected) cipher-suites. Consult the OpenSSL documentation for more information. -session [Optional, default is "s"] As stated further up, the cryptograhic operations in SSL/TLS handshaking can be very computationally expensive. These are the "PKC" operations (eg. RSA, DSA) involved in negotiating an SSL/TLS *session* over which the actual data will be carried (and encrypted). Once that session is established however, the on-the-wire encryption has a considerably lower overhead in most circumstances. As such, SSL/TLS supports a mechanism called "session resuming" - if two parties have already conducted an SSL handshake before, they closed the session cleanly when they disconnected, and both are satisfied to try and use that session again, they can agree to "resume" a previous session. The result is that the handshake still takes place, but involves less "discussion" and involves absolutely no "PKC" operations. Thus a session-resume is usually a lot faster and requires a *lot* less computation, particularly on servers. If the client does not ask to resume a session, or if it does but the server declines the offer (eg. perhaps it did not cache/remember the session from last time, or perhaps it has timed-out the previous session), then the "resume" will automatically rollback to negotiating a brand new session. Web browsers in particular usually support "session resuming" for the reason that they can download a secure page, then go back and grab any dependant objects securely without having to do a complete SSL/TLS handshake for every individual request - they simply resume the session that was used to download the original page. Some even remember and attempt to resume sessions from one web page to the next. The security arguments behind all this are often deeply obscured by "user friendly" considerations - but that's for another rant at another time. You can have "swamp" attempt to use a mixture of new sessions (the normal behaviour) with session resumes (what this switch is for) by providing a string containing a pattern of 's' and 'r' characters. If "-num" is more than 1, each simulated client will observe this pattern independantly of each other. Eg. "srrsrrrr" will negotiate a session, then in the next two requests attempt to resume the previous session, then negotiate a new session, and finally attempt to resume the previous session for the next four requests. It will then loop back to the start of the string and start again. When using session strings containing "r" characters (resumes), you can expect the perceived performance of the SSL/TLS server to go up - because proportionally less of the operations/requests will actually involve the server having to perform expensive PKC operations. NB: The (+xx,-xx) output in swamp, if you use the "-update" switch, indicates the number of "r" resume attempts that *did* result in a session resume and the number that did not (and thus resulted in full session negotiations instead). This allows you to see how well your server is actually dealing with (and honouring) session resumption. -session_ids [Optional] The sessions (as outlined in the previous item) used in SSL/TLS are represented by "session id"s (which doesn't contain the security-critical keying information, it's simply like a binary "name" for a session). When a client asks a server to resume a session, it will send the "id" of the session it wishes to resume to the server who will look for matching session data internally. If it finds a match, and likes what it finds, it will accept this request otherwise reject it and continue with a full negotiation. If you specify this switch, *every* operation (be it a full negotiation, a resume attempt that worked, or a resume attempt that rolled back to a full negotiation) will log the session_id it used to stdout. This results in one long, noisy line of output for every single operation so you should be careful with how you record this output (and perhaps consider using "-count" or "-time"). This is mostly for those doing server debugging and in most cases will cause swamp itself to slow down dramatically. -engine [Optional, default is to let OpenSSL choose] If the version of OpenSSL swamp was built with supported the "ENGINE" API, then this switch will be available, otherwise swamp will not be compiled with this switch. The "id" is the name OpenSSL uses to identify the particular "ENGINE" you wish to use for performing the cryptographic operations swamp needs to operate. This can allow you to use hardware cryptographic acceleration with swamp, for example. The default is that no ENGINE will be specified and all cryptographic operations will be performed with OpenSSL's builtin software implementations. This is equivalent to specifying the "openssl" ENGINE. Distributing swamp requests across multiple servers (advanced) -------------------------------------------------------------- -distribute [Optional, default is to let swamp choose] Swamp can be configured to spray subsequent requests across multiple servers in various patterns. If "-connect" is specified times, the servers are added in order to an internal server list and are referred to be the numbers "1" through "". By default, swamp will then configure itself to send requests to each of those servers in order before starting again at the beginning of the list. NB, if "-num" is specified and greater than 1, then each of these simulated "parallel" clients will perform this cycle pattern independantly of the others, and each will start their cycle from subsequent servers (though this is simply a "starting" point, the timing of the communications and server responses usually result in some getting ahead/behind others and sometimes settling into "resonance" patterns). The "-distribute" switch allows the user to override the pattern with which multiple servers are swamped - this can achieve two things; (i) testing a server cluster's ability to resume sessions on one node that were negotiated on another node, and (ii) weighting the load on particular servers relative to others. - (i) If a "-session" switch is used, eg. "-session srr", then as each simulated client cycles across the servers for subsequent requests it will also be cycling across the session pattern to decide whether or not to try and resume sessions. To ensure the session caching behaviour between the server nodes is being tested properly, it may be desirable to override the order the servers are hit with. Eg. if there are three servers, the "srr" pattern will always negotiate new sessions with the same server and the other two will always be asked to resume those sessions. In this case, one might set "-distribute 1,2,3,2,3,1,3,1,2" - ensuring that all servers will receive both new session negotiations and attempts to resume sessions negotiated from the others. It also keeps the weighting balanced so that each server is receiving the same number of requests over time. - (ii) If you specify two servers with two "-connect" switches but do not specify any "-distribute" switch, then subsequent requests will alternate between the servers. If one server is an unoptimised bloated server running on a 66Mhz 486 and the other is a kernel-based web server running on a 2Ghz Athlon, then both will still be receiving the same number of requests over time and the stronger server will barely register a blip on the CPU usage. You can tweak this, for example, by specifying a non-default pattern for the order of requests, eg. "-distribute 1,2,2,2,2,2,1,2,2,2,2". This would ensure the the second server gets 4.5 times more requests than the first server over time. ==================================== Section III: Hints about using swamp ==================================== Controlling how long swamp runs ------------------------------- If you don't want to run "swamp" indefinitely, then use either the "-count" or "-time" switch to set it to stop after a certain number of requests or seconds (respectively). Making sure the speeds you observe are accurate ----------------------------------------------- (1) Ephemeral cipher suits; If you run swamp against your server and you think; "I'm sure my server is faster than *that*!", try specifying something like; -cipher RC4-MD5 when executing swamp and compare the results. Often server applications or users just blindly assume OpenSSL will control cipher-lists the way everyone could possibly want them, despite the fact that it usually gives preference to extremely slow (albeit more secure) cipher-suites, or more importantly, to cipher-suites whose session setup workload is heavier (such as the EDH cipher-suites). This is because EDH-RSA-DES-CBC3-SHA (for example) has "perfect forward secrecy", whereas RC4-MD5 does not. RC4-MD5 is however a good deal faster. Similarly DES-CBC3 based suites normally have priority over IDEA and RC4, because generally (triple)DES is the cipher people tend to trust most. It's also slower than the others. You can use "-cipher" to control the cipher-suites you want to test with, but you should also consider that perhaps what you *really* want to do is to prevent the *server* using certain cipher-suites. Changing your command-line for "swamp" does not change what clients can use with your server, and that is perhaps where you need to turn your attention if you think the speed you observe is unacceptable. (2) Swamp overhead; It's always a good idea to keep an eye on swamp itself. For example I have heard of one user who couldn't understand why he couldn't obtain higher numbers than a certain value he was seeing, despite the fact his server appeared to be comparitively idle during the testing. The reason was that he had been using client certificates in his tests and the machine swamp was running on was running at capacity while (unsuccessfully) trying to saturate the server. Running "top" during a practice run with swamp can help indicate whether the client machine itself is going to have an affect on your results. BTW: it's better to not run anything non-trivial while running "swamp" to gather benchmarks for this reason - a spike of CPU usage by another application (even switching desktops in X) can make it appear that the server performance went down temporarily when in fact it was the "swamp" host responsible for the drop. (3) Network overhead; Even if you are testing a secure webserver by sending a very small request (eg. "GET /\r\n") and receiving a very small static page in response (eg. a short 20-byte HTML document), the SSL/TLS layer will probably result in the entire operation taking a good deal more bandwidth than the "cleartext" indicates. It's typical, for example, that such an example might result in around 5Kb of data between the client and the server. Moreover, that data is actually a number of (typically) smaller packets of data travelling over the network, and together with other TCP network considerations (IP protocol overheads, transmission error correction, (dis)connection overhead, etc) can actually take a bigger chunk out of your bandwidth than you'd expect. Eg. if your network is 10 megabit ethernet, you may find that the "5Kb" of data actually has the same impact, in terms of how far you can scale your tests, as a single 15Kb UDP message would. If that were so, for example, the 10 megabit ethernet link would then have a theoretical upper-limit of 83 operations a second. Even if your server is capable of many-thousand SSL/TLS session negotiations a second and your host system running "swamp" is similarly up to the job, the network (or the network cards) can and do impose their own limits that can skew the results you obtain. Monitor network usage during practice runs with swamp and check that network throughput is not the limiting factor in your testing. Or if it is, accept that network throughput may be the factor that most limits your server's maximum possible performance and *not* considerations about SSL/TLS or application performance. In situations where the server has multiple network feeds to avoid this problem, but trying to use "swamp" can not, the easiest way to test is to run multiple copies of swamp from different client machines. Even this requires that the network links between those client machines can scale to the capacities you are interested in. If the server has 10 high-speed network feeds coming in, but all your test client machines are hitting the server over the same network interface, you may still find the throughput on that interface becomes the limiting factor. Ie. use a little common sense with your network. :-) Simulating realistic loads -------------------------- swamp is undergoing some active development to make it possible to simulate SSL/TLS loads more realisitically, but for now "all-out-assault" is the only language swamp understands. Ie. you configure whatever settings you want, including the level of "parallelism" you want (using "-num"), and then swamp will simply operate those simulated clients to always (if possible) be waiting on the server. The result is that if the client machine is fast enough to do it, the server will be pushed until its own performance and responsiveness is what is holding swamp back from running faster. That in itself isn't terribly real-worldly, but fortunately most of the things about the real-world that you might be scared about and thus, that you want to test your server for, can be reasonably simulated using this approach. (1) Session resumes If you read the documentation of the "-session" switch, you'll realise that not all SSL/TLS activity on your server will necessarily involve complete SSL handshakes. Some webservers will show you, if you enable detailed logging, that a surprisingly high proportion of "secure" traffic is travelling over SSL/TLS sessions that were *resumed*. A resumed session imposes a significantly different profile on your server, network, and latency than a full session negotiation does, and thus it requires some thought when attempting to simulate what your server will face in the real world. The best bet is to get some idea of what sort of behaviour you expect in this regard (eg. using web server logs from the "real world" or by asking someone else with experience in this sort of thing) and try to simulate that. If you read the previous subsection you will already know that networking *can* be a bottleneck. If you get a higher proportion of session resumes than you expect, this could result in your server's computation capacity being more than you needed and/or your network capacity being less than you need. Similarly a higher proportion of full session negotiation than you hoped for (eg. 9am Monday morning will probably experience more than its fair share of "new" traffic) can hit your server with a massive "spike" of computational overhead that can quite conceivably grind everything else to a halt. Moreover, if that traffic continues to arrive at a rate faster than you can process it, each additional client will perceive an response time that grows without limit. Many could simply think your service has "gone down" when in fact it's just bad capacity planning causing insane response times. Even worse, this could be misdiagnosed as *network* congestion rather than computational overload - resulting in wasted time and money attempting to fix the wrong thing. (2) Session caching A server can only honour a client's attempt to resume a session if it has "remembered" the session from last time they spoke together (and if the server "likes" the session enough to resume it). If you are using load-balancing solutions to allow multiple servers to split the workload ("NAT" is a common technique for this), or you are distributing requests to multiple servers, you may find the session caching problem backfires on you. As mentioned, resuming SSL/TLS sessions requires a lot less computational overhead on the server than a full session negotiation does. However, if each connection into "the server cluster" is directed to a server chosen according to "load", "round-robin", or any other semantic that isn't SSL/TLS-sensitive, you may find that most session resume requests are *never* being honoured because they're not being directed to the server that remembers the session the client wants to resume. If your expected session-resume proportions are high, adding a couple of servers with a NAT switch can make your real-world capacity go *down*. Yes, I'm serious. If there's a distributed session cache being shared by all the web-servers then this should not be a problem. To test stuff like this, you can run "swamp" tests to not (just) measure throughput, but to measure session resuming behaviour. For this, use the "-session" switch, and the "-session_ids" switch if you really want nasty low-level details, and pay attention to the session resume "hit"/"miss" statistics. In a NAT-switching environment, you will be looking out for a high number of "miss"es. Post-processing a "swamp"ing ---------------------------- (1) If you want to script "swamp", possibly running lots of copies on the same or different machines, and be able to post-process the results, experiment with the "-csv" switch. It produces a comma-delimited output file that is indexed by absolute time (with an entry each second) - for NTP-synchronised test machines, this can allow you to collate statistics quite well for some large and distributed tests. If you're also interested in capturing stdout (for use with "-session_ids" or whatever) then "-nologo" may also be handy.