NAME
    HTTP::LoadGen - a HTTP load generator toolset

SYNOPSIS
     use HTTP::LoadGen;

     use HTTP::LoadGen qw/:all/;

     # ask import() to replace the built-in 'rand' operator by our
     # thread-specific RNG (uses *CORE::GLOBAL::rand)
     use HTTP::LoadGen qw/-rand :all/;

     ######################
     # the load generator #
     ######################

     HTTP::LoadGen::loadgen \%config;

     #######################
     # auxiliary functions #
     #######################

     # process management

     # create a collection of worker processes
     $handle=HTTP::LoadGen::create_proc $nproc, $inithnd, $handler, $exithnd;

     # start main processing and wait for then to finish
     %result=%{HTTP::LoadGen::start_proc $handle};

     # thread management

     # create a collection of threads
     $sem=HTTP::LoadGen::ramp_up
         $procnr, $nproc, $start, $max, $duration, $handler;

     # wait for them to finish
     $sem->down;

     # idle a bit
     HTTP::LoadGen::delay $prefix, \%param;

     # get current thread number
     $nr=HTTP::LoadGen::threadnr;

     # get the configuration hash
     $config=HTTP::LoadGen::options;

     # get/set thread-specific user data
     $data=HTTP::LoadGen::userdata;
     HTTP::LoadGen::userdata=$data;

     # get/set thread specific random number generator
     $rng=HTTP::LoadGen::rng;
     HTTP::LoadGen::rng=$rng;

     # next random number
     $random=HTTP::LoadGen::rnd $max;

INSTALLATION
     perl Makefile.PL
     make
     make test
     make install

DEPENDENCIES
    *   perl 5.8.8

    *   IPC::ScoreBoard

    *   Coro

    *   AnyEvent

    *   Async::Interrupt

    *   Net::SSLeay

DESCRIPTION
    This module implements a multi-process and multi-thread load generator
    for HTTP. It uses Coro threads. So, in reality it does not use threads
    but event-based IO.

  Features
    *   limited support for SSL connections

    *   keep-alive connections

    *   configurable delay before and after each request

    *   run a list of URLs many times

    *   compute next URL based on the current request

    *   DNS cache can be preinitialized

    *   slow ramp up

    *   request bodies

    *   custom request headers

  Overview
    Note, this POD is best view via Apache2::PodBrowser.

   Parallelism
    The load generator follows a 2-level supervisor-worker pattern. The
    central function, "loadgen", creates a certain number of child
    processes. Each child process then creates in a slow ramp up phase
    worker threads up to a configurable total upper thread limit.

    The thread limit is configured independent on the number of worker
    processes. You configure a number of processes that is about 1.5-5 times
    the number of available CPUs. The number of threads can then be say 50
    or 500 or even 5996 or so. Processes and threads are numbered starting
    from 0.

    So, assuming there are 3 processes and 10 threads configured the
    following table shows how the threads are spread among the processes:

     Process | Threads
     --------+------------
           0 | 0 3 6 9
           1 | 1 4 7
           2 | 2 5 8

    Process 0 will run 4 threads, the other 2 processes 3 threads each. The
    number of threads per process can be calculated as:

     $TotalThreadCount / $NProc + ($ProcNr < $TotalThreadCount % $NProc)

    where $NProc is the number of processes used, $ProcNr the number of the
    current process and $TotalThreadCount the system-wide thread number.
    $ProcNr ranges from 0 to "$NProc - 1".

    At the beginning ot the ramp-up phase each process starts up a certain
    number of threads (maybe 0) to reach the configured start-up thread
    number. The configured ramp-up duration then determines in which
    intervals new threads are added. So assuming the threads run long enough
    you start up with a certain level of parallelism which increases
    linearly over a certain time interval up to the configured maximum.

   The Scoreboard
    The multi-process model of "HTTP::LoadGen" means that each process knows
    only about its own threads. Sometimes you may want to log for example
    the overall number of active requests when a new request is started. Or
    you may want to increment a shared variable for each request to see the
    progress of an active load run.

    HTTP::LoadGen::ScoreBoard or IPC::ScoreBoard may be used to achieve
    that.

   The Logger
    "HTTP::LoadGen" doesn't have logging built-in. Instead
    HTTP::LoadGen::Logger is provided.

   Random numbers and repeatable results
    "loadgen" needs for certain operations random numbers. If you need
    repeatable results that is you want to repeat the same test with the
    same delays between requests later then you need the same sequence of
    random numbers for each thread. But the random number generator built-in
    to Perl is process-wide.

    "HTTP::LoadGen" provides an interface to set an RNG per thread. CPAN
    modules like Math::Random::MT use an object oriented approach. So, it
    may be a good idea to create such an object for each thread and register
    it with "HTTP::LoadGen". A "ThreadInit" handler is a good place to do
    that.

    If the "import" function of "HTTP::LoadGen" is called with the "-rand"
    parameter ("use HTTP::LoadGen qw/-rand/") the Perl built-in "rand"
    operator is overwritten (by means of *CORE::GLOBAL::rand) to use the
    thread-specific RNG. Though, occurences of "rand" in the code that have
    been compiled before "HTTP::LoadGen" is loaded continue to use the
    built-in operator.

   Phases
    There are several phases in the lifetime of a load run, a process, a
    thread or a request that can be hooked. A hook is a code reference.

    ParentInit and ParentExit
        these 2 hooks run in the parent process. The "loadgen" function
        checks the configuration and then calls "ParentInit". "ParentExit"
        is called just before "loadgen" returns.

        "ParentInit" can start Coro threads. They will run while the process
        is waiting for the worker children to finish.

        One thing to consider to do in a "ParentInit" hook is the creation
        of a HTTP::LoadGen::ScoreBoard.

    ProcInit and ProcExit
        these 2 hooks are called in each worker process. When a worker
        process is started "ProcInit" is called. But before the actual load
        generation is started the process waits for a signal from the parent
        process that is sent when every worker process has finished its
        "ProcInit" phase. So, even if the "ProcInit" phase takes a bit
        longer it does not influence the load generation other than it is
        started a bit later.

        "ProcInit" can start Coro threads. They will run while the process
        is waiting for the signal from the parent process to start load
        generation and of course after that until they finish.

        One thing that should probably be done in a "ProcInit" handler is
        reseeding of the random number generator. If you need repeatable
        results then you need a random number generator per thread. The
        built-in RNG is no help then. However, there are several object
        oriented RNGs on CPAN. Use the "rng" function to set a
        thread-specific RNG and "rnd" instead of the built-in "rand" to call
        it.

        Another one is the creation of a logger, see HTTP::LoadGen::Logger.

        "ProcExit" is called after the load generation is over just before
        the worker process exists.

        If a "ProcExit" hook is installed its return value determines the
        exit code of the worker process.

        Closing the logger would be good here.

    ThreadInit and ThreadExit
        these 2 hooks wrap the load generation phase of each thread. If a
        thread needs private data the "ThreadInit" handler can create and
        return it. It is then passed to all other hooks called during the
        lifetime of the thread.

        Things to consider to do in a "ThreadInit" handler would include

        1. registration of a thread-specific random number generator
        2. registration of the thread with the scoreboard

    InitURLs
        during the load generation phase each thread fetches a list of URLs
        several times. The actual list is not given as an array or similar
        but as an interator generator, that is a function that returns a
        function that returns an URL to be fetched. The "InitURLs" iterator
        generator is called each time a thread starts another round of
        fetching URLs. The iterator itself is then called to get the next
        URL to be fetched. If it returns "undef" or the empty list the
        current round is over. Then if the configured number of rounds is
        reached the thread ends or the next round is started (and the
        "InitURLs" handler is called again).

        In most cases this complex URL handling is not necessary. Instead
        one simply needs to check off all items of a predefined list. For
        these situations a few predefined iterator generators exist.

    ReqStart and ReqDone
        these 2 hooks wrap each HTTP request.

        Here the request would be accounted with the scoreboard. In
        "ReqDone" logging would occur.

  HTTP::LoadGen::loadgen \%data
    "loadgen" is the central function of this module. It starts up child
    processes, creates threads, generates the load and waits for that all to
    finish. It returns when all is done.

    The %data hash passed by reference configures "loadgen" and describes
    what to do. "loadgen" copies the hash so that the original hash is not
    changed but the copying is not recursive. If a hash value is an array
    and one of the hooks changes it that change will be reflected in the
    original %data hash. However, if you add new hash elements in a hook
    function they won't show up in %data after "loadgen" returns.

   Request descriptor and return element
    A number of elements of the %data hash are hook functions. Some of them
    are passed parametes $rq and/or $rc. Both are lists. HTTP::LoadGen::Run
    exports constants to access the list elements. The structure of the
    request descriptor $rq is explained under URLList below.

    For the lack of a better place the $rc element is described here.

    RC_STATUS (0)
        the HTTP status code. If the request failed because the connection
        couldn't be established a code 599 is set here. "RC_STATUSLINE"
        describes the problem in more detail in that case.

    RC_STATUSLINE (1)
        the HTTP status message. If the server responds with the following
        first line for example:

         HTTP/1.1 501 Method Not Implemented

        "RC_STATUS" is 501 while "RC_STATUSLINE" is "Method Not
        Implemented".

    RC_HTTPVERSION (2)
        the server HTTP protocol version. Normally 1.1 or 1.0.

    RC_STARTTIME (3)
        when the request has been started, fractional number.

    RC_CONNTIME (4)
        when the connection has been established, fractional number.

    RC_FIRSTTIME (5)
        when the first line of output has been received, fractional number.

    RC_HEADERTIME (6)
        when the response HTTP header has been completely received,
        fractional number.

    RC_BODYTIME (7)
        when the response body has been completely received, fractional
        number.

    RC_HEADERS (8)
        a hash containing the response HTTP headers. The values of this hash
        are arrays since HTTP header fields can be given multiple times.
        Keys (header names) are converted to lower case.

        Example:

         {
          'content-type' => ['text/html; charset=iso-8859-1'],
          'connection' => ['close'],
          'date' => ['Sun, 04 Jul 2010 18:21:12 GMT'],
          'content-length' => ['217'],
          'allow' => ['GET,HEAD,POST,OPTIONS,TRACE'],
          'server' => ['Apache'],
         }

    RC_BODY (9)
        the response body

    RC_DNSCACHED (10)
        boolean: has the DNS cache lookup resulted in a hit (1) or miss (0)?

    RC_CONNCACHED (11)
        boolean: has the has a kept-alive connection been used?

   The %data hash
    So, what can be specified in %data? Note, all keys here are case
    sensitive.

    NWorker (optional)
        specifies the number of worker processes to be used. Default is 1.

    RampUpStart (optional)
        the number of threads to started up immediately (after the
        "ProcInit" phase is over). Default is 1 thread per worker process,
        that is "NWorker".

    RampUpMax (optional)
        the number of threads that have to be started up after the ramp-up
        phase is over. That means all processes together will start this
        number of threads. If a thread finishes before the ramp-up phase is
        over this maximum level of parallelism will never be reached.

        Default is the same as "RampUpStart".

    RampUpDuration (optional)
        the duration of the ramp-up phase in seconds (may be fraction).

        Default is 300 (5 minutes).

    ParentInit (optional)
        the "ParentInit" handler called as

         $data->{ParentInit}->();

        One thing to do here is to create a scoreboard for interprocess
        communication, see HTTP::LoadGen::ScoreBoard or IPC::ScoreBoard.

        Example:

         ParentInit=>sub {
           # no parameters

           # create scoreboard
           # options() returns the config hash itself. The NWorker parameter
           # is known. SbSlotsz and SbExtra are new. This is to demonstrate
           # that the hook routines can access the configuration and evaluate
           # and even add custom parameters.
           HTTP::LoadGen::ScoreBoard::init_once
               @{HTTP::LoadGen::options()}{qw/NWorker SbSlotsz SbExtra/};
         }

    ParentExit (optional)
        the "ParentExit" handler called as

         $data->{ParentExit}->();

        If a scoreboard is used remember to disconnect.

        Example:

         ParentExit=>sub {
           # no parameters
           undef HTTP::LoadGen::ScoreBoard::scoreboard;
         }

    ProcInit (optional)
        the "ProcInit" handler called as

         $data->{ProcInit}->($procnr);

        $procnr is the 0 based number of the process. It ranges up to
        "NWorker - 1".

        If you plan to use the built-in random number generator this hook is
        a good place to reseed it.

        Another good thing to do here is to acquire a logger.
        HTTP::LoadGen::Logger may help here.

        If a scoreboard is used save $procnr as slot number.

        Example:

         ProcInit=>sub {
           my ($procnr)=@_;

           # set my slot number
           HTTP::LoadGen::ScoreBoard::slot=$procnr;

           # acquire a logger
           $logger=HTTP::LoadGen::Logger::get;
         }

    ProcExit (optional)
        the "ProcExit" handler called as

         $rc=$data->{ProcExit}->($procnr);

        The return value of this hook determines the exit code of the
        process. If omitted the exit code is 0.

        The thing to do here is perhaps to close the logger.

        Example:

         ProcExit=>sub {
           my ($procnr)=@_;
           $logger->();         # close the logger
         }

    ThreadInit (optional)
        the "ThreadInit" handler called as

         $userdata=$data->{ThreadInit}->();

        The return value of this hook is saved as thread-specific user data.

        This hook is a good place to initialize a thread specific random
        number generator if you need repeatable results.

        Example:

         ThreadInit => sub {
           # no parameters

           # thread accounting with the scoreboard
           HTTP::LoadGen::ScoreBoard::thread_start;

           # set a thread specific RNG
           HTTP::LoadGen::rng=Math::Random::MT->new(@seed);

           return [];             # initializes thread specific user data
         }

    ThreadExit (optional)
        the "ThreadExit" handler called as

         $data->{ThreadExit}->();

        Remember to notify the scoreboard.

        Example:

         ThreadExit=>sub {
           # no parameters
           HTTP::LoadGen::ScoreBoard::thread_done;
         }

    ReqStart (optional)
        the "ReqStart" handler called as

         $data->{ReqStart}->($rq);

        $rq is an array specifying the current request. It is generated by
        the URL iterator. The "ReqStart" handler is allowed modify the
        array.

        If a scoreboard is used check in the request. One can also save some
        current state from the scoreboard to the thread-specific storage to
        log it later.

        Example:

         ReqStart=>sub {
           my ($rq)=@_;
           HTTP::LoadGen::ScoreBoard::req_start;
           @{HTTP::LoadGen::userdata()}=(HTTP::LoadGen::ScoreBoard::thread_count,
                                         HTTP::LoadGen::ScoreBoard::req_started,
                                         HTTP::LoadGen::ScoreBoard::req_success,
                                         HTTP::LoadGen::ScoreBoard::req_failed);
         }

    ReqDone (optional)
        the "ReqDone" handler called as

         $data->{ReqDone}->($rc, $rq);

        $rc is the request result. See "run_url" in HTTP::LoadGen::Run. The
        "ReqDone" handler may modify this array. But it's not recommended to
        do that.

        $rq is an array specifying the current request.

        Here you do request accounting and of course logging.

        Example:

         ReqDone=>sub {
           my ($rc, $rq)=@_;

           HTTP::LoadGen::ScoreBoard::req_done
               scalar($rc->[RC_STATUS]=~/^[23]/), $rc->[RC_HEADERS], $rc->[RC_BODY];

           $logger->(HTTP::LoadGen::threadnr,
                     @{$rc}[RC_DNSCACHED, RC_CONNCACHED],
                     @{HTTP::LoadGen::userdata()},
                     HTTP::LoadGen::ScoreBoard::req_success,
                     HTTP::LoadGen::ScoreBoard::req_failed,
                     HTTP::LoadGen::ScoreBoard::header_count,
                     HTTP::LoadGen::ScoreBoard::header_bytes,
                     HTTP::LoadGen::ScoreBoard::body_bytes,
                     $rc->[RC_STARTTIME],
                     $rc->[RC_CONNTIME]-$rc->[RC_STARTTIME],
                     $rc->[RC_FIRSTTIME]-$rc->[RC_STARTTIME],
                     $rc->[RC_HEADERTIME]-$rc->[RC_STARTTIME],
                     $rc->[RC_BODYTIME]-$rc->[RC_STARTTIME],
                     $rc->[RC_STATUS], $rc->[RC_STATUSLINE],
                     length($rc->[RC_BODY]),
                     sprintf('%s(%s://%s:%s%s)',
                             @{$rq}[RQ_METHOD, RQ_SCHEME, RQ_HOST, RQ_PORT, RQ_URI]));
         }

    times (optional)
        the number of times the URL iterator is charged. That many times the
        URL list is fetched.

        If omitted or "<=0" the test runs forever.

    dnscache (optional)
        "loadgen" caches DNS query results. One can prevent DNS queries
        completely in 2 ways. One of them is to provide a hash here that
        maps names to IP addresses. The other is to have the URL iterator
        generate IP addresses instead of host names and optionally "Host"
        request header fields.

        Another use of this item is to cheat host name resolution. One can
        for example test a newly installed or development server while the
        real server continues to work unaffected.

        Example:

         dnscache=>{
                    'foertsch.name'=>'127.0.0.1',
                   },

    InitURLs (either InitURLs or URLList or both must be present)
        "InitURLs" initializes the URL iterator. It may be a string
        describing one of the predefined iterators or a "CODE" reference.

        In the latter case it is called without parameters as

         $it=$data->{InitURLs}->();

        It is expected to return a function that when called as

         $new_rq=$it->($rc, $rq);

        returns the next request item or "undef" when it runs out of items.
        The parameters $rc and $rq describe the previous request ($rq) and
        its result ($rc).

        For a description of the $rq and $new_rq format see URLList below.

        Example:

         InitURLs=>sub {
           my $url=[qw!GET http foertsch.name 80 /-redir!,
                    {
                     keepalive=>KEEPALIVE,
                     headers=>[
                               'X-auth'=>1, # necessary to trigger 401 for that URL
                              ],            # it also shows a custom request header
                    }];
           return sub {
             my ($rc, $rq)=@_;
             if( $rc->[RC_STATUS]==401 ) {
               # redo with Authorization header
               push @{$rq->[RQ_PARAM]->{headers}}, Authorization=>'Basic YmxhOmJsdWI=';
               return $rq;
             }
             my $new_rq=$url;
             undef $url;                # next time return undef (out-of-requests)
             return $new_rq;
           };
         }

        The iterator generator initializes the variable $url and then
        returns a closure. Hence, $url is a static variable with respect to
        the returned iterator.

        The iterator itself checks the HTTP code of the previous request. In
        case of a 401 (Authorization Required) it adds an "Authorization"
        header to the request header list and retries the operation.

        If the previous operation has ended with an other HTTP code it
        copies $url to an auxiliary variable, undefines it and returns the
        auxiliary variable. Thus, only the first time the iterator is called
        it returns $url. After that it is always "undef" which signals
        *Out-of-Requests*.

        If "InitURLs" is a string it is the name of a predefined iterator
        generator.

        Example:

         InitURLs=>'follow'

        There are currently 4 such generators. All of them expect an
        "URLList" (see below) to be provided.

        default
            simply walks the "URLList" from start to end.

            This one is also used if "InitURLs" is omitted.

        random_start
            similar to "default" but starts at a random offset in "URLList".
            At the end of the list it continues at the beginning until all
            "URLList" elements are done once.

        follow
            similar to "default" but if a request results in a "3xx" HTTP
            code and a "Location" header is provided by the server it tries
            to follow it recursively.

            If the request starting a series of redirections contains a
            "postdelay" statement (see below) the delay is postponed until
            after the last request of the series. Subsequent requests are
            issues without delay.

            Subsequent requests inherit the "User-Agent" and "Referer" HTTP
            headers of the originating request, see follow_3XX() below.

            This iterator is a bit special in that can turn other iterators
            into following ones. Normally an iterator generator is called
            without parameters. This one can take one parameter that in turn
            may be an iterator. It returns then a following iterator based
            on the passed one.

            Infact, the built-in "random_start_follow" iterator is
            implemented for example as

             register_iterator random_start_follow=>sub {
                @_=get_iterator('random_start')->();
                goto &{get_iterator 'follow'};
             };

            To turn your own iterator into a following you could write:

             InitURLs=>sub {
               return get_iterator->('follow')->($my_own_iterator);
             }

            where $my_own_iterator is an iterator function.

        random_start_follow
            a combination of the 2 above.

        You can register your own named iterators by calling
        register_iterator below.

    URLList (either InitURLs or URLList or both must be present)
        See also InitURLs above.

        An "URLList" is an array of arrays. Each of these sub-arrays
        describes one request. If consists of 6 elements:

         [$method, $scheme, $host, $port, $uri, $param]

        $method is the HTTP request method, e.g. "GET", "POST", ...

        $scheme is either "http" or "https".

        $host is the hostname or IP address of the server, e.g.
        "foertsch.name" or 109.73.51.50.

        $port is the server port to connect. Usually port 80 is used for
        "http" and port 443 for "https".

        $uri is the request URI normally starting with a slash ("/"), e.g.
        "/impressum.html".

        $param is a hash with further options.

        To access the elements of a request description HTTP::LoadGen::Run
        exports a few constants. They may be used to increase readability.

         RQ_METHOD == 0
         RQ_SCHEME == 1
         RQ_HOST   == 2
         RQ_PORT   == 3
         RQ_URI    == 4
         RQ_PARAM  == 5

        Example:

         URLList=>[
                   [qw!GET http 109.73.51.50 80 /-redir!,
                    {
                     keepalive=>KEEPALIVE,
                     headers=>[
                               Authorization=>'Basic YmxhOmJsdWI=',
                               Host=>'foertsch.name',
                              ],
                    }],
                   [qw!HUGO https www.kabatinte.net 443 /!,
                    {
                     keepalive=>KEEPALIVE,
                     predelay=>0.5,
                     prejitter=>1,
                     postdelay=>3,
                     postjitter=>1.5,
                     body=>'blablub',
                    }]
                  ]

        This "URLList" contains 2 requests, one for a server with the IP
        address 109.73.51.50 and one for the host "www.kabatinte.net".

        The first one will send the following HTTP request to the server (IP
        109.73.51.50, port 80):

         GET /-redir HTTP/1.1
         Authorization: Basic YmxhOmJsdWI=
         Host: foertsch.name

        If you need more header fields, "User-Agent" for example, add them
        to the "headers" array of the options hash.

        The second request is converted into the following HTTP message sent
        over SSL to "84.38.75.176:443" assuming that "www.kabatinte.net"
        resolves to 84.38.75.176:

         HUGO / HTTP/1.1
         Host: www.kabatinte.net
         Content-Length: 7

         blablub

        Although no "Host" header is specified in the request element one is
        sent. If the request element does not contain a "Host" header one is
        added automatically based on $host and $port.

        You may also notice the "Content-Length" header. It is sent because
        a request body is specified (the "body" item in $param).

        So, what can be specified in the $param part?

        keepalive
            HTTP::LoadGen::Run exports 3 constants to be used as values.
            "KEEPALIVE_USE" permits to use a previously kept alive
            connection. "KEEPALIVE_STORE" allows to keep the connection
            alive after the request. "KEEPALIVE" combines both of the above.

            If you hate readability you can also use the numerical values:

             KEEPALIVE_USE==1
             KEEPALIVE_STORE==2
             KEEPALIVE==3

        predelay and prejitter
            These statements define a period to wait before sending the
            request. The wait is done after the request description has been
            pulled off the iterator but before the "ReqStart" handler is
            run.

            Both numbers can be fractions. Read them as

             predelay ± prejitter

            The actual waiting time is calculated as

             interval = predelay - prejitter + rand( 2 * prejitter )

            If "prejitter >= predelay" interval can become negative. In this
            cases you won't jump back in time but simply not wait.

            To achieve repeatable results a thread-specific random number
            generator must be used. See the "rng" function below.

        postdelay and postjitter
            The same as "predelay" but waiting occurs after the request is
            done or more precisely after the "ReqDone" handler returns.

        headers
            an array (not a hash!) of header fields to be appended to the
            HTTP request.

        body
            a request body

        conn_timeout
            here you can specify the return value of the prepare-callback
            function passed to "AnyEvent::Socket::tcp_connect" when
            establishing a connection.

            See AnyEvent::Socket for more information.

        timeout
            the "timeout" parameter used when a connection is converted into
            a AnyEvent::Handle object.

            See AnyEvent::Handle for more information.

        tls_ctx
            the "tls_ctx" parameter used when a connection is converted into
            a AnyEvent::Handle object.

            See AnyEvent::Handle for more information.

            By now AnyEvent::Handle supports SSL features like client
            certificates and server certificate verification. However, some
            things are still missing like SSL session caching. How about
            server initiates renegotiations I am not sure.

            Note, "conn_timeout", "timeout" and "tls_ctx" are not very well
            tested by now.

  Useful functions to be used in hooks
   HTTP::LoadGen::threadnr
    returns the number of the thread currently running.

   HTTP::LoadGen::userdata
    returns the thread-specific user data. Normally this is assigned to by
    returning something useful from a "ThreadInit" handler. But it's a
    lvalue-function. Hence the following will work too:

     # assign new thread-specific data
     HTTP::LoadGen::userdata={something=>'useful'};

   HTTP::LoadGen::options
    returns the copy of the configuration hash used by "loadgen()".

   HTTP::LoadGen::rng
    returns and sets the thread-specific random number generator. It sets
    the RNG used by "HTTP::LoadGen::rnd".

   HTTP::LoadGen::rnd $upper_limit
    use the thread-specific random number generator or if none set the
    built-in one.

    Returns a pseudo-random number.

   HTTP::LoadGen::delay $prefix, $param
    this function implements the "predelay" and "postdelay" operations.

    $prefix is a prefix, e.g. "pre" or "post".

    $param is a "RQ_PARAM" hash of a request descriptor containing keys
    "$prefix.'delay'" and "$prefix.'jitter'".

   HTTP::LoadGen::done
    if a thread needs a preliminary exit call

     HTTP::LoadGen::done=1

    in a "ReqStart" or "ReqDone" handler. The current request will be
    performed except for "postdelay". Then the thread finishes. This can be
    used to stop the run when a certain load level has been reached.

   $new_rq=HTTP::LoadGen::follow_3XX $rc, $rq
    This function implements the following part of the built-in "follow"
    iterator.

    It is called with the result and the request descriptor of the previous
    request and returns a new request descriptor if the result is a HTTP
    redirect.

    Otherwise an empty list is returned.

    The new request preserves the "User-Agent" and "Referer" request header
    fields.

  Other auxiliary functions
   HTTP::LoadGen::register_iterator $name, $code_ref
    registers a known iterator. This can be used by other modules, e.g.

     package My::Iterator;
     use HTTP::LoadGen ':all';
     BEGIN {
       register_iterator 'my_iterator'=>sub {
         ...
       };
     }

   $code_ref=HTTP::LoadGen::get_iterator $name
    returns an iterator by name.

   $handle=HTTP::LoadGen::create_proc $nproc, $init_hnd, $hnd, $exit_hnd
    create $nproc child processes and have them finish the "ProcInit" phase
    $init_hnd.

    $init_hnd and $hnd are passed just one parameter, the 0-based process
    number. $exit_hnd get that plus the scalar return value of $hnd. The
    return value of $exit_hnd determines the exit code of the child process.

    $init_hnd and $exit_hnd may be "undef". $hnd may not.

    Returns an opaque handle that can be passed to "start_proc".

   $status=HTTP::LoadGen::start_proc $handle
    When "create_proc" returns all children have finished their $init_hnd
    and wait for a signal to continue with $hnd. "start_proc" sends that
    signal and waits for all children to finish.

    It returns a hash that maps operating system process IDs to their exit
    code, killing signal and a coredump flag.

    Example:

     {
      '7273' => [7, 0, 0],     # PID 7273 exits normally with code 7
      '7275' => [0, 11, 1],    # PID 7275 has been killed by signal 11
                               #   + core has been dumped
      '7274' => [8, 0, 0],     # PID 7274 exits normally with code 8
     }

   $semaphore=HTTP::LoadGen::ramp_up $procnr, $nproc, $start, $max, $duration, $handler
    implements the ramp-up phase.

    returns a semaphore that can be used to wait for the created threads to
    finish. It waits only for the threads running in the current process:

     $semaphore->down;      # wait for my threads to finish

    "ramp_up" may finish almost immediately but may also take some time
    while the load generation is already running. It depends on the
    $duration parameter. Don't expect it to return before the load
    generation starts.

  Internal functions
    HTTP::LoadGen::conncache
    HTTP::LoadGen::tlscache

DEBUGGING
    Sometimes its useful to see what requests are made. If the environment
    variable "HTTP__LoadGen__Run__dbg" is set when HTTP::LoadGen::Run is
    compiled a source filter is used to compile in debugging output to
    STDERR.

EXPORT
    The following Exporter tags are defined:

    common
        exports "loadgen", "threadnr", "done", "userdata", "options", "rng",
        "rnd" and "delay"

    const
        exports all symbols that HTTP::LoadGen::Run exports by default.

    all all of the above.

        Additionally it pulls in HTTP::LoadGen::ScoreBoard and exports all
        that is exported by it.

        Also, HTTP::LoadGen::ScoreBoard is loaded. Then function named
        "get_logger" is created as an alias for "HTTP::LoadGen::Logger::get"
        and exported.

EXAMPLE CONFIGURATION
     #!/usr/bin/loadgen
     # -*-perl-*-

     use strict;
     use Math::Random::MT;
     use Coro;
     use Coro::Timer ();
     no warnings 'ambiguous';

     # possible hook parameters:
     # $procnr   -- the current process number 0 .. NWorker-1
     # $el       -- an URL element to fetch (ARRAY)
     #              use RQ_* constants from HTTP::LoadGen::Run to access
     # $rc       -- an result element (ARRAY)
     #              use RC_* constants from HTTP::LoadGen::Run to access

     my $logger;
     +{
       NWorker=>3,                  # use 3 processes
       RampUpStart=>2,              # start 2 threads immediately
       RampUpMax=>13,               # then add 11 threads over 5 seconds
       RampUpDuration=>5,           # that makes 2.2 new threads per second

       ParentInit=>sub {
         # no parameters

         # create scoreboard
         sbinit undef, options->{NWorker};
       },
       ParentExit=>sub {
         # no parameters
         undef scoreboard;
       },

       ProcInit=>sub {
         my ($procnr)=@_;

         # set my slot number
         slot=$procnr;

         # acquire a logger
         my $fmt='%-2d %d %d %2d %2d %3d %3d %.3f %.3f %.3f %.3f %.3f %s %d '.
                 "%s(%s://%s:%s%s) %s\n";
         $logger=get_logger undef, sub {sprintf $fmt, @_};
       },
       ProcExit=>sub {
         my ($procnr)=@_;
         $logger->();               # close the logger
       },

       ThreadInit=>sub {
         # no parameters

         # thread accounting
         thread_start;

         # set a thread specific RNG
         rng=Math::Random::MT->new(threadnr);

         return [];                 # initializes thread specific user data
       },
       ThreadExit=>sub {
         # no parameters
         thread_done;
       },

       ReqStart=>sub {
         my ($el)=@_;

         # request accounting
         req_start;

         # started - succeeded - failed = currently pending number of requests
         @{userdata()}=(thread_count, req_started-req_success-req_failed);
       },

       ReqDone=>sub {
         my ($rc, $el)=@_;

         # request accounting: HTTP status 2xx and 3xx are successful
         #                     other requests are counted as failures.
         req_done +($rc->[RC_STATUS]=~/^[23]/), $rc->[RC_HEADERS], $rc->[RC_BODY];

         $logger->(threadnr,
                   @{$rc}[RC_DNSCACHED, RC_CONNCACHED],
                   @{userdata()},
                   req_success,
                   req_failed,
                   $rc->[RC_STARTTIME],
                   $rc->[RC_CONNTIME]-$rc->[RC_STARTTIME],
                   $rc->[RC_FIRSTTIME]-$rc->[RC_STARTTIME],
                   $rc->[RC_HEADERTIME]-$rc->[RC_STARTTIME],
                   $rc->[RC_BODYTIME]-$rc->[RC_STARTTIME],
                   $rc->[RC_STATUS],
                   length($rc->[RC_BODY]),
                   @{$el}[RQ_METHOD, RQ_SCHEME, RQ_HOST, RQ_PORT, RQ_URI],
                   $rc->[RC_STATUSLINE]);
       },

       dnscache=>{
                  localhost=>'127.0.0.1',
                  'kabatinte.net'=>'84.38.75.176',
                  'www.kabatinte.net'=>'84.38.75.176',
                  'foertsch.name'=>'109.73.51.50',
                 },

       times=>3,                    # run the URL list 3 times

       InitURLs=>'random_start',

       URLList=>do {
         my $o={
                keepalive=>KEEPALIVE,
                qw!predelay 0.05 prejitter 0.1 postdelay 0.5 postjitter 1!,
               };
         [[qw!GET http foertsch.name 80 /-redir!, $o],
          [qw!HUGO https www.kabatinte.net 443 /!, $o]
         ];
       },
      }

SEE ALSO
    *   HTTP::LoadGen::Run

    *   HTTP::LoadGen::ScoreBoard

    *   HTTP::LoadGen::Logger

    *   loadgen

AUTHOR
    Torsten Förtsch, <torsten.foertsch@gmx.net>

COPYRIGHT AND LICENSE
    Copyright (C) 2010 by Torsten Förtsch

    This library is free software; you can redistribute it and/or modify it
    under the same terms as Perl itself, either Perl version 5.10.0 or, at
    your option, any later version of Perl 5 you may have available.

