[PEAK] Re: [Web-SIG] PEAK now provides various WSGI gateway and server options

[Phillip J. Eby]

At 03:05 PM 10/13/04 -0500, Ian Bicking wrote:
>Phillip J. Eby wrote:
>>The CVS version of PEAK now offers three options for running WSGI 
>>applications: CGI, FastCGI, and SimpleHTTPServer.  For example, this command:
>>     peak launch WSGI import:my_app.application
>>will do this:
>>  1. Import 'application' from 'my_app', treating it as a WSGI application
>>  2. Start a SimpleHTTPServer listening to an arbitrary port on 'localhost'
>>  3. launch a browser window pointing to that local server
>
>I'm noticing that peak serve WSGI import:... does the same thing, but 
>without launching a web browser.

Yes, but it's less convenient to use since you have to set up a 
configuration file to specify the port and hostname and such.  "peak 
launch" selects an available port and tells your web browser about 
it.  However, if you want to use 'peak serve', you can put something like this:

   [peak.tools.server]
   url = "tcp://fqdn.goes.here:8000"

in a configuration file, and then point to it with PEAK_CONFIG.  E.g.:

   PEAK_CONFIG=myserver.conf peak serve WSGI import:my_app.application

Or, if you want to just make the whole thing an easy-to-run application:

   #!invoke peak runIni

   [peak.running]
   app = = commands.Alias(command=['serve','WSGI','import:my_app.application'])

   [peak.tools.server]
   url = "tcp://fqdn.goes.here:8000"

And then make the file executable, so you can run it directly.  Now, you've 
got a ready-made setup to run a specific application.  You can also use 
'launch' instead of 'serve'; it will start the web browser on the 'http' 
version of the given URL.


>Is there any way to start the server up on a known port and interface? 
>When I do "launch" it opens itself up in "localhost.my.hostname", and I'm 
>not sure where localhost.my.hostname is coming from.

 From 'socket.getfqdn(serversocket.getsockname())'.  Specifically, the 
default address is 'localhost:0', which translates to any available port on 
'localhost'.  Apparently, your local resolver considers your FQDN to be 
'localhost.my.hostname', so I'd check /etc/resolv.conf or some such if 
you're on a Unix-like machine.  If you're on Windows or OS/X machine, I 
have no idea what to do.

Your problem does suggest that maybe I should change local_server to 
consider its address to be whatever it was configured to be, and not ask 
for the "official" socket address.  That way, it won't rely on a properly 
configured resolver, just to set up a localhost server.


>I was able to get "peak CGI WSGI import:..." working successfully, so the 
>basic system is all installed and working.  I tried FastCGI a little, but 
>I got stuck on installing mod_fastcgi for the moment.  I'm assuming that 
>if I create a script like:
>
>#!/bin/sh
>peak FastCGI WSGI import:...
>
>In a .fcgi, executable script, with "AddHandler fastcgi-script .fcgi" in 
>my httpd.conf, it'll just work...?

Something like that, yes.  It's been a while since I used that 
approach;  I've mainly used stuff that's more like:

    <Files some_app>
    SetHandler fastcgi-script
    </Files>

For the most part, mod_fastcgi is a bitch to set up for non-trivial 
applications, even *with* the PEAK supervisor tool, as many of its options 
are either poorly documented, or buggy, depending on whether you consider 
the code or documentation to be the thing that's wrong.  :)


>I'm also not sure what the concurrency is for these.  Multithreaded, 
>multiple processes, single process?  Configurable?

CGI/FastCGI are both single thread, multi-process.  The supervisor is also 
multi-process, but forking.   If your application module wants to set up 
caches, import lots of modules, etc., this will be done in the parent 
process, so that child processes will already have the work done.


>   Does the supervisor start on its own, or does that have to be configured?

mod_fastcgi starts its own process manager as needed.  Based on the 
settings in httpd.conf, it will start multiple processes for you, up to the 
maximum you specify.  It will also kill them off by signalling them when 
they become idle.  (The PEAK FastCGI implementations detect this and shut 
down gracefully.)

If you are using PEAK's process supervisor tool (peak.tools.supervisor) to 
manage an application, then you should configure mod_fastcgi to start one 
and only one process for that application.  Or, you can have the 
application start independently, listening on a known socket (e.g 
/tmp/myapp.sock), and configure mod_fastcgi not to manage the start/stop of 
processes.  The  process supervisor will take care of the rest for you.  If 
you start a supervised application that's already running, the new copy 
will get ready to run, and then signal the old copy to terminate 
gracefully, allowing currently-running requests to finish.  This is 
intended to make it easy to do a "warm restart" of your application to e.g. 
upgrade the code of a production application.  Alternately, you can simply 
issue a soft kill signal to the running parent process, and mod_fastcgi 
will take care of restarting it, if you've used the "start exactly one" 
approach.

To run an app under the PEAK "supervisor" tool, you need to create a 
configuration file, at minimum, something like:

     #!invoke peak supervise
     Command FastCGI fd.socket:stdin WSGI import:my_app.application
     PidFile /var/run/my_app.pid

This assumes that your OS supports using PATH to interpret "#!" lines; if 
not, you'll need an absolute path to 'invoke'.  ('invoke' is a C program 
that comes with PEAK in the 'scripts' directory, that you can install to 
more easily use PEAK tools as interpreters.)

The 'FastCGI fd.socket:stdin' means to use standard input as the connect 
socket for FastCGI; if you are using the "standalone" configuration, you'll 
want to replace that with a 'unix:/path/to/a_socket' or 
'tcp://localhost:1234' URL, as appropriate.  (For more detailed info on 
PEAK socket URL's, see the 'peak.net.sockets' module.)

The 'PidFile' spec is required; it's how the supervisor ensures that 
there's only one "master" process for the application at a given time, and 
it also makes it easy to shut down the application.  (There are also some 
other files used, whose names default to variations on the PidFile's 
filename, such as the "startup lock" file and the "pid lock" file; see the 
"Supervisor.xml" file in the 'peak.tools.supervisor' package directory for 
detailed info on these and all other configuration options for the 
"supervise" tool.)

Anyway, the configuration file can contain other options, like:

   MinProcesses 1       # Always have one request-handling process
   MaxProcesses 4       # and up to 4 if needed
   StartInterval 15s    # Don't start children more often than 1 per 15 seconds
   Import some.module   # force module to be imported in parent, that child 
might need

Note that 'Import' directives do not do anything with the contents of the 
named module; they just ensure the module is imported before the supervisor 
considers itself "started".  This is useful if your application's initial 
import doesn't load all the modules it's going to use, and you don't want 
to slow down the startup of new child processes by making them import the 
module.

Whew.  Anyway, so, the minimum to use PEAK's supervise tool in place of the 
mod_fastcgi process supervisor is to make a configuration file specifying 
the command and pidfile, and it should be run using 'peak 
supervise'.  Ideally, you can do that with a '#!' line as shown above, but 
you can also do it with a shell script, e.g.:

   #!/bin/sh
   peak supervise config_file_for_my_app

Note that you can probably get by for a while without PEAK's supervise 
tool; it's fairly "industrial strength" and exists mainly to work around 
performance flaws in mod_fastcgi's process manager that affect 
slow-starting applications that need multiple processes in order to handle 
the server's request volume, and to make it easier to control a running 
application (e.g. easy warm restart).  If you don't have an application 
that costs measurable amounts of money for every second of delayed 
response, you may not need "peak supervise".

Finally, note that there's a very nice tutorial at

    http://peak.telecommunity.com/DevCenter/IntroToPeak

that covers lots of basic "how to set up configuration files and make them 
executable" stuff for PEAK.  There's also some useful information in 
INSTALL.txt, under "SCRIPTS, BATCH FILES, and #!":

    http://peak.telecommunity.com/doc/INSTALL.txt.html