[PEAK] PEAK design pointers
Phillip J. Eby
pje at telecommunity.com
Fri Mar 5 18:23:39 EST 2004
At 08:45 PM 3/5/04 +0000, Paul Moore wrote:
>What I'm trying to write is an application (more precisely, a
>framework) for database monitoring. I have a central repository
>database, and a large number (100+) of target databases. My
>application needs to connect to each target, run a query, collect the
>results and save them in a table in the repository. I'll ultimately
>have a number of different instances of this application, each
>collecting the results of a different query, and possibly handling the
>results in slightly different ways.
>Objects (components?) I see a need for include:
> * A "target database"
> * A "repository"
I assume these are DBAPI SQL databases?
> * A "thread pool" (too many targets to query in sequence - I need
> to do the queries in the background).
I'll probably be adding one of these to PEAK eventually, but you might want
to just use Twisted's threadpool support, i.e.
'reactor.deferToThread()'. The resulting Deferred objects are usable with
'peak.events' to control "tasks".
> * A "report" (to pack up the results of a run for publishing)
> * A "mailer" (reports are typically sent by mail)
>But I can't work out how to connect them all together (well, I can,
>but that's when all the nice modularity and flexibility breaks
Well, if I understand you correctly, I'd probably write something that
would spawn an events.Task for each of the databases to be monitored, and
create another Task that accepted the results of each of the other tasks in
order to prepare the consolidated report and mail it out.
But, I don't really know enough about what you're trying to even know if
that's really appropriate, or if something else would be better.
The other thing I imagine I'd do is tie the components together with
ZConfig, creating a top-level schema with sections to describe the
individual databases, etc. Then, my app would literally be assembled by a
config document, like this:
URL pgsql:blah:blah at blah
URL sybase:blah:blah at blah/blah
URL dcoracle2:blah:blah at blah
All of the DB things would get put into a list attribute on the
application-level object, and the repository would be put in the repository
attribute, etc. The class that DB sections mapped to would have a binding
to automatically start their data collection task, and of course you could
have lots of other configuration settings on them.
>But as I say, my question is less about the specifics of my
>application, and more about how to *think* about this sort of design.
If you're thinking about the kind of problems I think you're thinking
about, this is probably the way to go. That is, design in terms of
pluggable components, and let ZConfig do the selection, configuration, and
assembly of those components for you.
More information about the PEAK