[PEAK] Making PyProtocols work for aeve

[Bob Ippolito]

On Nov 25, 2003, at 11:54 AM, Phillip J. Eby wrote:

> At 10:45 AM 11/25/03 -0500, Bob Ippolito wrote:
>> I have a project that I'm thinking about using PyProtocols for.
>> Currently, it's using something conceptually similar that I wrote 
>> myself.  The software is aeve, an Apple Events bridge for MacPython.
>> The way you communicate with the Apple Event subsystem is via this 
>> mostly opaque type called an AEDesc (which is kinda like a 
>> PyObject*).
>> An AEDesc has a 4 character code that represents what it is, and then 
>> some (usually) opaque bytes.
>>
>> Some AEDescs are simply decoded into Python types, for example lists, 
>> floats, records (like a dict, except the keys are always 4 character 
>> codes), etc.  However, some of them represent classes.  A class code 
>> is specific to a particular application, not global to the Apple 
>> Event subsystem (as in floats, ints, etc.).  So when these are 
>> adapted to Python types (there is a Python "shadow" of the class that 
>> mimics all of the properties, methods, etc) they must be adapted in 
>> the context of the application (or the global context, if not found 
>> in the application).  One problem is that you can get these class 
>> types inside container AEDescs, so the adaptation needs to carry 
>> around a context with it.
>>
>> I've been having the following conceptual issues making this fit into 
>> PyProtocols
>
> Just a foreword here...  it's kind of difficult for me to make clear 
> recommendations, because I don't know enough about the use case of 
> this.  That is, what kind of API you want to expose to the user of 
> these events.

There are two APIs.. one API lets you do this:

(convert to/from python as soon as possible)
listOfNames = [x.name for x in app.someElements]

the other API lets you do this:

(convert to/from python as late as possible)
listOfNames = app.someElements.all.name.get()

The reason for this is that the first API will make more than 
2*len(app.someElements) dispatches and the second case will happen in a 
single dispatch (of a more complicated AEDesc).  There are huge 
performance and convenience differences between both of these APIs.

The second reason is that some people like to do things like 
app.window.first.document.paragraphs[2].words.first.get() to get the 
first word of the third paragraph, where via the first API you would've 
ended up with a python string at app.window.first.document and you 
would have to use Python APIs to split paragraphs and words.

> For example, do you want to let people declare adaptations from 
> AEDesc.types?  Or is that going to be hidden behind some other 
> interface you're creating?

Hidden

> Anyway, there are many ways to address your questions, and the "right" 
> answers will depend a lot on what you're trying to do.
>
>
>> 1)  I really need to adapt by object introspection (The AEDesc.type 
>> four character code), not by type.  Everything is an AEDesc on the 
>> incoming side.  Right now I'm playing around with the idea that I 
>> make interfaces for every since four character code that I know 
>> about, and an AEDesc wrapper that figures out which interface to use 
>> (based upon the application context) and does adviseObject(self, 
>> provides=(ISomeSpecificFourCharCode,)) in its __init__.  This is 
>> kinda ugly.
>
> The first thing that comes to mind here is using a URI namespace for 
> these, but I'm guessing you already thought of that.

Right now I use python module namespaces..  aeve.Applications is the 
global namespace, aeve.Applications.ApplicationName is the application 
namespace.  There used to be another level to aeve namespaces, but I've 
since realized that Apple Script uses the third namespace (suites) 
purely to avoid conflicts in four character codes, and for 
documentation purposes.

>> 2)  I need adaptation contexts (per bridged application).  I don't 
>> really see how the "Extending the Framework for Context" example can 
>> apply to my use case.
>
> I kind of need to understand what "per bridged application" means.  I 
> suspect that contextual protocols *will* suffice for what you want, 
> but it'll be a bit more complex due to having to deal with URIs at the 
> same time.
>
> So far, I see an architecture that looks like 
> 'ctx.protocolForDesc("ABCD")', where 'ctx' is one of your application 
> contexts.  'ctx' simply keeps a cache of type codes to protocol 
> instances.  This could be as simple as:
>
>
>     protocols = binding.Make(dict)
>
>     def protocolForDesc(self,code):
>         try:
>             return self.protocols[code]
>         except KeyError:
>             proto = protocols.Protocol()
>             self.protocols[code] = proto
>             return proto
>
> Then, as you described, created event objects would then:
>
>     adviseObject(self,provides=[ctx.protocolForDesc(self.type)])
>
> If you want to also support a non-contextual version using URI's, you 
> could do this:
>
>     def protocolForDesc(self,code):
>         try:
>             return self.protocols[code]
>         except KeyError:
>             proto = protocols.Variation(
>                 protocols.protocolForURI(self.URI_BASE+code),
>                 self
>             )
>             self.protocols[code] = proto
>             return proto
>
> Then, adapters registered from the URI protocol will apply to contexts 
> that don't have a context-specific adapter registered.

This definitely helps.  I know all the types when I create the 
application bindings, so I won't be making stuff up at protocolForDesc 
time.  If it's not in the application's context dictionary, it will get 
looked up in the global dictionary.

>> 3)  I need to be able to weight certain adaptations.  Some 
>> conversions are lossy (i.e. double -> float), and shouldn't be used 
>> unless that's what's asked for.
>
> You can do weighting using the depth indicators, but I'm a little 
> suspicious of your question here because it seems to be implying that 
> you're doing something different than what I'm thinking.  Clarifying 
> what you mean would be very helpful here.

Python has one float type, which is 64bits (on this platform).  Apple 
Events has a whole bunch of floats (32bit, 64bit, 128bit off the top of 
my head).  Here's the paths I have:

float -> IAEDesc_32bitfloat
float -> IAEDesc_64bitfloat

Technically I could do this:

float -> IAEDesc_64bitfloat
float -> IAEDesc_64bitfoat -> IAEDesc_32bitfloat

but that's a nastier conversion because it struct.pack's into an 
AEDesc, and then it converts that AEDesc to another AEDesc.

I'd prefer to just take the direct IAEDesc_32bitfloat route (a 
different struct.pack), but only if I say adapt(float, 
IAEDesc_32bitfloat), not adapt(float, IAEDesc).  Basically, float -> 
IAEDesc_32bitfloat should have a cost > 1 (since it's lossy) and 
IAEDesc_64bitfloat should have a cost == 1.  The cost of adapting an 
AEDesc_FOO to AEDesc_BAR should be pretty high, because I don't know if 
it works until I try it.  But maybe I have to use a fallback factory to 
make that happen anyways(?)

>> 4)  If you can't find an explicit adaptation path, you can ask the 
>> Apple Event subsystem to (attempt to) coerce it for you.  This has 
>> the problem of making way too many adapter declarations.
>
> Coerce it to *what*?  I'm a little lost here.  Why wouldn't you just 
> let the user of the bridge define what they want to do for their app's 
> purposes?

When you're taking something from "IPython" and converting to "IAEDesc" 
for purposes of putting it into a list or record, any sort of AEDesc 
will work, as long as it's an AEDesc.  However, when you're doing 
something like {set the name of the first track to 'hello'} (yes, 
AppleScript is backwards.. python equivalent is app.track[0].name = 
'hello') then the application probably expects a certain kind of AEDesc 
for the name (text, maybe) so you will want to give it an 
"IAEDesc_text" .. the adapter path for that is probably python str -> 
python unicode -> AEDesc 'utxt' ... AEDesc 'text' where -> are explicit 
adapters (python string to AEDesc utxt should be the least cost path to 
an IAEDesc) then the ... means it will just ask the Apple Events 
runtime to convert 'utxt' to 'text' because it doesn't know what else 
to do.  I don't want to have adapters from IAEDesc_utxt -> 
IAEDesc_text, IAEDesc_text -> IAEDesc_utxt, etc.. because there are too 
many of them and who knows which ones are valid and which aren't.

>> 5)  I have a "MetaInterface" .. "convert this AEDesc to something 
>> Pythonic" or "convert this Pythonic thing to an AEDesc".  I haven't 
>> quite figured out how best to set that up with PyProtocols.  I'm 
>> almost thinking that the four character code interfaces should also 
>> be the adapters to/from the base AEDesc, would that work well with 
>> PyProtocols?
>
> Now my head is exploding.  Perhaps I'd need to learn something about 
> AEDesc to be able to understand this.

It's a really old system (circa 1991).. but it has worked really well 
for years and is prolific even in the latest version of OS X.  That 
said, it's really hard to find documentation for it that makes any 
sense, and most of the documentation is in Pascal.. it's scary stuff.  
I hardly understand most of it myself ;)

> Then again, maybe all you're saying is that you have 'IPythonic' and 
> 'IAEDesc'.  If so, that's a lot simpler.  :)

Yeah, pretty much.  IPythonic would be anything that someone on the 
Python side can use, IAEDesc is anything you can pass over the bridge.

>> 6)  A lot of the adaptation is an actual conversion not a wrapping, 
>> so I would prefer to use a non-instantiated class (via classmethods) 
>> or function as the adapter from InterfaceA to InterfaceB.  I couldn't 
>> figure that out, it's pretty easy with Twisted's interfaces though.
>
> There are plenty of examples of that scattered through PEAK and 
> PyProtocols.  Look for 'factoryMethod=' to find classes that use class 
> methods for conversion, and 'declareAdapter()' calls to find 
> conversion functions.
>
> Generally speaking, to declare an adapter function, you use:
>
> declareAdapter(func, provides=[IProvideThis], forTypes=[...], 
> forProtocols=[...])
>
> 'forTypes' lists classes the function adapts from, and 'forProtocols' 
> lists protocols it adapts from.
>
>
>> 7)  AEDesc has an equivalent to None ('null').  This seems like it 
>> would be awfully hard to return from PyProtocols ;)  I don't really 
>> want to have all my interfaces use a getter function just so I can 
>> adapt this type to None!
>
> Why wouldn't you have your wrapper system just convert 'null' to None 
> in the first place?  (I.e., why go to the trouble of adapting it?)  
> Indeed, I'd assume that for types where there is no ambiguity or data 
> loss, you might as well convert them to Python types in the first 
> place.  (Why bother allocating a structure you're just going to 
> convert anyway?)

That's what I do now.. I guess the best way would be for me to use some 
of my existing adaptation infrastructure for these unambiguous cases 
and call out to adapt when it's less obvious what needs to be done.  
That just means I will have to have my own version of the "adapt" 
function, since it needs to be used when picking apart AEDesc 
containers or a raw AEDesc.

>> It sounds almost like I need to write some really custom __conform__ 
>> and __adapt__ special methods, so if I have to go that route, what 
>> would using PyProtocols really get me?
>
> Compatibility with PEP 246 and systems that use it.  Mainly, that 
> means PEAK and Zope 3.
>
>> Also, are there any relatively complex/large open source projects 
>> that use PyProtocols other than PEAK/PyProtocols?  I want to see how 
>> other people are using it, the examples in the PyProtocols 
>> documentation seem to be too trivial to be of much use to me.
>
> Given what you're trying to do, I can see why.  You are the first 
> person who's asked about interfacing to another (non-Python) interface 
> system.  I've interfaced PyProtocols so far only to Python interface 
> systems (Zope and Twisted's).  I'd like to help in any way I can, but 
> so far I'm still handicapped by not fully understanding your situation 
> or goals.

I'll let you know when I fully understand it (don't expect to hear that 
any time soon) ;)  I think I do have a somewhat clearer idea about how 
I should be using PyProtocols now.

-bob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2357 bytes
Desc: not available
Url : http://www.eby-sarna.com/pipermail/peak/attachments/20031125/495811c6/smime.bin