[PEAK] Making PyProtocols work for aeve

[Phillip J. Eby]

At 12:58 PM 11/25/03 -0500, Bob Ippolito wrote:

>There are two APIs.. one API lets you do this:
>
>(convert to/from python as soon as possible)
>listOfNames = [x.name for x in app.someElements]
>
>the other API lets you do this:
>
>(convert to/from python as late as possible)
>listOfNames = app.someElements.all.name.get()
>
>The reason for this is that the first API will make more than 
>2*len(app.someElements) dispatches and the second case will happen in a 
>single dispatch (of a more complicated AEDesc).  There are huge 
>performance and convenience differences between both of these APIs.

Okay.  But all the AEDescs have 4-character type codes, right?  Even the 
"more complicated" ones?

It seems to me that there are two levels to conversion here.  One is to 
convert unambiguous atomic types and containers to Python objects.  (I'm 
assuming that some things will clearly be lists, ints, strings, unicode 
etc.)  I'd assume that for such items, it suffices to have a simple, 
single-dictionary mapping from type codes to constructors, with a default 
constructor that makes a generic "AEObject".

Similarly, I'd assume that for Python types that have an unambiguous, 
lossless conversion to AE types, there would be an adapter from the builtin 
type to IAE_Base (or whatever you call it).

I also assume that there's some sort of pickling/unpickling process to/from 
the AE encoding, yes?  Or is this something you're doing as a C module?

If it's an encoding/decoding process, this is a lot cleaner to 
explain.  You don't really adapt to IAE_anything, you adapt to 
'IAE_Encoder', which has methods to write the data structures.

If it's more of an API that manipulates structures or calls over to the 
other applications (like COM on Windows), then it needs to go the other 
way, and actually convert to AE object things.   Hm.



>The second reason is that some people like to do things like 
>app.window.first.document.paragraphs[2].words.first.get() to get the first 
>word of the third paragraph, where via the first API you would've ended up 
>with a python string at app.window.first.document and you would have to 
>use Python APIs to split paragraphs and words.

Makes sense.


>>For example, do you want to let people declare adaptations from 
>>AEDesc.types?  Or is that going to be hidden behind some other interface 
>>you're creating?
>
>Hidden

This is beginning to sound a bit like OLE Automation (aka COM 
IDispatch).  That is, it's a scripting interface to another application, 
not really anything that requires adaptation.  But maybe I'm confused.


>This definitely helps.  I know all the types when I create the application 
>bindings, so I won't be making stuff up at protocolForDesc time.  If it's 
>not in the application's context dictionary, it will get looked up in the 
>global dictionary.



>>>3)  I need to be able to weight certain adaptations.  Some conversions 
>>>are lossy (i.e. double -> float), and shouldn't be used unless that's 
>>>what's asked for.
>>
>>You can do weighting using the depth indicators, but I'm a little 
>>suspicious of your question here because it seems to be implying that 
>>you're doing something different than what I'm thinking.  Clarifying what 
>>you mean would be very helpful here.
>
>Python has one float type, which is 64bits (on this platform).  Apple 
>Events has a whole bunch of floats (32bit, 64bit, 128bit off the top of my 
>head).  Here's the paths I have:
>
>float -> IAEDesc_32bitfloat
>float -> IAEDesc_64bitfloat
>
>Technically I could do this:
>
>float -> IAEDesc_64bitfloat
>float -> IAEDesc_64bitfoat -> IAEDesc_32bitfloat
>
>but that's a nastier conversion because it struct.pack's into an AEDesc, 
>and then it converts that AEDesc to another AEDesc.
>
>I'd prefer to just take the direct IAEDesc_32bitfloat route (a different 
>struct.pack), but only if I say adapt(float, IAEDesc_32bitfloat), not 
>adapt(float, IAEDesc).  Basically, float -> IAEDesc_32bitfloat should have 
>a cost > 1 (since it's lossy) and IAEDesc_64bitfloat should have a cost == 
>1.  The cost of adapting an AEDesc_FOO to AEDesc_BAR should be pretty 
>high, because I don't know if it works until I try it.  But maybe I have 
>to use a fallback factory to make that happen anyways(?)

What's beginning to bother me about all this is that I'm not clear on why 
you need to have these adaptations in the first place.

If this is a scripting interface, it seems that it should be possible to 
handle these things at method boundaries.  If it's a data encoding format, 
it seems you should encode what you have.


>>>4)  If you can't find an explicit adaptation path, you can ask the Apple 
>>>Event subsystem to (attempt to) coerce it for you.  This has the problem 
>>>of making way too many adapter declarations.
>>
>>Coerce it to *what*?  I'm a little lost here.  Why wouldn't you just let 
>>the user of the bridge define what they want to do for their app's purposes?
>
>When you're taking something from "IPython" and converting to "IAEDesc" 
>for purposes of putting it into a list or record, any sort of AEDesc will 
>work, as long as it's an AEDesc.  However, when you're doing something 
>like {set the name of the first track to 'hello'} (yes, AppleScript is 
>backwards.. python equivalent is app.track[0].name = 'hello') then the 
>application probably expects a certain kind of AEDesc for the name (text, 
>maybe) so you will want to give it an "IAEDesc_text" .. the adapter path 
>for that is probably python str -> python unicode -> AEDesc 'utxt' ... 
>AEDesc 'text' where -> are explicit adapters (python string to AEDesc utxt 
>should be the least cost path to an IAEDesc) then the ... means it will 
>just ask the Apple Events runtime to convert 'utxt' to 'text' because it 
>doesn't know what else to do.  I don't want to have adapters from 
>IAEDesc_utxt -> IAEDesc_text, IAEDesc_text -> IAEDesc_utxt, etc.. because 
>there are too many of them and who knows which ones are valid and which aren't.

I think part of what's confusing me is that I don't see what the use of 
adapting between different IAE types is.  Based on what I know so far, if I 
were doing this, I would *only* adapt to or from Python types, not between 
IAE types.  And I'd explicitly define whatever my app needed.

Further, I don't think I'd even bother with adapting anything more coarse 
grained than required for the scripting.  For example, in your example 
above of 'app.track[0].name="hello"', I'd assume that the AEObject stored 
in app.track[0] would have a 'setattr' or property that adapts the set 
value to IAEDesc_text, and that there'd be a str->IAEDesc_text 
adapter.  So, for all practical purposes I'd have no adapters except 
to/from atomic types and generic containers, and everything else would be 
an AEObject or a proxy type generated from metadata.

IOW, there'd be very little adaptation, unless an application wanted to 
define converters from certain URIs to application-defined interfaces that 
the application wanted to convert AE objects to.

So, I guess I'm saying that either you're making things too complicated, or 
that I don't understand yet why the complication is necessary.  :)

OTOH, my approach has the complication of needing Pythonic wrapper types 
for atomic types that can't be perfectly represented by Python built-in 
types.  For example, I'd probably create a wrapper for the 
IAEDesc_128bitfloat that implemented __add__, __mul__, etc. by calling down 
to AE API's.


>>Why wouldn't you have your wrapper system just convert 'null' to None in 
>>the first place?  (I.e., why go to the trouble of adapting it?)
>>Indeed, I'd assume that for types where there is no ambiguity or data 
>>loss, you might as well convert them to Python types in the first 
>>place.  (Why bother allocating a structure you're just going to convert 
>>anyway?)
>
>That's what I do now.. I guess the best way would be for me to use some of 
>my existing adaptation infrastructure for these unambiguous cases and call 
>out to adapt when it's less obvious what needs to be done.

Why adapt at all?  Why not just leave them as proxy objects that implement 
getattr/setattr/getitem/etc.?


>That just means I will have to have my own version of the "adapt" 
>function, since it needs to be used when picking apart AEDesc containers 
>or a raw AEDesc.

This sounds like you're trying to do it "eagerly" instead of "lazily".  But 
you haven't actually given a use case for eagerly adapting non-atomic 
constructs.

If you look at peak.model, there are two fundamentally different kinds of 
types - primitives, and structured types.  Primitives can't be subdivided 
any further: strings, ints, floats, etc.  These are reasonable candidates 
for eager conversion, even if it's to an extension type or proxy type that 
tries to "look like" a Python string or number or whatever.

But structured types are more appropriate for lazy conversion.  Instead of 
converting their contents, they just reference child AE objects.  And, they 
allow setting or getting them as AE objects.  When setting, adapt to the 
least-specific interface allowed by the type.  When getting, simply return 
objects as-is.  If the client wants a different type, *they* should adapt 
it.  However, for primitive types, the objects they receive on getting 
attributes should be quite usable.

I think that this can be so, even for your second API.


>I'll let you know when I fully understand it (don't expect to hear that 
>any time soon) ;)  I think I do have a somewhat clearer idea about how I 
>should be using PyProtocols now.

I'm guessing that this system also resembles CORBA typecodes in a 
way.  (See peak.model.datatypes).  CORBA typecodes have a bunch of 
primitive type possibilities, but they can also be a list or other 
structure of other typecodes, recursively.  So, I guess your second API 
example just returns a type meaning 'list of IAEDesc_text', and so that's 
the "more complicated" type you refer to.  If so, then I guess that 
requires a bit of thinking as to how to adapt/dispatch that cleanly.  Of 
course, that may simply mean you have a parameterized list type that 
already knows what adapter to use.  E.g.:

IAEDesc_list[x] -(AEList(x))-> IPythonic

If my guess about similarity to CORBA typecodes is correct, then I do see 
why things could get more complex than I've been thinking.  You might need 
to define parameterized protocol types, sort of like 
'protocols.sequenceOf()' does.  I've been assuming that all the type data 
is in a single 4-character code, and defines either an object type or a 
primitive type, where containers are generic types rather than 
parameterized ones.