[Traits] Proposal: lazy property

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[Traits] Proposal: lazy property

Anton Tyutin
Hi everyone,

I propose to add "lazy property" to the existing trait types. The
purpose is to avoid unnecessary computations of property values and
redundant trait notifications.

Consider an object implemented as a property trait. When such an
object is notified that one of the objects it depends on has changed
its value, it recomputes its own value, and propagates the
notification to its dependents.  Now consider a modification to this
behavior, which would make the object "lazy property". When the object
receives the notification, it does not update its value immediately,
but pospones the updating until the value is actually requested (by a
getattr method or alike). This "lazy evaluation" can be complemented
with "lazy notification", which means that if the object has sent
notifications to its dependents, it does not send further
notifications until its value is recomputed.

One could remind that each trait is supposed to send its both old and
new values with its notifications.What will be sent by a "lazy
property" as a new value? My response is that it seems to have little
sense to send the new value in the notification, at least for
properties (I would be greatful if anyone pointed me to a case where
it is really necessary). Nevertheless, should the "lazy properties"
conform the general notification convention, they could always send an
"Undefined" object as their new values.

As an illustration, I attach a simple implementation of "lazy
properties", which I have written for my personal needs. Hopefully,
this code clarifies my idea. Yet this implementation supports only the
"lazy evaluation" feature; "lazy notification" appears to require
changes to the existing Traits framework.

What do you think about this?

I would be happy to receive your professional feedback.

Thanks
Anton

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev

lazy_property.py (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Traits] Proposal: lazy property

bryce hendrix-2
On Fri, Feb 4, 2011 at 6:57 AM, Anton Tyutin <[hidden email]> wrote:
Hi everyone,

I propose to add "lazy property" to the existing trait types. The
purpose is to avoid unnecessary computations of property values and
redundant trait notifications.

Consider an object implemented as a property trait. When such an
object is notified that one of the objects it depends on has changed
its value, it recomputes its own value, and propagates the
notification to its dependents.  Now consider a modification to this
behavior, which would make the object "lazy property". When the object
receives the notification, it does not update its value immediately,
but pospones the updating until the value is actually requested (by a
getattr method or alike). This "lazy evaluation" can be complemented
with "lazy notification", which means that if the object has sent
notifications to its dependents, it does not send further
notifications until its value is recomputed.

One could remind that each trait is supposed to send its both old and
new values with its notifications.What will be sent by a "lazy
property" as a new value? My response is that it seems to have little
sense to send the new value in the notification, at least for
properties (I would be greatful if anyone pointed me to a case where
it is really necessary). Nevertheless, should the "lazy properties"
conform the general notification convention, they could always send an
"Undefined" object as their new values.

As an illustration, I attach a simple implementation of "lazy
properties", which I have written for my personal needs. Hopefully,
this code clarifies my idea. Yet this implementation supports only the
"lazy evaluation" feature; "lazy notification" appears to require
changes to the existing Traits framework.

What do you think about this?

I would be happy to receive your professional feedback.

Thanks
Anton


Anton,

Have you looked at the cached_property decorator?

class Example(HasTraits):
    a = Int()
    b = Property(Int, depends_on='a')

    @cached_property
    def _get_b(self):
        return a * 2

Bryce 

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Traits] Proposal: lazy property

Anton Tyutin
Hi Bryce,

Yes, I know the cached_value decorator; in fact I actively use it
(e.g., in the attached lazy_property.py code). But this decorator
serves a different purpose. Let me clarify the difference.

A cached_value property does not recalculate its value when repeatedly
accessed, provided the depends_on objects do not change their values.
A "lazy property" whould not recalculate its value even if they do,
until it is explicitly accessed.

As soon as any of the depends_on objects updates its value, the value
of a "usual" property (including cached_value properties) is
automatically recomputed, whether it is really needed (accessed) or
not.

Thanks for your comment,
Anton


On Fri, Feb 4, 2011 at 4:22 PM, bryce hendrix <[hidden email]> wrote:

> On Fri, Feb 4, 2011 at 6:57 AM, Anton Tyutin <[hidden email]> wrote:
>>
>> Hi everyone,
>>
>> I propose to add "lazy property" to the existing trait types. The
>> purpose is to avoid unnecessary computations of property values and
>> redundant trait notifications.
>>
>> Consider an object implemented as a property trait. When such an
>> object is notified that one of the objects it depends on has changed
>> its value, it recomputes its own value, and propagates the
>> notification to its dependents.  Now consider a modification to this
>> behavior, which would make the object "lazy property". When the object
>> receives the notification, it does not update its value immediately,
>> but pospones the updating until the value is actually requested (by a
>> getattr method or alike). This "lazy evaluation" can be complemented
>> with "lazy notification", which means that if the object has sent
>> notifications to its dependents, it does not send further
>> notifications until its value is recomputed.
>>
>> One could remind that each trait is supposed to send its both old and
>> new values with its notifications.What will be sent by a "lazy
>> property" as a new value? My response is that it seems to have little
>> sense to send the new value in the notification, at least for
>> properties (I would be greatful if anyone pointed me to a case where
>> it is really necessary). Nevertheless, should the "lazy properties"
>> conform the general notification convention, they could always send an
>> "Undefined" object as their new values.
>>
>> As an illustration, I attach a simple implementation of "lazy
>> properties", which I have written for my personal needs. Hopefully,
>> this code clarifies my idea. Yet this implementation supports only the
>> "lazy evaluation" feature; "lazy notification" appears to require
>> changes to the existing Traits framework.
>>
>> What do you think about this?
>>
>> I would be happy to receive your professional feedback.
>>
>> Thanks
>> Anton
>>
>
> Anton,
> Have you looked at the cached_property decorator?
> class Example(HasTraits):
>     a = Int()
>     b = Property(Int, depends_on='a')
>     @cached_property
>     def _get_b(self):
>         return a * 2
> Bryce
> _______________________________________________
> Enthought-Dev mailing list
> [hidden email]
> https://mail.enthought.com/mailman/listinfo/enthought-dev
>
>
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Traits] Proposal: lazy property

Peter Wang
On Feb 4, 2011, at 9:54 AM, Anton Tyutin wrote:

> As soon as any of the depends_on objects updates its value, the value
> of a "usual" property (including cached_value properties) is
> automatically recomputed, whether it is really needed (accessed) or
> not.

So you are really just wanting to separate out the invalidation from the recomputation aspects of the Property. I seem to recall that the immediate recomputation stems from the fact that the trait_property_changed() mechanism computes the new value of the property to stick into the trait event that fires for the property..


-Peter
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Traits] Proposal: lazy property

Chris Colbert
given the way that traits is currently architected, it would difficult to change from a push to a pull model. Not to say it can't be done, but what are the benefits? If want something to recompute only when you access it, just use a normal property.

On Fri, Feb 4, 2011 at 12:58 PM, Peter Wang <[hidden email]> wrote:
On Feb 4, 2011, at 9:54 AM, Anton Tyutin wrote:

> As soon as any of the depends_on objects updates its value, the value
> of a "usual" property (including cached_value properties) is
> automatically recomputed, whether it is really needed (accessed) or
> not.

So you are really just wanting to separate out the invalidation from the recomputation aspects of the Property. I seem to recall that the immediate recomputation stems from the fact that the trait_property_changed() mechanism computes the new value of the property to stick into the trait event that fires for the property..


-Peter
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev


_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Traits] Proposal: lazy property

Didrik Pinte-2
In reply to this post by Peter Wang

On 04 Feb 2011, at 18:58, Peter Wang wrote:

> On Feb 4, 2011, at 9:54 AM, Anton Tyutin wrote:
>
>> As soon as any of the depends_on objects updates its value, the value
>> of a "usual" property (including cached_value properties) is
>> automatically recomputed, whether it is really needed (accessed) or
>> not.
>
> So you are really just wanting to separate out the invalidation from the recomputation aspects of the Property. I seem to recall that the immediate recomputation stems from the fact that the trait_property_changed() mechanism computes the new value of the property to stick into the trait event that fires for the property..

Am I missing something , this is the behaviour , no ? In the code below, b is computed when accessed and not when a is modified :

In [2]: from enthought.traits.api import HasTraits, Property, cached_property, Int

In [3]: class Test(HasTraits):
   ...:     a = Int
   ...:     b = Property(depends_on='a')

   ...:     @cached_property
   ...:     def _get_b(self):
   ...:         print 'computing b'
   ...:         return self.a +1
   ...:    
   ...:    

In [4]: t = Test()

In [5]: t.a
Out[5]: 0

In [6]: t.b
computing b
Out[6]: 1

In [7]: t.a = 1

In [8]: t.b
computing b
Out[8]: 2

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Traits] Proposal: lazy property

bryce hendrix-2
On Fri, Feb 4, 2011 at 12:30 PM, Didrik Pinte <[hidden email]> wrote:

On 04 Feb 2011, at 18:58, Peter Wang wrote:

> On Feb 4, 2011, at 9:54 AM, Anton Tyutin wrote:
>
>> As soon as any of the depends_on objects updates its value, the value
>> of a "usual" property (including cached_value properties) is
>> automatically recomputed, whether it is really needed (accessed) or
>> not.
>
> So you are really just wanting to separate out the invalidation from the recomputation aspects of the Property. I seem to recall that the immediate recomputation stems from the fact that the trait_property_changed() mechanism computes the new value of the property to stick into the trait event that fires for the property..

Am I missing something , this is the behaviour , no ? In the code below, b is computed when accessed and not when a is modified :

In [2]: from enthought.traits.api import HasTraits, Property, cached_property, Int

In [3]: class Test(HasTraits):
  ...:     a = Int
  ...:     b = Property(depends_on='a')

  ...:     @cached_property
  ...:     def _get_b(self):
  ...:         print 'computing b'
  ...:         return self.a +1
  ...:
  ...:

In [4]: t = Test()

In [5]: t.a
Out[5]: 0

In [6]: t.b
computing b
Out[6]: 1

In [7]: t.a = 1

In [8]: t.b
computing b
Out[8]: 2


Its been a while since I last looked at the implementation, but I think what happens is there is a on_trait_change handler that marks the cache as dirty, then when the property is read the decorator exercise the property's code and store the result.

Bryce

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: [Traits] Proposal: lazy property

Anton Tyutin
I had to be more precise: when I stated that a property automatically
calculates its value when it receives a notification, I meant the
situation when there is another trait that depends on the property. If
the property is an "end point" in the notification network (as in
Didrik's example), its value is (re)computed only when accessed. But
if a property happens to be an "intermediate node" in the framework,
then it must compute its new value as part of the "trait change" event
that it fires (and this takes place whether its value is cached or
not).

I invite you to have a look at the code (lazy_property.py) that I sent
with the proposal; it is short, and together with a working
implementation of the "lazy property", it contains a usage example
that compares the behavior of "usual" and "lazy" properties. Please
try to run this code, and you will see the difference.

My implementation of "lazy evaluation" is very simple; the idea is to
place the "Undefined" object instead of the actual new value into the
"trait change" event.

I suppose that an implementation of "lazy notification" could also be
easy: the idea is to have an additional binary attribute
"value_needs_update" for a property, which is is set to "True" when
the property receives a notification, and is set to "False" each time
the property's value is computed. If value_needs_update == False then
the property fires its "trait change" event (=notifies other traits
that it needs recomputing), otherwise it does not (because the other
traits which depend on the property have already been informed). It
seems to me that it is not necessary to change the push to the pull
model. Unfortunately, a practical implementation of the "lazy
notification" goes beyond my knowledge of the Traits internals.

Where could "lazy properties" be useful? Clearly, the Traits
properties offer an elegant syntax to implement a function as a
composition of other functions (with an unlimited number of
"intermediate" decomposition layers). This is useful in coding many
mathematical models (in particular, in the context of Chaco
visualization). It can be said that using properties, one can enjoy
the expressiveness of functional programing ;-) But what about the
computational efficiency of such a framework? A function coded as a
"network" of properties reacts to any change of its inputs. This is
fine if one wishes to trace the changes in function values caused by
each change in each individual input. But suppose that we don't need
the function value be recomputed after each change. We may wish to to
change a subset of the inputs, and evaluate the function only after
that. Here the "usual" properties become inefficient, resulting in
unnecessary computations and notifications. Let me again refer to my
code  (lazy_property.py) for an illustration.

Thanks
Anton
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev