Property and recursion

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Property and recursion

Brennan Williams
I think I may be doing something wrong when using Property and def
_get_... as I'm getting into an endless loop due to recursion.

I have something like...

xmin=Property()
xmincount=Int(0)

and later....

def _get_xmin(self):
     self.xmincount+=1
     print '_get_xmin:count=',self.xmincount
     return self.xvector.xmin

I also have another property....

timemin=Property()

and...

def _get_timemin(self):
     return str(self.xmin)+str(self.xunits)

I was wondering if incrementing the xmincount trait inside the _get_xmin
method is causing the problem?

Any pointers on how I should tidy up my use of Property?

Brennan



_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Robert Kern
On Tue, Mar 20, 2012 at 6:01 AM, Brennan Williams
<[hidden email]> wrote:

> I think I may be doing something wrong when using Property and def
> _get_... as I'm getting into an endless loop due to recursion.
>
> I have something like...
>
> xmin=Property()
> xmincount=Int(0)
>
> and later....
>
> def _get_xmin(self):
>     self.xmincount+=1
>     print '_get_xmin:count=',self.xmincount
>     return self.xvector.xmin
>
> I also have another property....
>
> timemin=Property()
>
> and...
>
> def _get_timemin(self):
>     return str(self.xmin)+str(self.xunits)
>
> I was wondering if incrementing the xmincount trait inside the _get_xmin
> method is causing the problem?

Nothing leaps out from the code that you have posted. There's probably
something else at work. Try to post a self-contained example that
demonstrates the problem. Try to make it as minimal as possible.
Usually, this process will show you where the problem is. It can help
to use traceback.print_stack() in likely places to figure out what is
calling what.

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Adam Hughes
In your actual code, do you use the "depends_on" argument with the Property
definition.  It's really easy to spot the source of the infinite recursion
just by looking at this.  For example:

a=Property(depends_on='b')
b=Property(depends_on='a')

You can see in this example that a and b will continue to trigger one
another infinitely.  Are you sure this is not happening in your code?


On Tue, Mar 20, 2012 at 9:44 AM, Robert Kern <[hidden email]> wrote:

> On Tue, Mar 20, 2012 at 6:01 AM, Brennan Williams
> <[hidden email]> wrote:
> > I think I may be doing something wrong when using Property and def
> > _get_... as I'm getting into an endless loop due to recursion.
> >
> > I have something like...
> >
> > xmin=Property()
> > xmincount=Int(0)
> >
> > and later....
> >
> > def _get_xmin(self):
> >     self.xmincount+=1
> >     print '_get_xmin:count=',self.xmincount
> >     return self.xvector.xmin
> >
> > I also have another property....
> >
> > timemin=Property()
> >
> > and...
> >
> > def _get_timemin(self):
> >     return str(self.xmin)+str(self.xunits)
> >
> > I was wondering if incrementing the xmincount trait inside the _get_xmin
> > method is causing the problem?
>
> Nothing leaps out from the code that you have posted. There's probably
> something else at work. Try to post a self-contained example that
> demonstrates the problem. Try to make it as minimal as possible.
> Usually, this process will show you where the problem is. It can help
> to use traceback.print_stack() in likely places to figure out what is
> calling what.
>
> --
> Robert Kern
> Enthought
> _______________________________________________
> Enthought-Dev mailing list
> [hidden email]
> https://mail.enthought.com/mailman/listinfo/enthought-dev
>
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Brennan Williams
On 21/03/2012 6:06 a.m., Adam Hughes wrote:

> In your actual code, do you use the "depends_on" argument with the Property
> definition.  It's really easy to spot the source of the infinite recursion
> just by looking at this.  For example:
>
> a=Property(depends_on='b')
> b=Property(depends_on='a')
>
> You can see in this example that a and b will continue to trigger one
> another infinitely.  Are you sure this is not happening in your code?
>
I've looked for that sort of thing but I haven't found any cases where
I've done that yet.
I have a fairly complicated view with a lot of tabbed groups, some for
plots, some for plot options etc.

Inclued in the view are some property traits which tend to be floats or
strings, e.g. a minimum time in days or a minimum date. What seems to be
happening is that after the view is created (I have a def create_my_view
method) but before it is rendered to the screen (using edit_traits) the
property traits get re-evaluated (they are not cached) a large number of
times (20-40).

In the issue I described below where it got into some sort of recursive
loop, the xmin property was being reevaluated 1000's of times. The
xmincount trait was not in the view. I changed it from "Int" to "int"
and this seems to have fixed that problem but I'm looking into it further.

I haven't been using depends_on a lot because my property traits don't
seem to get updated correctly.
As Robert stated in his reply to my post I need to try and put a small
working example together that shows the behaviour.

Brennan

> On Tue, Mar 20, 2012 at 9:44 AM, Robert Kern<[hidden email]>  wrote:
>
>> On Tue, Mar 20, 2012 at 6:01 AM, Brennan Williams
>> <[hidden email]>  wrote:
>>> I think I may be doing something wrong when using Property and def
>>> _get_... as I'm getting into an endless loop due to recursion.
>>>
>>> I have something like...
>>>
>>> xmin=Property()
>>> xmincount=Int(0)
>>>
>>> and later....
>>>
>>> def _get_xmin(self):
>>>      self.xmincount+=1
>>>      print '_get_xmin:count=',self.xmincount
>>>      return self.xvector.xmin
>>>
>>> I also have another property....
>>>
>>> timemin=Property()
>>>
>>> and...
>>>
>>> def _get_timemin(self):
>>>      return str(self.xmin)+str(self.xunits)
>>>
>>> I was wondering if incrementing the xmincount trait inside the _get_xmin
>>> method is causing the problem?
>> Nothing leaps out from the code that you have posted. There's probably
>> something else at work. Try to post a self-contained example that
>> demonstrates the problem. Try to make it as minimal as possible.
>> Usually, this process will show you where the problem is. It can help
>> to use traceback.print_stack() in likely places to figure out what is
>> calling what.
>>
>> --
>> Robert Kern
>> Enthought
>> _______________________________________________
>> Enthought-Dev mailing list
>> [hidden email]
>> https://mail.enthought.com/mailman/listinfo/enthought-dev
>>
> _______________________________________________
> Enthought-Dev mailing list
> [hidden email]
> https://mail.enthought.com/mailman/listinfo/enthought-dev
>

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Brennan Williams
In reply to this post by Adam Hughes
On 21/03/2012 6:06 a.m., Adam Hughes wrote:

> In your actual code, do you use the "depends_on" argument with the Property
> definition.  It's really easy to spot the source of the infinite recursion
> just by looking at this.  For example:
>
> a=Property(depends_on='b')
> b=Property(depends_on='a')
>
> You can see in this example that a and b will continue to trigger one
> another infinitely.  Are you sure this is not happening in your code?
>
>
> On Tue, Mar 20, 2012 at 9:44 AM, Robert Kern<[hidden email]>  wrote:
>
>> On Tue, Mar 20, 2012 at 6:01 AM, Brennan Williams
>> <[hidden email]>  wrote:
>>> I think I may be doing something wrong when using Property and def
>>> _get_... as I'm getting into an endless loop due to recursion.
>>>
>>> I have something like...
>>>
>>> xmin=Property()
>>> xmincount=Int(0)
>>>
>>> and later....
>>>
>>> def _get_xmin(self):
>>>      self.xmincount+=1
>>>      print '_get_xmin:count=',self.xmincount
>>>      return self.xvector.xmin
>>>
>>> I also have another property....
>>>
>>> timemin=Property()
>>>
>>> and...
>>>
>>> def _get_timemin(self):
>>>      return str(self.xmin)+str(self.xunits)
>>>
>>> I was wondering if incrementing the xmincount trait inside the _get_xmin
>>> method is causing the problem?
>> Nothing leaps out from the code that you have posted. There's probably
>> something else at work. Try to post a self-contained example that
>> demonstrates the problem. Try to make it as minimal as possible.
>> Usually, this process will show you where the problem is. It can help
>> to use traceback.print_stack() in likely places to figure out what is
>> calling what.
This isn't about the recursion problem but the propblem I have with
property traits being updated. Below is a bit of code that has two
Property traits minvalue and maxvalue which should get updated when
nullvalue changes but they don't. I must be doing something wrong but I
can't work it out. I also put a mincount counter in and that shows that
def _get_minvalue is called 3 times when the view is being rendered.

Brennan


from traits.api import *
from traitsui.api import *
from traitsui.ui_editors.array_view_editor import ArrayViewEditor
import numpy as np

class MyData ( HasTraits ):

     mydata=Array(np.float)
     nullmask=Any
     maskeddata=Any
     nullvalue=Float
     minvalue=Property(depends_on=['nullvalue'])
#    minvaluecount=Int(0)
     maxvalue=Property(depends_on='nullvalue')
     view=View(
     VGroup(
             
Item('mydata',show_label=False,editor=ArrayViewEditor(show_index=False,titles=['Data'])),
             
Item('maskeddata',show_label=False,editor=ArrayViewEditor(show_index=False,titles=['Masked
Data'])),
     ),
         Item('nullvalue'),
     Item('minvalue',style='readonly'),
#    Item('minvalue'),
     Item('maxvalue',style='readonly'),
#    Item('maxvalue'),
     height=500,width=400,
     )

     def __init__(self):
     self.mydata=([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 10.0])
     self.nullvalue=-9.0e+20
     self.minvaluecount=0
     def _get_minvalue( self ):
     self.minvaluecount+=1
     print '_get_minvalue',self.minvaluecount
         return self.maskeddata.min()
     def _get_maxvalue( self ):
         return self.maskeddata.max()
     def _nullvalue_changed(self,old,new):
#    self.nullvalue=new
         print 'nullvalue=',self.nullvalue
     self.nullmask=self.mydata!=self.nullvalue
     print 'nullmask=',self.nullmask
     self.maskeddata=self.mydata[self.nullmask]
     print 'maskeddata=',self.maskeddata
#    self._minvalue=None
#    self._maxvalue=None


somedata=MyData()
somedata.configure_traits()


_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Robert Kern
On Tue, Mar 20, 2012 at 10:49 PM, Brennan Williams
<[hidden email]> wrote:

> This isn't about the recursion problem but the propblem I have with
> property traits being updated. Below is a bit of code that has two
> Property traits minvalue and maxvalue which should get updated when
> nullvalue changes but they don't. I must be doing something wrong but I
> can't work it out.

You need to call super(MyData, self).__init__() if you override the
HasTraits.__init__(). If you don't, much of the Traits notification
machinery won't be hooked up.

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Brennan Williams
On 21/03/2012 11:52 a.m., Robert Kern wrote:

> On Tue, Mar 20, 2012 at 10:49 PM, Brennan Williams
> <[hidden email]>  wrote:
>
>> This isn't about the recursion problem but the propblem I have with
>> property traits being updated. Below is a bit of code that has two
>> Property traits minvalue and maxvalue which should get updated when
>> nullvalue changes but they don't. I must be doing something wrong but I
>> can't work it out.
> You need to call super(MyData, self).__init__() if you override the
> HasTraits.__init__(). If you don't, much of the Traits notification
> machinery won't be hooked up.
>
OK thanks. Will see if that sorts out my other property problems - I
expect it will.
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Brennan Williams
In reply to this post by Robert Kern
On 21/03/2012 11:52 a.m., Robert Kern wrote:

> On Tue, Mar 20, 2012 at 10:49 PM, Brennan Williams
> <[hidden email]>  wrote:
>
>> This isn't about the recursion problem but the propblem I have with
>> property traits being updated. Below is a bit of code that has two
>> Property traits minvalue and maxvalue which should get updated when
>> nullvalue changes but they don't. I must be doing something wrong but I
>> can't work it out.
> You need to call super(MyData, self).__init__() if you override the
> HasTraits.__init__(). If you don't, much of the Traits notification
> machinery won't be hooked up.
>
Hmmm, adding super(RZVector,self).__init__() immediately after my def
__init__(...) for my RZVector class slows down my application by about
two orders of magnitude as I'm creating hundreds, if nt thousands of
instances of this class.

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Robert Kern
On Tue, Mar 20, 2012 at 11:47 PM, Brennan Williams
<[hidden email]> wrote:

> Hmmm, adding super(RZVector,self).__init__() immediately after my def
> __init__(...) for my RZVector class slows down my application by about
> two orders of magnitude as I'm creating hundreds, if nt thousands of
> instances of this class.

Trait notifications do have significant startup overhead. Traits may
not be appropriate for RZVector.

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Brennan Williams
On 21/03/2012 9:36 p.m., Robert Kern wrote:

> On Tue, Mar 20, 2012 at 11:47 PM, Brennan Williams
> <[hidden email]>  wrote:
>
>> Hmmm, adding super(RZVector,self).__init__() immediately after my def
>> __init__(...) for my RZVector class slows down my application by about
>> two orders of magnitude as I'm creating hundreds, if nt thousands of
>> instances of this class.
> Trait notifications do have significant startup overhead. Traits may
> not be appropriate for RZVector.
>
I'm using TreeEditor so I have a hierarchical tree with a few object
instances in the upper branches, tens to a few hundred in the middle
branches and then at the lower level (where my RZVector is) anywhere
from a few hundred to a hundred thousand.

Previously without calling super(RZVector,self).__init__() everything
loads reasonable quickly (up to a minute or so for a medium sized
dataset and everything works ok apart from the previously described
property problems. The data arrays and plots for each instance of
RZVector are not read from file and created until required (i.e. click
on that instance of RZVector in the tree to then create its view and the
create_vector_view method reads in the data and builds the plots, tables
etc.)

So I either don't call super and work around the property problem or
else I split RZVector into a "lite" RZVectorHeader and "heavy"
RZVectorBody and have RZVectorHeader as a node in the tree and hope that
the overhead of the super comes down to a reasonable level.

How does that sound? Is the overhead of super linear with the number of
trait definitions in the class?

Brennan

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Robert Kern
On Wed, Mar 21, 2012 at 8:55 AM, Brennan Williams
<[hidden email]> wrote:

> How does that sound? Is the overhead of super linear with the number of
> trait definitions in the class?

The cost is mostly in the dynamic trait notifications that are set up
during initialization. Property(depends_on=...) and DelegatesTo() are
the main culprits, especially if they are using the
extended.trait[].change:syntax.

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Brennan Williams
On 21/03/2012 10:21 p.m., Robert Kern wrote:

> On Wed, Mar 21, 2012 at 8:55 AM, Brennan Williams
> <[hidden email]>  wrote:
>
>> How does that sound? Is the overhead of super linear with the number of
>> trait definitions in the class?
> The cost is mostly in the dynamic trait notifications that are set up
> during initialization. Property(depends_on=...) and DelegatesTo() are
> the main culprits, especially if they are using the
> extended.trait[].change:syntax.
>
Well I'll look at removing my use of Property and see what effect that
has. I'm still not sure why I end up with some of the _get_... methods
being called tens if not hundreds of times when a view is being rendered
(and this is after I added the call to super(..,self).__init__()

The other approach I thought of is not to build the list of RZVector
instances in the parent object in the tree until that parent object is
selected in the tree. I think I'm still favouring the approach of
replacing RZVector with an RZVectorHeader object that has an
Instance(RZVectorBody) trait that only gets created when necessary (i.e.
when being viewed/plotted).

So tomorrow I'll run a few performance tests.

Thanks for your comments.

Brennan

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Adam Hughes
You know the code better than anynone Brennan, but is there anyway to
restructure it so that the elements in the tree don't all need to be
converted to a HasTraits object?  I had a code a while ago that was
examining materials and making them into objects, but when I tried to load
in hundreds of materials, it slowed down, so the only alternative was to
store the data in them into a native python data structure then instantiate
classes when the user called them.

Of course, such a solution might not be applicable to you.  You can store
data into tree structures in python without the use of traits.

On Wed, Mar 21, 2012 at 5:34 AM, Brennan Williams <
[hidden email]> wrote:

> On 21/03/2012 10:21 p.m., Robert Kern wrote:
> > On Wed, Mar 21, 2012 at 8:55 AM, Brennan Williams
> > <[hidden email]>  wrote:
> >
> >> How does that sound? Is the overhead of super linear with the number of
> >> trait definitions in the class?
> > The cost is mostly in the dynamic trait notifications that are set up
> > during initialization. Property(depends_on=...) and DelegatesTo() are
> > the main culprits, especially if they are using the
> > extended.trait[].change:syntax.
> >
> Well I'll look at removing my use of Property and see what effect that
> has. I'm still not sure why I end up with some of the _get_... methods
> being called tens if not hundreds of times when a view is being rendered
> (and this is after I added the call to super(..,self).__init__()
>
> The other approach I thought of is not to build the list of RZVector
> instances in the parent object in the tree until that parent object is
> selected in the tree. I think I'm still favouring the approach of
> replacing RZVector with an RZVectorHeader object that has an
> Instance(RZVectorBody) trait that only gets created when necessary (i.e.
> when being viewed/plotted).
>
> So tomorrow I'll run a few performance tests.
>
> Thanks for your comments.
>
> Brennan
>
> _______________________________________________
> Enthought-Dev mailing list
> [hidden email]
> https://mail.enthought.com/mailman/listinfo/enthought-dev
>
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Property and recursion

Brennan Williams
On 22/03/2012 6:03 a.m., Adam Hughes wrote:
> You know the code better than anynone Brennan, but is there anyway to
> restructure it so that the elements in the tree don't all need to be
> converted to a HasTraits object?  I had a code a while ago that was
> examining materials and making them into objects, but when I tried to load
> in hundreds of materials, it slowed down, so the only alternative was to
> store the data in them into a native python data structure then instantiate
> classes when the user called them.
Hmmm, well I'm using TreeEditor and TreeNode and the rhs panel displays
the view of the selected node in the tree so I believe that when I have
TreeNode(node_for=[RZVector]....) that RZVector needs to be a HasTraits
object so that a Traits view can be created for it.

So I probably need to have a new RZVector that is not a HasTraits object
and then when the user selects the parent node in the tree (i.e. the
object containing a list of RZVector's) I then instantiate a local list
of RZTraitsVector's and the tree gets locally updated.

Is that what you mean?

Brennan

> Of course, such a solution might not be applicable to you.  You can store
> data into tree structures in python without the use of traits.
>
> On Wed, Mar 21, 2012 at 5:34 AM, Brennan Williams<
> [hidden email]>  wrote:
>
>> On 21/03/2012 10:21 p.m., Robert Kern wrote:
>>> On Wed, Mar 21, 2012 at 8:55 AM, Brennan Williams
>>> <[hidden email]>   wrote:
>>>
>>>> How does that sound? Is the overhead of super linear with the number of
>>>> trait definitions in the class?
>>> The cost is mostly in the dynamic trait notifications that are set up
>>> during initialization. Property(depends_on=...) and DelegatesTo() are
>>> the main culprits, especially if they are using the
>>> extended.trait[].change:syntax.
>>>
>> Well I'll look at removing my use of Property and see what effect that
>> has. I'm still not sure why I end up with some of the _get_... methods
>> being called tens if not hundreds of times when a view is being rendered
>> (and this is after I added the call to super(..,self).__init__()
>>
>> The other approach I thought of is not to build the list of RZVector
>> instances in the parent object in the tree until that parent object is
>> selected in the tree. I think I'm still favouring the approach of
>> replacing RZVector with an RZVectorHeader object that has an
>> Instance(RZVectorBody) trait that only gets created when necessary (i.e.
>> when being viewed/plotted).
>>
>> So tomorrow I'll run a few performance tests.
>>
>> Thanks for your comments.
>>
>> Brennan
>>
>> _______________________________________________
>> Enthought-Dev mailing list
>> [hidden email]
>> https://mail.enthought.com/mailman/listinfo/enthought-dev
>>
> _______________________________________________
> Enthought-Dev mailing list
> [hidden email]
> https://mail.enthought.com/mailman/listinfo/enthought-dev
>

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev