controlling data change events

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

controlling data change events

Nishith Aggarwal

Hi folks,

 

I am using the enthought tools to construct an app that plots a live data stream. This data stream affects multiple plots on the UI. As a first attempt, on each incoming chunk of data, I used the set_data function on all the ArrayDataSource objects (2 plots * 2, 1 each  for x and y data) sequentially to update the plot.  But I can see that the plot doesn’t always refresh and it seems like the redraw events are being queued, most likely because I’m generating too many too fast (a new event one for each of my data sources).  I am thinking along this lines because I have a push-button that can force the plots to be created again and when I use it, I do see the latest data plotted.

 

My question is how can I improve the performance of my app? Perhaps limiting the redraw event to one for each data chunk might help, but it’s not immediately apparent how I would do this. I am not even sure which object in the hierarchy listens for data change. Any help would be appreciated.

 

Thanks,

Nishith

 




Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. Jump Trading, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product.

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: controlling data change events

Robert Kern
On Tue, Oct 2, 2012 at 4:33 PM, Nishith Aggarwal
<[hidden email]> wrote:

> Hi folks,
>
> I am using the enthought tools to construct an app that plots a live data
> stream. This data stream affects multiple plots on the UI. As a first
> attempt, on each incoming chunk of data, I used the set_data function on all
> the ArrayDataSource objects (2 plots * 2, 1 each  for x and y data)
> sequentially to update the plot.  But I can see that the plot doesn’t always
> refresh and it seems like the redraw events are being queued, most likely
> because I’m generating too many too fast (a new event one for each of my
> data sources).  I am thinking along this lines because I have a push-button
> that can force the plots to be created again and when I use it, I do see the
> latest data plotted.
>
> My question is how can I improve the performance of my app? Perhaps limiting
> the redraw event to one for each data chunk might help, but it’s not
> immediately apparent how I would do this. I am not even sure which object in
> the hierarchy listens for data change. Any help would be appreciated.

What happens is that the plot that feeds from the ArrayPlotData will
be notified of the data changing, then the plot will request a redraw.
It does this by asking its associated GUI control to add a new paint
event to the GUI event queue. The GUI toolkits are designed to
collapse adjacent paint events, so multiple requests are not a
problem. Because this is an asynchronous design, the code that
ultimately calls the set_data() method needs to relinquish control
back to the GUI event loop before the paint event will get processed.
I think part of the problem may be how you deal with that data stream.
Can you provide more details?

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: controlling data change events

Nishith Aggarwal
Thanks Robert.

The main thread is running the application and the data stream runs on its own thread. It retrieves the data, processes it and then calls set_data. There is no overlap of control. I have other UI controls on the app and they seem to be responsive even while the latest data has not yet shown up on the plot.

Nishith

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Robert Kern
Sent: Tuesday, October 02, 2012 10:41 AM
To: [hidden email]
Subject: Re: [Enthought-Dev] controlling data change events

On Tue, Oct 2, 2012 at 4:33 PM, Nishith Aggarwal <[hidden email]> wrote:

> Hi folks,
>
> I am using the enthought tools to construct an app that plots a live
> data stream. This data stream affects multiple plots on the UI. As a
> first attempt, on each incoming chunk of data, I used the set_data
> function on all the ArrayDataSource objects (2 plots * 2, 1 each  for
> x and y data) sequentially to update the plot.  But I can see that the
> plot doesn’t always refresh and it seems like the redraw events are
> being queued, most likely because I’m generating too many too fast (a
> new event one for each of my data sources).  I am thinking along this
> lines because I have a push-button that can force the plots to be
> created again and when I use it, I do see the latest data plotted.
>
> My question is how can I improve the performance of my app? Perhaps
> limiting the redraw event to one for each data chunk might help, but
> it’s not immediately apparent how I would do this. I am not even sure
> which object in the hierarchy listens for data change. Any help would be appreciated.

What happens is that the plot that feeds from the ArrayPlotData will be notified of the data changing, then the plot will request a redraw.
It does this by asking its associated GUI control to add a new paint event to the GUI event queue. The GUI toolkits are designed to collapse adjacent paint events, so multiple requests are not a problem. Because this is an asynchronous design, the code that ultimately calls the set_data() method needs to relinquish control back to the GUI event loop before the paint event will get processed.
I think part of the problem may be how you deal with that data stream.
Can you provide more details?

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev

________________________________

Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. Jump Trading, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product.
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: controlling data change events

Robert Kern
On Tue, Oct 2, 2012 at 4:52 PM, Nishith Aggarwal
<[hidden email]> wrote:
> Thanks Robert.
>
> The main thread is running the application and the data stream runs on its own thread. It retrieves the data, processes it and then calls set_data. There is no overlap of control. I have other UI controls on the app and they seem to be responsive even while the latest data has not yet shown up on the plot.

It's possible that the GUI toolkit does not like having the paint
event coming from a subordinate thread. Try using GUI.invoke_later()
to put the call to set_data() (or better, a function that bundles all
of your related set_data() calls) onto the GUI event loop:


from pyface.api import GUI

    ...
    GUI.invoke_later(array_plot_data.set_data, 'x', new_x)

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: controlling data change events

Nishith Aggarwal
Great call, it works now.

Thanks,
Nishith

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Robert Kern
Sent: Tuesday, October 02, 2012 10:59 AM
To: [hidden email]
Subject: Re: [Enthought-Dev] controlling data change events

On Tue, Oct 2, 2012 at 4:52 PM, Nishith Aggarwal <[hidden email]> wrote:
> Thanks Robert.
>
> The main thread is running the application and the data stream runs on its own thread. It retrieves the data, processes it and then calls set_data. There is no overlap of control. I have other UI controls on the app and they seem to be responsive even while the latest data has not yet shown up on the plot.

It's possible that the GUI toolkit does not like having the paint event coming from a subordinate thread. Try using GUI.invoke_later() to put the call to set_data() (or better, a function that bundles all of your related set_data() calls) onto the GUI event loop:


from pyface.api import GUI

    ...
    GUI.invoke_later(array_plot_data.set_data, 'x', new_x)

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev

________________________________

Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. Jump Trading, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product.
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev