Scatter inspectors and redraws

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Scatter inspectors and redraws

Brennan Sellner
Hi,

I'm having a bit of an efficiency problem with ScatterPlot and the
ScatterInspectorOverlay.  ScatterPlot triggers a complete redraw on the
change of any metadata:

     def _either_metadata_changed(self):
         self._selection_cache_valid = False
         self.invalidate_draw()
         self.request_redraw()

ScatterInspectorOverlay uses the 'hover' metadata key to determine which
data point to highlight.  The combination means that whenever we update
'hover', we don't just redraw the overlay (presumably a cheap
operation), we redraw the entire scatter plot.

It's not just ScatterPlot, though: ScatterInspectorOverlay requests a
full-plot redraw on metadata changes:

     def metadata_changed(self, object, name, old, new):
         if self.component is not None:
             self.component.request_redraw()
         return

I've attached a modified version of the scatter_inspector.py example
from Chaco 4.2.0 that prints whenever ScatterPlot is rendered, and
disables ScatterPlot's redraw-on-metadata-change.  This illustrates that
ScatterInspectorOverlay alone is triggering redraws of the underlying
ScatterPlot when its metadata changes (in this example, whenever the
mouse crosses a data point).

Now, my understanding was that the overlay layer was redrawn separately
from the rest of the plot, and a backbuffer maintained, so that the main
plot is just re-blitted, with the new overlay drawn on top.  Is that the
case?  If so, how do I go about ensuring that only the overlay is redrawn?

This is a problem for me because I've implemented a custom tool that
finds and highlights the nearest data point to the mouse, using
ScatterInspectorOverlay to draw the highlight and a metadata entry in
the underlying data source to link the two.  Because this is attempting
to update on every pixel of mouse movement, once the scatter plot gets
up to a couple thousand points, the highlight lags the mouse
significantly due to all the unnecessary redraws.  I was hoping to scale
this to data sets at least in the tens of thousands.

Any suggestions would be appreciated!

I'm using Chaco 4.2.0 on Ubuntu 12.04, with pyside.

Thanks,

-Brennan
Email Confidentiality Notice

The information contained in this transmission is confidential, proprietary or privileged and may be subject to protection under the law. This message is intended for the sole use of the individual or entity to whom it's addressed. If you are not the intended recipient, you are notified that any use, distribution or copying of the message is strictly prohibited and may subject you to criminal or civil penalties. If you received this transmission in error, please contact the sender immediately by replying to this email and delete the material from any computer.

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev

scatter_inspector.py (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Scatter inspectors and redraws

Robert Kern
On Wed, May 8, 2013 at 7:36 PM, Brennan Sellner <[hidden email]> wrote:

> Hi,
>
> I'm having a bit of an efficiency problem with ScatterPlot and the
> ScatterInspectorOverlay.  ScatterPlot triggers a complete redraw on the
> change of any metadata:
>
>     def _either_metadata_changed(self):
>         self._selection_cache_valid = False
>         self.invalidate_draw()
>         self.request_redraw()
>
> ScatterInspectorOverlay uses the 'hover' metadata key to determine which
> data point to highlight.  The combination means that whenever we update
> 'hover', we don't just redraw the overlay (presumably a cheap
> operation), we redraw the entire scatter plot.

You're right, that will be pretty annoying in your case.
Unfortunately, we don't expose what metadata is changed, so this
method can't determine if it is the 'selection' metadata, which this
renderer uses, is the one that changed.

Short workaround: subclass ScatterPlot and override this method to do nothing.

Longer term workaround: we should probably put the contents of this
method under an `if self.show_selection:` clause. That way, if you are
not showing the selection, it doesn't have to update.

Fix(?): this would need a more severe architectural change to pass
around at least the name of the updated metadata.

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Scatter inspectors and redraws

Brennan Sellner
On 05/08/2013 05:40 PM, Robert Kern wrote:

> On Wed, May 8, 2013 at 7:36 PM, Brennan Sellner <[hidden email]> wrote:
>> Hi,
>>
>> I'm having a bit of an efficiency problem with ScatterPlot and the
>> ScatterInspectorOverlay.  ScatterPlot triggers a complete redraw on the
>> change of any metadata:
>>
>>      def _either_metadata_changed(self):
>>          self._selection_cache_valid = False
>>          self.invalidate_draw()
>>          self.request_redraw()
>>
>> ScatterInspectorOverlay uses the 'hover' metadata key to determine which
>> data point to highlight.  The combination means that whenever we update
>> 'hover', we don't just redraw the overlay (presumably a cheap
>> operation), we redraw the entire scatter plot.
>
> You're right, that will be pretty annoying in your case.
> Unfortunately, we don't expose what metadata is changed, so this
> method can't determine if it is the 'selection' metadata, which this
> renderer uses, is the one that changed.
>
> Short workaround: subclass ScatterPlot and override this method to do nothing.

I attempted this in the modified scatter_inspector demo I attached
earlier: it prevents ScatterPlot from triggering a redraw, but
ScatterInspectorOverlay's metadata_changed does the same thing:

     def metadata_changed(self, object, name, old, new):
         if self.component is not None:
             self.component.request_redraw()
         return

...where self.component is the ScatterPlot.  In the spirit of blind
experimentation, I've tried just self.request_redraw(), but that
triggers the ScatterPlot draw as well...  Obviously, completely stubbing
this out won't work, since then ScatterInspectorOverlay will never
redraw itself.

Is there any way to request a redraw of only the overlay?

Thanks!

-Brennan

Email Confidentiality Notice

The information contained in this transmission is confidential, proprietary or privileged and may be subject to protection under the law. This message is intended for the sole use of the individual or entity to whom it's addressed. If you are not the intended recipient, you are notified that any use, distribution or copying of the message is strictly prohibited and may subject you to criminal or civil penalties. If you received this transmission in error, please contact the sender immediately by replying to this email and delete the material from any computer.
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Scatter inspectors and redraws

Robert Kern
On Wed, May 8, 2013 at 10:47 PM, Brennan Sellner <[hidden email]> wrote:

> On 05/08/2013 05:40 PM, Robert Kern wrote:
>>
>> On Wed, May 8, 2013 at 7:36 PM, Brennan Sellner <[hidden email]>
>> wrote:
>>>
>>> Hi,
>>>
>>> I'm having a bit of an efficiency problem with ScatterPlot and the
>>> ScatterInspectorOverlay.  ScatterPlot triggers a complete redraw on the
>>> change of any metadata:
>>>
>>>      def _either_metadata_changed(self):
>>>          self._selection_cache_valid = False
>>>          self.invalidate_draw()
>>>          self.request_redraw()
>>>
>>> ScatterInspectorOverlay uses the 'hover' metadata key to determine which
>>> data point to highlight.  The combination means that whenever we update
>>> 'hover', we don't just redraw the overlay (presumably a cheap
>>> operation), we redraw the entire scatter plot.
>>
>>
>> You're right, that will be pretty annoying in your case.
>> Unfortunately, we don't expose what metadata is changed, so this
>> method can't determine if it is the 'selection' metadata, which this
>> renderer uses, is the one that changed.
>>
>> Short workaround: subclass ScatterPlot and override this method to do
>> nothing.
>
>
> I attempted this in the modified scatter_inspector demo I attached
> earlier: it prevents ScatterPlot from triggering a redraw, but
> ScatterInspectorOverlay's metadata_changed does the same thing:
>
>
>     def metadata_changed(self, object, name, old, new):
>         if self.component is not None:
>             self.component.request_redraw()
>         return
>
> ...where self.component is the ScatterPlot.  In the spirit of blind
> experimentation, I've tried just self.request_redraw(), but that
> triggers the ScatterPlot draw as well...  Obviously, completely stubbing
> this out won't work, since then ScatterInspectorOverlay will never
> redraw itself.
>
> Is there any way to request a redraw of only the overlay?

Ah, yes. This is where backbuffering comes in. Take a look at
examples/demo/bigdata.py:

https://github.com/enthought/chaco/blob/master/examples/demo/bigdata.py#L38

In short, you add `use_backbuffer=True` to your PlotContainer. I don't
think you need to do anything else. The RangeSelectionOverlay used in
this example does a similar `self.component.request_redraw()` on
metadata changes, and it is still fast with backbuffering.

Backbuffering allows the drawing of the overlay to be completely
separate from the drawing of everything underneath. Normally, the
overlays are just drawn on the same GraphicsContext, just after all of
the underlying layers. With backbuffering, the other layers are drawn
into a backbuffer GraphicsContext. This backbuffer is blitted into the
main GraphicsContext, then the overlay gets drawn on top. When the
underlying layers don't change (i.e. you don't call
`invalidate_draw()` on the ScatterPlot), the backbuffer does not need
to get updated, so you only pay the cost of blitting in the backbuffer
and drawing the overlay whenever the overlay changes. Here is the
relevant bit of code:

https://github.com/enthought/enable/blob/master/enable/component.py#L724

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Scatter inspectors and redraws

Brennan Sellner
On 05/10/2013 06:02 AM, Robert Kern wrote:

> On Wed, May 8, 2013 at 10:47 PM, Brennan Sellner <[hidden email]> wrote:
>> On 05/08/2013 05:40 PM, Robert Kern wrote:
>>>
>>> On Wed, May 8, 2013 at 7:36 PM, Brennan Sellner <[hidden email]>
>>> wrote:
>> Is there any way to request a redraw of only the overlay?
>
> Ah, yes. This is where backbuffering comes in. Take a look at
> examples/demo/bigdata.py:
>
> https://github.com/enthought/chaco/blob/master/examples/demo/bigdata.py#L38
>
> In short, you add `use_backbuffer=True` to your PlotContainer. I don't
> think you need to do anything else. The RangeSelectionOverlay used in
> this example does a similar `self.component.request_redraw()` on
> metadata changes, and it is still fast with backbuffering.
>
> Backbuffering allows the drawing of the overlay to be completely
> separate from the drawing of everything underneath. Normally, the
> overlays are just drawn on the same GraphicsContext, just after all of
> the underlying layers. With backbuffering, the other layers are drawn
> into a backbuffer GraphicsContext. This backbuffer is blitted into the
> main GraphicsContext, then the overlay gets drawn on top. When the
> underlying layers don't change (i.e. you don't call
> `invalidate_draw()` on the ScatterPlot), the backbuffer does not need
> to get updated, so you only pay the cost of blitting in the backbuffer
> and drawing the overlay whenever the overlay changes. Here is the
> relevant bit of code:
>
> https://github.com/enthought/enable/blob/master/enable/component.py#L724

Ah, I thought that was enabled by default.  That did indeed do the trick
-- updates are now *much* faster.  Thanks!

In case it's useful to others, I ended up implementing the following
metadata handler in a subclass of ScatterPlot.  We don't use selections
at the moment, so I've only tested the "doesn't update without
selections" case.

     def _either_metadata_changed(self):
         if not self.show_selection:
             return

         haveSelection = False
         for meta in ( self.index.metadata, self.value.metadata ):
             if ( meta.get ( 'selection_masks', None )
                  or meta.get ( 'selections', None ) ):
                 haveSelection = True
                 break

         if haveSelection:
             self._selection_cache_valid = False
             self.invalidate_draw()
             self.request_redraw()

One other gotcha on this front is needing to apply the same treatment to
other subclasses of ScatterPlot, such as QuiverPlot.

Thanks for the help!

-Brennan


Email Confidentiality Notice

The information contained in this transmission is confidential, proprietary or privileged and may be subject to protection under the law. This message is intended for the sole use of the individual or entity to whom it's addressed. If you are not the intended recipient, you are notified that any use, distribution or copying of the message is strictly prohibited and may subject you to criminal or civil penalties. If you received this transmission in error, please contact the sender immediately by replying to this email and delete the material from any computer.
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev