strange conflict between traitsui.toolkit and subprocess

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

strange conflict between traitsui.toolkit and subprocess

Jonathan Guyer
We've encountered a strange problem in our test suite where

{{{
from mayavi.sources.vtk_file_reader import VTKFileReader
}}}

in one test causes `Popen` failures in subsequent tests.


I've been able to isolate the problem down to:

{{{
from traitsui.toolkit import toolkit
toolkit()

import subprocess
subprocess.Popen("echo broken",
                 stderr=subprocess.PIPE, shell=True).stderr.readlines()
}}}

which prints

{{{
broken
}}}

and then raises

{{{
Traceback (most recent call last):
  File "traitsTest.py", line 100, in <module>
    stderr=subprocess.PIPE, shell=True).stderr.readlines()
IOError: [Errno 4] Interrupted system call
}}}

This does not fail 100% of the time, but most of the time.

The same thing happens if I define and read stdout.


----

Mac OS X Snow Leopard 10.6.8

Python 2.7.2 via Homebrew

numpy version 1.6.1 via pip
scipy version 0.9.0 via pip
configobj version 4.7.2 via pip
envisage version 4.0.0 via pip
mayavi version 4.0.0 via pip

traits version 4.0.0 and traitsui version 4.0.1 were installed automatically by pip in the process of installing the above.
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: strange conflict between traitsui.toolkit and subprocess

Robert Kern
On Fri, Sep 23, 2011 at 9:07 AM, Jonathan Guyer <[hidden email]> wrote:

> We've encountered a strange problem in our test suite where
>
> {{{
> from mayavi.sources.vtk_file_reader import VTKFileReader
> }}}
>
> in one test causes `Popen` failures in subsequent tests.
>
>
> I've been able to isolate the problem down to:
>
> {{{
> from traitsui.toolkit import toolkit
> toolkit()
>
> import subprocess
> subprocess.Popen("echo broken",
>                 stderr=subprocess.PIPE, shell=True).stderr.readlines()
> }}}
>
> which prints
>
> {{{
> broken
> }}}
>
> and then raises
>
> {{{
> Traceback (most recent call last):
>  File "traitsTest.py", line 100, in <module>
>    stderr=subprocess.PIPE, shell=True).stderr.readlines()
> IOError: [Errno 4] Interrupted system call
> }}}
>
> This does not fail 100% of the time, but most of the time.
>
> The same thing happens if I define and read stdout.

This is a known issue with some GUI toolkits. They install signal
handlers that interfere with this style of subprocess calls.

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: strange conflict between traitsui.toolkit and subprocess

Jonathan Guyer

On Oct 4, 2011, at 10:14 PM, Robert Kern wrote:

> On Fri, Sep 23, 2011 at 9:07 AM, Jonathan Guyer <[hidden email]> wrote:
>> We've encountered a strange problem in our test suite where
>>
>> {{{
>> from mayavi.sources.vtk_file_reader import VTKFileReader
>> }}}
>>
>> in one test causes `Popen` failures in subsequent tests.

> This is a known issue with some GUI toolkits. They install signal
> handlers that interfere with this style of subprocess calls.

Ugh. Any workarounds, other than just "try another GUI"?
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: strange conflict between traitsui.toolkit and subprocess

Robert Kern
On Wed, Oct 5, 2011 at 8:24 AM, Jonathan Guyer <[hidden email]> wrote:

>
> On Oct 4, 2011, at 10:14 PM, Robert Kern wrote:
>
>> On Fri, Sep 23, 2011 at 9:07 AM, Jonathan Guyer <[hidden email]> wrote:
>>> We've encountered a strange problem in our test suite where
>>>
>>> {{{
>>> from mayavi.sources.vtk_file_reader import VTKFileReader
>>> }}}
>>>
>>> in one test causes `Popen` failures in subsequent tests.
>
>> This is a known issue with some GUI toolkits. They install signal
>> handlers that interfere with this style of subprocess calls.
>
> Ugh. Any workarounds, other than just "try another GUI"?

Here is a chunk of code that shows how IPython works around this problem:

https://github.com/ipython/ipython/blob/master/IPython/utils/rlineimpl.py#L55

I do recommend using .communicate() instead of fiddling with the
.stdout/.stderr attributes. You want to avoid any impedance mismatches
between how fast the program produces text and how fast you consume
it.

Another approach for things that need to execute before continuing on
is to wrap it in a "while True:" and break once it executes
completely.

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Using Chaco LineInspector to pull a slice out of a 2d image

Dharhas Pothina

Hi,

I'm trying to build a simple chaco plot that displays two plots: a 2D image on the left and a 1D vertical slice of that image as determined by LineInspector on the right. This is something similar to the scalar_image_function_editor example, however I'm finding that example fairly difficult to understand since the way chaco is used in that example seems much more complicated than the simpler way I've picked up from the tutorials and other examples. I'm sure it is a better way to do things but I'm just having difficulty wrapping my head around the approach.

I've tried creating a simpler example but I'm unable to work out how to get the index from LineInspector to extract the data for the second plot.
According the LineInspector api reference online 'writes the current data space point to the appropriate data source's metadata'. I haven't been able to work out where this metadata is.

I'm attaching my non-functioning script to this email. I'm not sure what the function _profile_plot_changed() should be called and how to get the x index it requires from LineInspector.

Any pointers are appreciated.

- dharhas

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Chaco LineInspector to pull a slice out of a 2d image

Dharhas Pothina
In reply to this post by Robert Kern


Couldn't tell if my attached script came through so I'm pasted the code below:

#imports
import numpy as np
import random

from enable.api import Component, ComponentEditor
from traits.api import HasTraits, Instance
from traitsui.api import Item, View, Group
from chaco.api import Plot, ArrayPlotData, HPlotContainer, reverse, gray
from chaco.tools.api import PanTool, ZoomTool, LineInspector

class TracePlotUI(HasTraits):
    #Traits view definitions:
    traits_view = View(
        Group(Item('container',
                   editor=ComponentEditor(size=(800,600)),
                   show_label=False)),
        #buttons=NoButtons,
        resizable=True)

    #---------------------------------------------------------------------------
    # Public View interface
    #---------------------------------------------------------------------------
    def __init__(self, image, *args, **kwargs):
        super(TracePlotUI, self).__init__(*args, **kwargs)
        self.image = image
        self.pd = ArrayPlotData(traces = image,
                                )
        self.create_plot()

    def create_plot(self):
        self.profile_plot = Plot(self.pd)
        self.profile_plot.x_axis.orientation = 'top'
        self.profile_plot.default_origin = 'top left'
        self.profile_plot.img_plot("traces", colormap=reverse(gray))
        self.profile_plot.overlays.append

        # add tools to plot
        self.profile_plot.tools.append(PanTool(self.profile_plot,
                                           constrain_key="shift"))
        self.profile_plot.overlays.append(ZoomTool(component=self.profile_plot,
                                            tool_mode="box", always_on=False))
        self.profile_plot.overlays.append(LineInspector(component=self.profile_plot,
                                               axis='index_x',
                                               inspect_mode="indexed",
                                               write_metadata=True,
                                               is_listener=False,
                                               color="blue"))
         
        self.pd.set_data("line_index", np.arange(self.image.shape[0]))
        self.pd.set_data("line_value", np.array([]))

        self.trace_plot = Plot(self.pd, orientation='v')
        self.trace_plot.default_origin = 'top left'

        self.trace_plot.plot(("line_index", "line_value"),
                             line_style="dot")

        self.container = HPlotContainer()
        self.container.add(self.profile_plot)
        self.container.add(self.trace_plot)

    #what should this be?
    def _profile_plot_changed(self):
        idx = random.randint(0, self.image.shape[1]) #get index idx from lineinspector somehow
        print idx
        self.pd.set_data("line_value", self.image[:,idx])
        self.container.invalidate_draw()
        self.container.request_redraw()
         

#data = np.load('test_profile.npy')
data = np.random.uniform(0,256,(200,400))

viewer = TracePlotUI(data.T)
viewer.configure_traits()


>>> Dharhas Pothina 10/5/2011 12:07 PM >>>

Hi,


I'm trying to build a simple chaco plot that displays two plots: a 2D image on the left and a 1D vertical slice of that image as determined by LineInspector on the right. This is something similar to the scalar_image_function_editor example, however I'm finding that example fairly difficult to understand since the way chaco is used in that example seems much more complicated than the simpler way I've picked up from the tutorials and other examples. I'm sure it is a better way to do things but I'm just having difficulty wrapping my head around the approach.


I've tried creating a simpler example but I'm unable to work out how to get the index from LineInspector to extract the data for the second plot.
According the LineInspector api reference online 'writes the current data space point to the appropriate data source's metadata'. I haven't been able to work out where this metadata is.


I'm attaching my non-functioning script to this email. I'm not sure what the function _profile_plot_changed() should be called and how to get the x index it requires from LineInspector.


Any pointers are appreciated.


- dharhas
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Chaco LineInspector to pull a slice out of a 2d image

John Wiggins
In reply to this post by Dharhas Pothina
Hello Dharhas,

I've had to do roughly the same thing in my own application. Here's what I did:

- When creating the image plot, I add a LineInspector to the overlays:
img_plot = plot.img_plot(...)[0]
img_plot.overlays.append(LineInspector(img_plot, axis="index_x",
color="white", inspect_mode="indexed", write_metadata=True))

- Add a traits listener for the metadata:
def line_inspector_update():
    metadata = img_plot.index.metadata
    if metadata is not None and "selections" in metadata:
        try:
            line_index = metadata["selections"][0]
        except IndexError:
            line_index = 0
img_plot.on_trait_change(line_inspector_update, "index.metadata[]")

line_index will then be some column index in your image data.

- John

On Wed, Oct 5, 2011 at 12:07 PM, Dharhas Pothina
<[hidden email]> wrote:

>
> Hi,
>
> I'm trying to build a simple chaco plot that displays two plots: a 2D image on the left and a 1D vertical slice of that image as determined by LineInspector on the right. This is something similar to the scalar_image_function_editor example, however I'm finding that example fairly difficult to understand since the way chaco is used in that example seems much more complicated than the simpler way I've picked up from the tutorials and other examples. I'm sure it is a better way to do things but I'm just having difficulty wrapping my head around the approach.
>
> I've tried creating a simpler example but I'm unable to work out how to get the index from LineInspector to extract the data for the second plot.
> According the LineInspector api reference online 'writes the current data space point to the appropriate data source's metadata'. I haven't been able to work out where this metadata is.
>
> I'm attaching my non-functioning script to this email. I'm not sure what the function _profile_plot_changed() should be called and how to get the x index it requires from LineInspector.
>
> Any pointers are appreciated.
>
> - dharhas
>
> _______________________________________________
> Enthought-Dev mailing list
> [hidden email]
> https://mail.enthought.com/mailman/listinfo/enthought-dev
>
>
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Chaco LineInspector to pull a slice out of a 2d image

John Wiggins
Here's the same basic thing, using your example:

#imports
import numpy as np
import random

from enable.api import Component, ComponentEditor
from traits.api import Any, HasTraits, Instance, on_trait_change
from traitsui.api import Item, View, Group
from chaco.api import Plot, ArrayPlotData, HPlotContainer, reverse, gray
from chaco.tools.api import PanTool, ZoomTool, LineInspector

class TracePlotUI(HasTraits):

    container = Any
    image_plot = Any
    #Traits view definitions:
    traits_view = View(
                    Group(
                        Item('container',
                              editor=ComponentEditor(size=(800,600)),
                              show_label=False,
                            ),
                        ),
                    #buttons=NoButtons,
                    resizable=True,
                    )

    #---------------------------------------------------------------------------
    # Public View interface
    #---------------------------------------------------------------------------
    def __init__(self, image, *args, **kwargs):
        super(TracePlotUI, self).__init__(*args, **kwargs)
        self.image = image
        self.pd = ArrayPlotData(traces = image,)
        self.create_plot()

    def create_plot(self):
        self.profile_plot = Plot(self.pd)
        self.profile_plot.x_axis.orientation = 'top'
        self.profile_plot.default_origin = 'top left'
        self.image_plot = self.profile_plot.img_plot("traces",
colormap=reverse(gray))[0]


        # add tools to plot
        self.profile_plot.tools.append(PanTool(self.profile_plot,
                                  constrain_key="shift"))
        self.profile_plot.overlays.append(ZoomTool(component=self.profile_plot,
                                   tool_mode="box", always_on=False))
        self.image_plot.overlays.append(LineInspector(component=self.image_plot,
                                             axis='index_x',
                                             inspect_mode="indexed",
                                             write_metadata=True,
                                             color="blue"))

        self.pd.set_data("line_index", np.arange(self.image.shape[0]))
        self.pd.set_data("line_value", np.array([]))

        self.trace_plot = Plot(self.pd, orientation='v')
        self.trace_plot.default_origin = 'top left'

        self.trace_plot.plot(("line_index", "line_value"),
                    line_style="dot")

        self.container = HPlotContainer()
        self.container.add(self.profile_plot)
        self.container.add(self.trace_plot)

    @on_trait_change("image_plot.index.metadata[]")
    def _line_inspector_moved(self):
        metadata = self.image_plot.index.metadata

        if metadata is not None and "selections" in metadata:
            try:
                idx = metadata["selections"][0]
            except:
                idx = 0
            self.pd.set_data("line_value", self.image[:,idx])
            if self.container is not None:
                self.container.invalidate_draw()
                self.container.request_redraw()


#data = np.load('test_profile.npy')
data = np.random.uniform(0,256,(200,400))

viewer = TracePlotUI(data.T)
viewer.configure_traits()


On Wed, Oct 5, 2011 at 12:31 PM, John Wiggins <[hidden email]> wrote:

> Hello Dharhas,
>
> I've had to do roughly the same thing in my own application. Here's what I did:
>
> - When creating the image plot, I add a LineInspector to the overlays:
> img_plot = plot.img_plot(...)[0]
> img_plot.overlays.append(LineInspector(img_plot, axis="index_x",
> color="white", inspect_mode="indexed", write_metadata=True))
>
> - Add a traits listener for the metadata:
> def line_inspector_update():
>    metadata = img_plot.index.metadata
>    if metadata is not None and "selections" in metadata:
>        try:
>            line_index = metadata["selections"][0]
>        except IndexError:
>            line_index = 0
> img_plot.on_trait_change(line_inspector_update, "index.metadata[]")
>
> line_index will then be some column index in your image data.
>
> - John
>
> On Wed, Oct 5, 2011 at 12:07 PM, Dharhas Pothina
> <[hidden email]> wrote:
>>
>> Hi,
>>
>> I'm trying to build a simple chaco plot that displays two plots: a 2D image on the left and a 1D vertical slice of that image as determined by LineInspector on the right. This is something similar to the scalar_image_function_editor example, however I'm finding that example fairly difficult to understand since the way chaco is used in that example seems much more complicated than the simpler way I've picked up from the tutorials and other examples. I'm sure it is a better way to do things but I'm just having difficulty wrapping my head around the approach.
>>
>> I've tried creating a simpler example but I'm unable to work out how to get the index from LineInspector to extract the data for the second plot.
>> According the LineInspector api reference online 'writes the current data space point to the appropriate data source's metadata'. I haven't been able to work out where this metadata is.
>>
>> I'm attaching my non-functioning script to this email. I'm not sure what the function _profile_plot_changed() should be called and how to get the x index it requires from LineInspector.
>>
>> Any pointers are appreciated.
>>
>> - dharhas
>>
>> _______________________________________________
>> Enthought-Dev mailing list
>> [hidden email]
>> https://mail.enthought.com/mailman/listinfo/enthought-dev
>>
>>
>
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Chaco LineInspector to pull a slice out of a 2d image

Dharhas Pothina


Thanks. I just got it working myself, but your code looks cleaner.

- dharhas

>>> John Wiggins <[hidden email]> 10/5/2011 1:39 PM >>>
Here's the same basic thing, using your example:

#imports
import numpy as np
import random

from enable.api import Component, ComponentEditor
from traits.api import Any, HasTraits, Instance, on_trait_change
from traitsui.api import Item, View, Group
from chaco.api import Plot, ArrayPlotData, HPlotContainer, reverse, gray
from chaco.tools.api import PanTool, ZoomTool, LineInspector

class TracePlotUI(HasTraits):

    container = Any
    image_plot = Any
    #Traits view definitions:
    traits_view = View(
                    Group(
                        Item('container',
                              editor=ComponentEditor(size=(800,600)),
                              show_label=False,
                            ),
                        ),
                    #buttons=NoButtons,
                    resizable=True,
                    )

    #---------------------------------------------------------------------------
    # Public View interface
    #---------------------------------------------------------------------------
    def __init__(self, image, *args, **kwargs):
        super(TracePlotUI, self).__init__(*args, **kwargs)
        self.image = image
        self.pd = ArrayPlotData(traces = image,)
        self.create_plot()

    def create_plot(self):
        self.profile_plot = Plot(self.pd)
        self.profile_plot.x_axis.orientation = 'top'
        self.profile_plot.default_origin = 'top left'
        self.image_plot = self.profile_plot.img_plot("traces",
colormap=reverse(gray))[0]


        # add tools to plot
        self.profile_plot.tools.append(PanTool(self.profile_plot,
                                  constrain_key="shift"))
        self.profile_plot.overlays.append(ZoomTool(component=self.profile_plot,
                                   tool_mode="box", always_on=False))
        self.image_plot.overlays.append(LineInspector(component=self.image_plot,
                                             axis='index_x',
                                             inspect_mode="indexed",
                                             write_metadata=True,
                                             color="blue"))

        self.pd.set_data("line_index", np.arange(self.image.shape[0]))
        self.pd.set_data("line_value", np.array([]))

        self.trace_plot = Plot(self.pd, orientation='v')
        self.trace_plot.default_origin = 'top left'

        self.trace_plot.plot(("line_index", "line_value"),
                    line_style="dot")

        self.container = HPlotContainer()
        self.container.add(self.profile_plot)
        self.container.add(self.trace_plot)

    @on_trait_change("image_plot.index.metadata[]")
    def _line_inspector_moved(self):
        metadata = self.image_plot.index.metadata

        if metadata is not None and "selections" in metadata:
            try:
                idx = metadata["selections"][0]
            except:
                idx = 0
            self.pd.set_data("line_value", self.image[:,idx])
            if self.container is not None:
                self.container.invalidate_draw()
                self.container.request_redraw()


#data = np.load('test_profile.npy')
data = np.random.uniform(0,256,(200,400))

viewer = TracePlotUI(data.T)
viewer.configure_traits()


On Wed, Oct 5, 2011 at 12:31 PM, John Wiggins <[hidden email]> wrote:

> Hello Dharhas,
>
> I've had to do roughly the same thing in my own application. Here's what I did:
>
> - When creating the image plot, I add a LineInspector to the overlays:
> img_plot = plot.img_plot(...)[0]
> img_plot.overlays.append(LineInspector(img_plot, axis="index_x",
> color="white", inspect_mode="indexed", write_metadata=True))
>
> - Add a traits listener for the metadata:
> def line_inspector_update():
>    metadata = img_plot.index.metadata
>    if metadata is not None and "selections" in metadata:
>        try:
>            line_index = metadata["selections"][0]
>        except IndexError:
>            line_index = 0
> img_plot.on_trait_change(line_inspector_update, "index.metadata[]")
>
> line_index will then be some column index in your image data.
>
> - John
>
> On Wed, Oct 5, 2011 at 12:07 PM, Dharhas Pothina
> <[hidden email]> wrote:
>>
>> Hi,
>>
>> I'm trying to build a simple chaco plot that displays two plots: a 2D image on the left and a 1D vertical slice of that image as determined by LineInspector on the right. This is something similar to the scalar_image_function_editor example, however I'm finding that example fairly difficult to understand since the way chaco is used in that example seems much more complicated than the simpler way I've picked up from the tutorials and other examples. I'm sure it is a better way to do things but I'm just having difficulty wrapping my head around the approach.
>>
>> I've tried creating a simpler example but I'm unable to work out how to get the index from LineInspector to extract the data for the second plot.
>> According the LineInspector api reference online 'writes the current data space point to the appropriate data source's metadata'. I haven't been able to work out where this metadata is.
>>
>> I'm attaching my non-functioning script to this email. I'm not sure what the function _profile_plot_changed() should be called and how to get the x index it requires from LineInspector.
>>
>> Any pointers are appreciated.
>>
>> - dharhas
>>
>> _______________________________________________
>> Enthought-Dev mailing list
>> [hidden email]
>> https://mail.enthought.com/mailman/listinfo/enthought-dev
>>
>>
>
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: strange conflict between traitsui.toolkit and subprocess

Jonathan Guyer
In reply to this post by Robert Kern

On Oct 5, 2011, at 11:06 AM, Robert Kern wrote:

> Here is a chunk of code that shows how IPython works around this problem:
>
> https://github.com/ipython/ipython/blob/master/IPython/utils/rlineimpl.py#L55

That's very helpful, thanks.

How does the `p.returncode == 4` ever get checked, though? I get an IOError raised when I call .communicate() and if I catch that, p.returncode is None.

> I do recommend using .communicate() instead of fiddling with the
> .stdout/.stderr attributes. You want to avoid any impedance mismatches
> between how fast the program produces text and how fast you consume
> it.

Good point, thanks.

> Another approach for things that need to execute before continuing on
> is to wrap it in a "while True:" and break once it executes
> completely.

That's probably all I need, but the exponential (geometric?) timer is good to know about.


_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev
Reply | Threaded
Open this post in threaded view
|

Re: strange conflict between traitsui.toolkit and subprocess

Robert Kern
On Wed, Oct 5, 2011 at 8:35 PM, Jonathan Guyer <[hidden email]> wrote:

>
> On Oct 5, 2011, at 11:06 AM, Robert Kern wrote:
>
>> Here is a chunk of code that shows how IPython works around this problem:
>>
>> https://github.com/ipython/ipython/blob/master/IPython/utils/rlineimpl.py#L55
>
> That's very helpful, thanks.
>
> How does the `p.returncode == 4` ever get checked, though? I get an IOError raised when I call .communicate() and if I catch that, p.returncode is None.

Honestly, I'm not really sure.

--
Robert Kern
Enthought
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev