OpenMP issues when importing an Intel MKL extension

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

OpenMP issues when importing an Intel MKL extension

bdforbes
I've made an extension using f2py which calls a Fortran routine, which in turn calls MKL FFTs. The problem is, the OpenMP mk2iomp5md.dll in Enthought is conflicting with the Intel MKL OpenMP libiomp5md.dll. I get an error:

importing helloworld...
OMP: Error #15: Initializing mk2iomp5md.dll, but found libiomp5md.dll already initialized.
OMP: Hint: This may cause performance degradation and correctness issues. Set environment variable KMP_
DUPLICATE_LIB_OK=TRUE to ignore this problem and force the program to continue anyway. Please note that
 the use of KMP_DUPLICATE_LIB_OK is unsupported and using it may cause undefined behavior. For more inf
ormation, please see http://www.intel.com/software/products/support/.

If I set the environment variable as suggested, the problem goes away, but I don't like undefined behaviour.

I've compared the two dlls (mk2iomp5md.dll and libiomp5md.dll), they are identical. Is there a way for me to get my extensions .dll to refer to Enthought's OpenMP routines? I've been playing around with dlltool, trying to make inteface libraries, but whenever I try to export all the symbols from the dll, it says "no symbols".
Reply | Threaded
Open this post in threaded view
|

Re: OpenMP issues when importing an Intel MKL extension

Ilan Schnell
Hello,

to avoid conflicts with existing MKL installations on Windows, we
renamed the Fortran runtime and MKL libraries in EPD.
If you change the references to libiomp5md.dll by references to
mk2iomp5md.dll in your helloworld.pyd file, the problem should
go away.  The easiest way to do that is to use the attached Python
script that I wrote.  It looks for unrenamed strings inside DLLs
(which include .pyd files) and does a simple string replacement.

- Ilan


On Wed, Apr 27, 2011 at 5:39 AM, bdforbes <[hidden email]> wrote:

> I've made an extension using f2py which calls a Fortran routine, which in
> turn calls MKL FFTs. The problem is, the OpenMP mk2iomp5md.dll in Enthought
> is conflicting with the Intel MKL OpenMP libiomp5md.dll. I get an error:
>
> importing helloworld...
> OMP: Error #15: Initializing mk2iomp5md.dll, but found libiomp5md.dll
> already initialized.
> OMP: Hint: This may cause performance degradation and correctness issues.
> Set environment variable KMP_
> DUPLICATE_LIB_OK=TRUE to ignore this problem and force the program to
> continue anyway. Please note that
>  the use of KMP_DUPLICATE_LIB_OK is unsupported and using it may cause
> undefined behavior. For more inf
> ormation, please see http://www.intel.com/software/products/support/.
>
> If I set the environment variable as suggested, the problem goes away, but I
> don't like undefined behaviour.
>
> I've compared the two dlls (mk2iomp5md.dll and libiomp5md.dll), they are
> identical. Is there a way for me to get my extensions .dll to refer to
> Enthought's OpenMP routines? I've been playing around with dlltool, trying
> to make inteface libraries, but whenever I try to export all the symbols
> from the dll, it says "no symbols".
>
>
> --
> View this message in context: http://enthought-dev.117412.n3.nabble.com/OpenMP-issues-when-importing-an-Intel-MKL-extension-tp2869906p2869906.html
> Sent from the Enthought Dev mailing list archive at Nabble.com.
> _______________________________________________
> Enthought-Dev mailing list
> [hidden email]
> https://mail.enthought.com/mailman/listinfo/enthought-dev
>

_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev

rename_mkl.py (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OpenMP issues when importing an Intel MKL extension

bdforbes
Thanks, that worked perfectly.

I am still a little hazy on how Windows does libraries. Is the following correct?
-When I compile my Fortran source with MKL support, the object file has a .drectve which indicates a number of interface .libs to link against.
-Those .libs contain no actual code, but instead, references to particular symbols inside a particular .dll on the system (that will be delay-loaded).
-The problem I was having was that two equivalent but differently named OpenMP .dlls were both loaded into the same memory space, and they were aware of that, so they raised an exception to avoid a conflict.
-The solution was to change the name of the delay-loaded .dlls in my module to match those loaded by Enthought Python. The system will not load the same .dll twice.

Also, do you know of any good references or tools for dealing with libraries?
Reply | Threaded
Open this post in threaded view
|

Re: OpenMP issues when importing an Intel MKL extension

Ilan Schnell
Glad it worked so well.

There are several tools for dealing with libraries on
Windows, MSVC includes a command called dumpbin.
This allows you to check for dependencies of a dll for
example.  And then there are the mingw tools which
are part of EPD already.  I'm not sure what to tell you
because I don't know what you're trying to achieve.
I don't even know you're using Intel, MSVC, MinGW
or something else.
Please check the answers below as well.

- Ilan


On Wed, Apr 27, 2011 at 6:34 PM, bdforbes <[hidden email]> wrote:
> Thanks, that worked perfectly.
>
> I am still a little hazy on how Windows does libraries. Is the following
> correct?
> -When I compile my Fortran source with MKL support, the object file has a
> .drectve which indicates a number of interface .libs to link against.

Not exactly sure about this on Windows, but from my understand,
which is based on the Unix world, the object file only includes the
symbols which it defines and which it requires.  It is then the task of
the linker, to either dynamically link against the correct library which
contains the missing symbols, or (static linking) include the symbol
from an archive (.a on Unix, and this is can be a .lib on Windows).

> -Those .libs contain no actual code, but instead, references to particular
> symbols inside a particular .dll on the system (that will be delay-loaded).

The .lib can contain actual code, but can also be a wrapper for
importing the symbols for the actual dll.  In MKL you see that there
are *_dll.lib files.  There are the wrappers, but the corresponding
*.lib contain actual code for static linkage and are larger.

> -The problem I was having was that two equivalent but differently named
> OpenMP .dlls were both loaded into the same memory space, and they were
> aware of that, so they raised an exception to avoid a conflict.
> -The solution was to change the name of the delay-loaded .dlls in my module
> to match those loaded by Enthought Python. The system will not load the same
> .dll twice.

This sounds right.

> Also, do you know of any good references or tools for dealing with
> libraries?
>
>
> --
> View this message in context: http://enthought-dev.117412.n3.nabble.com/OpenMP-issues-when-importing-an-Intel-MKL-extension-tp2869906p2872591.html
> Sent from the Enthought Dev mailing list archive at Nabble.com.
> _______________________________________________
> Enthought-Dev mailing list
> [hidden email]
> https://mail.enthought.com/mailman/listinfo/enthought-dev
>
_______________________________________________
Enthought-Dev mailing list
[hidden email]
https://mail.enthought.com/mailman/listinfo/enthought-dev