Bug 4071 – Missing support to share memory and objects between DLLs and executable

Sharing gc-allocated or c-runtime-allocated memory, files, threads and other objects between different D-DLLs and the D-executable can be desirable, especially when working in a larger environment with dynamically loaded DLLs. The current support for this is very limited, supplying a proxy for the GC, but its implementation is incomplete, e.g. with respect to TLS memory and new threads. The following comments will show an implementation of a phobos.dll that can be accessed by a number of DLLs and the executable to create objects than can be shared freely. It also allows the usage of a single GC.

Created attachment 600 add options -exportall and -sharedlib to dmd Let's start with a few patches to dmd: 1. add option -exportall to export any mangled symbol defined by an object file (a workaround for #3956 is needed, patching obj_export to avoid indeterministic removal of leading '_' from exported symbols by optlink) 2. add option -sharedlib to switch to different default libraries phobos_shared.lib and snn_shared.lib instead of phobos.lib and snn.lib

Created attachment 601 druntime support for building phobos.dll The major changes are in druntime and involve splitting files into the part that can be shared in a single DLL and the functions that must exist per binary: - move moduleCtor/Dtor, etc. from object_.di into new file moduleinit.d (had to copy some ModuleInfo (back?) declarations to object.di to compile) - move extern(C) main() from dmain2.d into new file cmain.d - exclude __LDBLULLNG() in llmath.d because it is already in snn.lib - build druntime.obj with "-exportall" instead of druntime.lib excluding moduleinit.d, cmain.d and a few more - extracted stuff from dll_helper.d into new file shared_dll_helper.d and added a few more functions to support patching relocations

Created attachment 602 changes to phobos to build phobos.dll The patches to phobos are more-or-less limited to the makefile and adding 2 files: - phobos: build phobos.obj with "-exportall" instead of phobos.lib - add def-file exporting any sensible symbol from the C compiled modules and snn.lib - add dllmain.d that works along the line of the new dll-support in dmd 2.042 - build phobos.dll from druntime.obj, C compiled modules and the object files not included in druntime.obj using the def-file - use implib to create import library phobos_shared.lib (the optlink /IMLIB creates a corrupted lib) - add modules to the lib that must exist in each binary: minit.obj, moduleinit.obj, cmain.obj, dll_helper.obj, etc. - extracted some modules from snn.lib (constart, dllstart, winstart, excptlst, cinit, ehinit, setargv, tlsdata, tlsseg, clearbss) and put them in new lib snn_shared.lib

Created attachment 603 example using the shared dll This is the example from dll.html modified to work with the shared DLL. Please note that an executable/DLL needs a slightly different initialization: - patch relocations to stubs to import table to the destination value to allow data symbols to be accessed correctly - do not init gc - inform gc about TLS usage - add check to module(tls)ctors to avoid calling modules in other dlls These are already covered by the patches to druntime. If you apply the patches, you might need to check the path to the tools at the top of the makefiles. current restrictions: - there is no global moduleinfo-array spanning all binaries - trace module not included (always produces output) - no version check, so welcome to DLL hell!

forgot to mention that the patches are against dmd rev 431, druntime rev 282 and phobos rev 1477. you can build the dll by executing make -f win32.mak dll in the phobos directory.

I have no experiences of DLLs but here are my experiences on converting Tango to a dynamic library on Mac OS X. Don't know if any of this would work on Windows. Comment 1 An option to build as an dynamic library would be nice Comment 2 The way I solved the problems here was to declare the D main function as a weak symbol in a C file. The module constructors are handled by looping through all loaded images (binaries, dynamic libraries) and collect the module info arrays. Then combining all the arrays into one and running all the module constructors. The same is done with the exception handling tables. The following have I not done yet, but I hope it will work: In a C file, declare a function to be a module initializer that initializes the D runtime if no D main function is present. Don't know if this helps anything.

I'm not sure the way you did it on OSX is feasable on Windows aswell, because you might not have access to all the info in the DLLs (maybe it's possible if you export all necessary symbols in each DLL). But I guess it will not work for dynamically loaded DLLs. My implementation does not need a "supervising" function to run initializers, each binary just initializes the part it contains. I haven't done anything with respect to exception handling, but if I understand it correctly, the exception frames are local to the functions and binary they belong to, so they still are unwinded correctly. A simple test (catching an exception from inside phobos.dll) was working correctly.

That may actually a better idea, to let every binary handle its own initialization. Then I guess there will be no problem with module constructors that are run when they shouldn't.

Another question: since you have moved the module initialization into a new file and let every binary take care of its own initialization, couldn't that cause problems like circular dependencies that aren't detected? For example, two modules from two different libraries, each compiled into its own dll, which import the other module.

Circular dependencies between binaries are detected at link time, because you'll have the circular reference issue when creating the DLLs. moduleCtor() checks whether imported modules are in the same binary, and if not, intializers are not called assuming that they are initialized already, because the DLL they are in is already fully loaded. The proposed implementation is meant to produce a shared phobos.dll. If you want to apply it to other libraries, some functions need to be extended (especially patching relocations to data values which currently works only for a single DLL).

I have recently done a ModuleInfo refactoring along this line. I'd like to avoid having to link in a static library. The planned ELF mechanism works like the following: - The compiler emits a static init function into each object, library or executable. This function is a comdat so every DLL or EXE will have a single init function. - _minfo_beg/_minfo_end and _deh_beg/_deh_end are made static symbols. - The init function registers it's modules/EH tables with a global function in druntime. There we create an entry and store additional data like TLS index and writeable segments for GC. - The module initialization must now only iterate over all dlls and initialize them in the order of registration. If we can figure out how to create a static init function the same mechanism should work for Windows. By splitting registration and initialization we should be able to perform module construction outside DllMain of.

A bounty has been placed on this issue: https://www.flipcause.com/secure/cause_pdetails/ODcyMDE=

Any chance of this ever being solved? 😔

Rainer's add options -exportall and -sharedlib to dmd Index: backend/cdef.h =================================================================== --- backend/cdef.h (revision 431) +++ backend/cdef.h (working copy) @@ -675,6 +675,7 @@ # define WFsaveds 0x2000 // use push/pop DS for far functions # define WFdsnedgroup 0x4000 // DS != DGROUP # define WFexe 0x8000 // generating code for Windows EXE +# define WFexpall 0x10000 // generate export definition for all symbols char inline8087; /* 0: emulator 1: IEEE 754 inline 8087 code Index: backend/cgobj.c =================================================================== --- backend/cgobj.c (revision 431) +++ backend/cgobj.c (working copy) @@ -2285,16 +2285,20 @@ { char *coment; size_t len; - coment = (char *) alloca(4 + 1 + (IDMAX + IDOHD) + 1); // allow extra byte for mangling + coment = (char *) alloca(4 + 1 + (2*IDMAX + IDOHD) + 1); // allow extra byte for mangling len = obj_mangle(s,&coment[4]); assert(len <= IDMAX + IDOHD); - coment[1] = 0xA0; // comment class - coment[2] = 2; // why??? who knows - if (argsize >= 64) // we only have a 5 bit field - argsize = 0; // hope we don't need callgate - coment[3] = (argsize + 1) >> 1; // # words on stack - coment[4 + len] = 0; // no internal name - objrecord(COMENT,coment,4 + len + 1); // module name record + coment[0] = 0x80; // comment type (no purge bit set) + coment[1] = 0xA0; // comment class + coment[2] = 2; // why??? who knows + if (argsize >= 64) // we only have a 5 bit field + argsize = 0; // hope we don't need callgate + coment[3] = (argsize + 1) >> 1; // # words on stack + if(config.wflags & WFexpall) + len += obj_mangle(s,coment+4+len); // workaround for linker inconsistently removing first char + else + coment[4 + len++] = 0; // no internal name + objrecord(COMENT,coment,4 + len); // module name record } /******************************** Index: backend/out.c =================================================================== --- backend/out.c (revision 431) +++ backend/out.c (working copy) @@ -141,6 +141,8 @@ ty = s->ty(); if (ty & mTYexport && config.wflags & WFexpdef && s->Sclass != SCstatic) obj_export(s,0); // export data definition + else if(config.wflags & WFexpall && type_mangle(s->Stype)) + obj_export(s,0); // export data definition for (dt = dtstart; dt; dt = dt->DTnext) { //printf("dt = x%p, dt = %d\n",dt,dt->dt); @@ -1436,6 +1438,8 @@ !(sfunc->Sclass == SCinline && !(config.flags2 & CFG2comdat)) && sfunc->ty() & mTYexport) obj_export(sfunc,Poffset); // export function definition + else if(config.wflags & WFexpall && type_mangle(sfunc->Stype)) + obj_export(sfunc,Poffset); // export function definition if (config.fulltypes) cv_func(sfunc); // debug info for function Index: glue.c =================================================================== --- glue.c (revision 431) +++ glue.c (working copy) @@ -610,8 +610,11 @@ else if (strcmp(s->Sident, "main") == 0 && linkage == LINKc) { #if TARGET_WINDOS - objextdef("__acrtused_con"); // bring in C startup code - obj_includelib("snn.lib"); // bring in C runtime library + objextdef("__acrtused_con"); // bring in C startup code + if(global.params.sharedlib) + obj_includelib("snn_shared.lib"); // bring in shared part of C runtime library + else + obj_includelib("snn.lib"); // bring in C runtime library #endif s->Sclass = SCglobal; } Index: mars.c =================================================================== --- mars.c (revision 431) +++ mars.c (working copy) @@ -165,7 +165,7 @@ #endif fprintf(stdmsg, "\n"); fflush(stdmsg); -//halt(); +halt(); } global.errors++; } @@ -216,7 +216,7 @@ */ void halt() { -#ifdef DEBUG +#if 0 //def DEBUG *(char*)0=0; #endif } @@ -254,6 +254,7 @@ -debuglib=name set symbolic debug library to name\n\ -defaultlib=name set default library to name\n\ -deps=filename write module dependencies to filename\n%s\ + -exportall export any suitable public symbol\n\ -g add symbolic debug info\n\ -gc add symbolic debug info, pretend to be C\n\ -H generate 'header' file\n\ @@ -279,6 +280,7 @@ -quiet suppress unnecessary messages\n\ -release compile release version\n\ -run srcfile args... run resulting program, passing args\n\ + -sharedlib link against shared runtime library\n\ -unittest compile in unit tests\n\ -v verbose\n\ -version=level compile in version code >= level\n\ @@ -301,6 +303,7 @@ int status = EXIT_SUCCESS; int argcstart = argc; int setdebuglib = 0; + int setdefaultlib = 0; char noboundscheck = 0; const char *inifilename = NULL; @@ -659,6 +662,7 @@ } else if (memcmp(p + 1, "defaultlib=", 11) == 0) { + setdefaultlib = 1; global.params.defaultlibname = p + 1 + 11; } else if (memcmp(p + 1, "debuglib=", 9) == 0) @@ -668,13 +672,21 @@ } else if (memcmp(p + 1, "deps=", 5) == 0) { - global.params.moduleDepsFile = p + 1 + 5; - if (!global.params.moduleDepsFile[0]) - goto Lnoarg; - global.params.moduleDeps = new OutBuffer; - } - else if (memcmp(p + 1, "man", 3) == 0) - { + global.params.moduleDepsFile = p + 1 + 5; + if (!global.params.moduleDepsFile[0]) + goto Lnoarg; + global.params.moduleDeps = new OutBuffer; + } + else if (memcmp(p + 1, "exportall", 9) == 0) + { + global.params.exportall = true; + } + else if (memcmp(p + 1, "sharedlib", 9) == 0) + { + global.params.sharedlib = true; + } + else if (memcmp(p + 1, "man", 3) == 0) + { #if _WIN32 #if DMDV1 browse("http://www.digitalmars.com/d/1.0/dmd-windows.html"); @@ -753,6 +765,8 @@ return EXIT_FAILURE; } + if (!setdefaultlib) + global.params.defaultlibname = global.params.sharedlib ? "phobos_shared" : "phobos"; if (!setdebuglib) global.params.debuglibname = global.params.defaultlibname; @@ -805,9 +819,11 @@ global.params.libname = global.params.objname; global.params.objname = NULL; - // Haven't investigated handling these options with multiobj - if (!global.params.cov && !global.params.trace) - global.params.multiobj = 1; + // Haven't investigated handling these options with multiobj + // multiobj causes class/struct debug info to be attached to init-data, + // but this will not be linked into the executable, so this info is lost + if (!global.params.cov && !global.params.trace && !global.params.symdebug) + global.params.multiobj = 1; } else if (global.params.run) { @@ -851,6 +867,8 @@ if (global.params.useUnitTests) VersionCondition::addPredefinedGlobalIdent("unittest"); #endif + if(global.params.sharedlib) + VersionCondition::addPredefinedGlobalIdent("sharedlib"); // Initialization Type::init(); Index: mars.h =================================================================== --- mars.h (revision 431) +++ mars.h (working copy) @@ -159,6 +159,8 @@ char nofloat; // code should not pull in floating point support char Dversion; // D version number char ignoreUnsupportedPragmas; // rather than error on them + char exportall; // export any suitable symbol + char sharedlib; // link against the shared version of phobos.lib and snn.lib char *argv0; // program name Array *imppath; // array of char*'s of where to look for import modules Index: msc.c =================================================================== --- msc.c (revision 431) +++ msc.c (working copy) @@ -67,6 +67,8 @@ if (len >= 4 && stricmp(params->exefile + len - 3, "exe") == 0) config.wflags |= WFexe; } + if(params->exportall) + config.wflags |= WFexpall; config.flags4 |= CFG4underscore; #endif #if TARGET_LINUX

> Rainer's add options -exportall and -sharedlib to dmd That's pretty outdated, here is a reboot: https://github.com/dlang/dmd/pull/14849

Fixed with PR: https://github.com/dlang/dmd/pull/14849

dlang/dmd pull request #15974 "fix issue 4071: improve support for shared runtime" was merged into master: - 1f4e7217e16b784684cd0e4992810fc686373b86 by Rainer Schuetze: fix issue 4071: improve support for shared runtime - register module info of clients and run module ctors/dtors - register .data section of clients with the GC https://github.com/dlang/dmd/pull/15974