I am running into some issues where moduleinfo and classinfo that should be included in the runtime type info is missing, but only when linking via static libs.
An example:
libci.d:
module libci;
class C
{
}
extern(C) void foo()
{
import std.stdio;
auto c = new C;
writeln("classinfo of C: ", typeid(c));
auto x = ClassInfo.find("libci.C");
writeln("searching for it: ", x);
}
version(works) static this()
{
import std.stdio;
writeln("here");
}
testit.d:
import std.stdio;
import libci;
void main()
{
foo();
auto ci = ClassInfo.find("libci.C");
writeln("in main: ", ci);
foreach(m; ModuleInfo)
{
writeln(m.name);
}
}
If I build with:
dmd testit.d libci.d
Then I get:
classinfo of C: libci.C
searching for it: libci.C
in main: libci.C
testit
libci
object
... // all other modules
If I build with this:
dmd -lib libci.d
dmd -L-L. -L-lci testit.d
I get:
classinfo of C: libci.C
searching for it: null
in main: null
testit
object
... // all other modules, no libci!
So somehow the libci ModuleInfo isn't being linked, even though the classinfo is usable if you use it directly. IMO, the ModuleInfo is being incorrectly trimmed from the executable.
If you enable version=works when building the library, then the ModuleInfo is included.
Unsure if this is druntime or dmd bug, marked as dmd.
Note, this behavior is really old, and on both Linux and OSX. Tested as far back as 2.064.
Comment #1 by doob — 2016-08-24T07:03:20Z
Hmm, I actually fixed this, but only for Objective-C module info. I'll see if I can come up with a fix.
It goes against the intention of static libraries to drag unused classes and module into a binary, and that also goes against our permanent fight against giant binaries.
In fact there is issue 14555 and a PR (https://github.com/dlang/dmd/pull/4638) in the adverse direction.
If you really want to drag in all classes from a static library, one approach is to reference all of them from a common module.
Having an explicit list of all classes kind of contradicts the purpose of Object.factory, but I don't see how we can easily achieve both.
Another approach is to generate object files for each module and explicitly link against all of them (the linker won't discard object files).
Dub supports this as --build-mode=singleFile IIRC.
Yet another approach is to use shared libraries instead.
Comment #4 by code — 2016-09-20T22:40:59Z
Another rather obvious solution is to tell the linker to include the whole archive, sensibly named --whole-archive/--no-whole-archive for ld.
Unfortunately dmd currently reorders linker flags (see issue 15574), so the following doesn't work.
-L--whole-archive mystaticlib.a -L--no-whole-archive
Instead you'll have to invoke cc for linking yourself atm.
Comment #5 by schveiguy — 2016-09-21T15:03:45Z
(In reply to Martin Nowak from comment #3)
> It goes against the intention of static libraries to drag unused classes and
> module into a binary, and that also goes against our permanent fight against
> giant binaries.
This is penny-wise and pound-foolish. First we need to solve megabyte sized template symbol names. Then we need full shared library support. The few kilobytes that moduleinfo adds is nothing. Trim out unused classinfo? Then we need to remove Object.factory method.
> In fact there is issue 14555 and a PR
> (https://github.com/dlang/dmd/pull/4638) in the adverse direction.
>
> If you really want to drag in all classes from a static library, one
> approach is to reference all of them from a common module.
I referenced the class info from within the module, and called that function. Why is the linker excluding it? This seems like a real bug.
In fact, I actually TRY your suggestion to solve the original problem and it doesn't work. See the PR: https://github.com/dlang/phobos/pull/4744#issuecomment-241876407
> Having an explicit list of all classes kind of contradicts the purpose of
> Object.factory, but I don't see how we can easily achieve both.
We can't have Object.factory and then trim out all classinfo that aren't used directly. But in this case, I AM using it directly, and it's STILL being trimmed out.
(In reply to Steven Schveighoffer from comment #5)
> This is penny-wise and pound-foolish. First we need to solve megabyte sized
> template symbol names. Then we need full shared library support. The few
> kilobytes that moduleinfo adds is nothing. Trim out unused classinfo? Then
> we need to remove Object.factory method.
- Yes we need to solve the long symbol names, but they don't bloat the code segment or affect the performance
- shared libraries were just one workaround I named (and they can be fully used on linux/FreeBSD atm.)
- It's not the ModuleInfo or the ClassInfo, it's the fact, that this drags in the complete tree of symbols reachable by any referenced class, i.e. the whole module in most cases. The static constructors and class references are one of the main reasons for our huge binaries (code segment, not file size which is just a distribution problem).
- Object.factory is slow by design, it's in the interest of everyone to avoid that facility if possible, and there aren't many use-cases that really require it
If we have enough prove that it's a necessary pattern, I'd be in favor of adding a global, synchronized, namespaced dynamic class registration facility in phobos or druntime. That could be used for certain patterns such as registering deserializable subclasses, factory.register!("orange.deserializer", MyOpenSubclass).
It could also make use of a hash table to speed up instantiating, and maybe even support non-standard constructors.
The question is really, do we really need it, and who builds it.
Comment #8 by schveiguy — 2016-10-11T13:43:10Z
Whether we need it or not, it's currently a feature of the runtime. Do we not care about existing code anymore?
I'm totally in favor of deprecating Object.factory, and also in favor of your update to std.encoding that makes this issue moot for phobos at least. But there may be code out there which expects Object.factory to work, and in some cases, it will not.
Note, that in the example I give, including the module info does little to increase the binary size. The ClassInfo is already present in the binary, and therefore pulls in everything else the class may use.
Not only that, but building with the -lib switch does not work, but building without does. This kind of inconsistency is not good for D, regardless of the utility of the Object.factory feature. It makes no sense that the compiler is "stupider" about trimming fat when it has full code access than the linker.
Comment #9 by ketmar — 2016-10-11T14:03:15Z
(In reply to Steven Schveighoffer from comment #8)
> But there may be code out there which expects Object.factory to work, and in
> some cases, it will not.
and there is. some of my UI parsers *depends* on the fact that class list and factory are there. i did that exactly to avoid manual registering of UI classes, which is highly error-prone.
that is, some features are not removed 'cause there *may* be the code that depends on those. now there *is* the code that depends on full module/class infos and factory. this alone should rule "let's remove object.factory" out. let's see if real code has more weight than imaginary code. ;-)
Comment #10 by ketmar — 2016-12-22T03:23:16Z
by the way. applying 6076 broke windows builds: phobos refused to build with "duplicate ModuleInfo for object.d" error.
Comment #11 by doob — 2016-12-22T07:26:40Z
(In reply to Ketmar Dark from comment #10)
> by the way. applying 6076 broke windows builds: phobos refused to build with
> "duplicate ModuleInfo for object.d" error.
Yeah, I noticed that four months ago when the PR did not pass the tests ;)
Comment #12 by ketmar — 2016-12-22T07:36:35Z
funny, it doesn't cause any problems on GNU/Linux, only windows builds are affected.
Comment #13 by doob — 2016-12-22T19:35:01Z
(In reply to Ketmar Dark from comment #12)
> funny, it doesn't cause any problems on GNU/Linux, only windows builds are
> affected.
It seems to work on all platforms except Windows. Might be something with the linker that is different.
Comment #14 by ketmar — 2016-12-22T19:47:33Z
i suspect optlink. sadly, i can't check with ms linker.
(ketmar dreaming of getting rid of optlink and moving the defaults to mingw's binutils)
Comment #15 by ketmar — 2016-12-22T19:49:10Z
(just a speculation) maybe ld merges identical symbols from different libraries by default or something, and optlink doesn't.
Comment #16 by robert.schadek — 2024-12-13T18:49:38Z