Bug 22960 – importC: K&R-style functions assume variadic calling convention

Status
RESOLVED
Resolution
FIXED
Severity
normal
Priority
P3
Component
dmd
Product
D
Version
D2
Platform
x86_64
OS
Linux
Creation time
2022-03-29T22:37:23Z
Last change time
2023-04-19T07:51:39Z
Keywords
ImportC, pull
Assigned to
No Owner
Creator
duser

Comments

Comment #0 by duser — 2022-03-29T22:37:23Z
on 64-bit linux/posix: // file1.c void other(int); long return_arg1(long x) { return x; } int main() { return_arg1(-1); // put 0xff in AL other(0); return 0; } // file2.c void other(x) int x; { // never reached } compile using "dmd file1.c file2.c", run to observe segfault it crashes because of the variadic function prologue in other(): https://en.wikipedia.org/wiki/X86_calling_conventions#System_V_AMD64_ABI > If the callee is a variadic function, then the number of floating point arguments passed to the function in vector registers must be provided by the caller in the AL register. it's called through a non-variadic prototype and the body is in a different file so dmd doesn't know AL has to be cleared first the segfault is because the variadic code in other() does a jump depending on the value of AL to only save the used registers, which fails if AL contains a garbage value other than 0-8 the zlib library has function bodies in K&R style and assumes that this works if __STDC_VERSION__ is defined (function prototypes in headers will contain the parameters instead of () in that case). the zlib bindings in phobos also don't use variadic functions so they wouldn't work with a dmd-compiled zlib because of this
Comment #1 by bugzilla — 2022-03-30T08:46:30Z
I don't really understand this. int printf(); is variadic. All K+R functions are variadic.
Comment #2 by ibuclaw — 2022-03-30T09:05:32Z
(In reply to Walter Bright from comment #1) > I don't really understand this. > > int printf(); > > is variadic. All K+R functions are variadic. I guess the difference is whether the body of `other` has code generated as-if it is variadic. Yes, variadic in the sense of "we accept any number of arguments to be called to us". But not variadic in that those extraneous arguments are ever read by the callee.
Comment #3 by ibuclaw — 2022-03-30T09:14:31Z
DMD's undoing here is that it set's up a va_list in the prologue, but k&r functions are pseudo-variadic, so it can get away without having va_list. In fact, it's an error to construct a va_list object in a k&r function in gcc: --- void other(x) int x; { __builtin_va_list argp; __builtin_va_start(argp, x); } // file2.c: In function ‘other’: // file2.c:5:3: error: ‘va_start’ used in function with fixed arguments ---
Comment #4 by duser — 2022-03-30T15:05:46Z
i thought this might be hard to explain the issue is that all K&R functions have this in the body: % dmd -c -vasm file2.c other: 0000: 55 push RBP 0001: 48 8B EC mov RBP,RSP 0004: 48 81 EC D0 00 00 00 sub RSP,0D0h 000b: 48 89 B5 38 FF FF FF mov -0C8h[RBP],RSI 0012: 48 89 95 40 FF FF FF mov -0C0h[RBP],RDX 0019: 48 89 8D 48 FF FF FF mov -0B8h[RBP],RCX 0020: 4C 89 85 50 FF FF FF mov -0B0h[RBP],R8 0027: 4C 89 8D 58 FF FF FF mov -0A8h[RBP],R9 002e: 0F B6 C0 movzx EAX,AL 0031: C1 E0 02 shl EAX,2 0034: 4C 8D 1D 2A 00 00 00 lea R11,[02Ah][RIP] 003b: 49 29 C3 sub R11,RAX 003e: 48 8D 45 DF lea RAX,-021h[RBP] 0042: 41 FF E3 jmp R11D 0045: 0F 29 78 F1 movaps -0Fh[RAX],XMM7 0049: 0F 29 70 E1 movaps -01Fh[RAX],XMM6 004d: 0F 29 68 D1 movaps -02Fh[RAX],XMM5 0051: 0F 29 60 C1 movaps -03Fh[RAX],XMM4 0055: 0F 29 58 B1 movaps -04Fh[RAX],XMM3 0059: 0F 29 50 A1 movaps -05Fh[RAX],XMM2 005d: 0F 29 48 91 movaps -06Fh[RAX],XMM1 0061: 0F 29 40 81 movaps -07Fh[RAX],XMM0 0065: C7 40 01 08 00 00 00 mov dword ptr 1[RAX],8 006c: C7 40 05 30 00 00 00 mov dword ptr 5[RAX],030h 0073: 4C 8D 5D 10 lea R11,010h[RBP] 0077: 4C 89 58 09 mov 9[RAX],R11 007b: 48 2D AF 00 00 00 sub EAX,0AFh 0081: 48 89 80 C0 00 00 00 mov 0C0h[RAX],RAX 0088: C9 leave 0089: C3 ret the jump there depends on the value of AL and will malfunction if it has a value other than 0-8 K&R functions are interchangeable with normal ones in gcc and clang, but not in dmd because of the dependence on AL/RAX being set to a valid value before calling (as is done when calling a variadic function) some libraries like zlib (and its bindings in phobos) depend on K&R functions being callable the same way as normal ones gcc/clang don't emit the code to save registers in K&R functions, and if va_start/etc can't be used inside one then there's no reason to
Comment #5 by bugzilla — 2023-04-14T06:46:10Z
Consider: -------------- int abc(); int def(const char *, ...); int ghi(const char *, int i); int main() { abc("hello world %d\n", 1); def("hello world %d\n", 2); ghi("hello world %d\n", 3); return 0; } ------------- Compiled with gcc: ------------- main: push RBP mov RBP,RSP mov ESI,1 mov EDI,offset FLAT:.rodata@32 mov EAX,0 call abc@PC32 mov ESI,2 mov EDI,offset FLAT:.rodata@32 mov EAX,0 call def@PC32 mov ESI,3 // look ma, no mov EAX,0 !! mov EDI,offset FLAT:.rodata@32 call ghi@PC32 mov EAX,0 pop RBP ret -------- The code generated for calls to abc() and def() is the same, it is ghi() that is different.
Comment #6 by dlang-bot — 2023-04-15T07:51:29Z
@WalterBright created dlang/dmd pull request #15107 "fix Issue 22960 - importC: K&R-style functions assume variadic callin…" fixing this issue: - fix Issue 22960 - importC: K&R-style functions assume variadic calling convention https://github.com/dlang/dmd/pull/15107
Comment #7 by dlang-bot — 2023-04-19T07:51:39Z
dlang/dmd pull request #15107 "fix Issue 22960 - importC: K&R-style functions assume variadic callin…" was merged into master: - cf3fe2d4c73541332cea045509c89e47532a9cb3 by Walter Bright: fix Issue 22960 - importC: K&R-style functions assume variadic calling convention https://github.com/dlang/dmd/pull/15107