Bug 16987 – ABI error wrt. COM interfaces returning structs

Status
RESOLVED
Resolution
FIXED
Severity
major
Priority
P1
Component
dmd
Product
D
Version
D2
Platform
All
OS
Windows
Creation time
2016-12-19T23:07:14Z
Last change time
2018-06-25T18:44:50Z
Assigned to
No Owner
Creator
kinke

Comments

Comment #0 by kinke — 2016-12-19T23:07:14Z
From http://forum.dlang.org/post/[email protected], relevant wrt. interfacing with DirectX libraries: COM classes/interfaces are apparently subject to a separate ABI. Unfortunately, googling it hasn't turned up any official (and not even some inofficial) documentation so far. Based on the first few tests on Win64, integers are returned in RAX, floats (and I guess doubles too) in XMM0, and structs (incl. 2x int32 and 2x float) via hidden sret pointer, with `this` pointer in RCX (1st arg) and `sret` in RDX (2nd arg). Compared to the normal Win64 C++ ABI it just seems more conservative by always returning structs via hidden pointer. While at it, please also fix https://issues.dlang.org/show_bug.cgi?id=16527 by simply swapping the order of `this` and `sret` pointer arguments on Windows (32-bit: extern(C++) methods only, 64-bit: always; LDC already does so).
Comment #1 by kinke — 2018-06-25T18:44:50Z
This was fixed by https://github.com/dlang/dmd/pull/8330, for Win32 and Win64, and apparently wasn't COM-specific, but a Visual C++ ABI quirk.