Bug 5591 – EBX register not preserved when calling stdcall function pointer
Status
RESOLVED
Resolution
INVALID
Severity
critical
Priority
P2
Component
dmd
Product
D
Version
D2
Platform
x86
OS
Windows
Creation time
2011-02-15T03:58:00Z
Last change time
2011-02-17T14:08:30Z
Keywords
wrong-code
Assigned to
nobody
Creator
hypothermia.frost
Comments
Comment #0 by hypothermia.frost — 2011-02-15T03:58:17Z
According to D's calling convention EBX register is preserved across function calls, but when when I call a pointer to an extern(Windows) function, EBX gets filled with some garbage value. It only happens the first time this function is called, and the following times it behaves as expected!
The following sample code can be run to show it:
extern(Windows) void foo(int i){
size_t ebx;
asm { mov ebx,EBX; }
std.stdio.writefln(" foo (EBX = %s,%s)",ebx,i);
asm { mov EBX,10; }
}
alias extern(Windows) void function(int) fooT;
fooT bar;
void main(){
size_t ebx=0;
bar=&foo;
asm { mov EBX,1; } //try normal call
foo(2);
asm { mov ebx,EBX; }
std.stdio.writefln("EBX(after foo) = %x",ebx);
asm { mov EBX,3; } // now try pointer call for the 1st time
bar(4);
asm { mov ebx,EBX; }
std.stdio.writefln("EBX(after bar) = %x",ebx);
asm { mov EBX,5; } //2nd time
bar(6);
asm { mov ebx,EBX; }
std.stdio.writefln("EBX(after bar 2nd time round) = %x",ebx);
main2();
}
void main2(){ //and 3rd time!
size_t ebx=0;
asm { mov EBX,7; }
bar(8);
asm { mov ebx,EBX; }
std.stdio.writefln("EBX(after bar 3rd time round in other func) = %x",ebx);
}
output:
foo (EBX = 1,2)
EBX(after foo) = 1
foo (EBX = 5518328,4)
EBX(after bar) = 5433f8
foo (EBX = 5,6)
EBX(after bar 2nd time round) = 5
foo (EBX = 7,8)
EBX(after bar 3rd time round in other func) = 7
What's going on with EBX on the 1st call?
Comment #1 by hypothermia.frost — 2011-02-15T09:23:06Z
After doing some more tests I found out that if you compile with -O switch, the code behaves as expected and output is:
foo (EBX = 1,2)
EBX(after foo) = 1
foo (EBX = 3,4)
EBX(after bar) = 3
...
Looks like it's a bug in DMD code gen without the -O switch.
Comment #2 by hypothermia.frost — 2011-02-16T12:10:39Z
After disassembly I found out that this code was generated(no -O switch):
//asm { mov EBX,3; } bar(4);
mov ebx, 3
push 4
mov ebx, large fs:2Ch
mov esi, [ebx]
call dword ptr [esi+4F4h]
mov [ebp+var_20], ebx
//the next round ecx is used...
//asm { mov EBX,5; } bar(6);
mov ebx, 5
push 6
mov ecx, large fs:2Ch
mov edx, [ecx]
call dword ptr [edx+4F4h]
mov [ebp+var_20], ebx
Why is DMD not preserving the EBX register in the first call? The worst thing is that the compiler doesn't even know tha there's something in EBX... This problem happened to me with when EBX had a this pointer and then DMD didn't save it and after the function pointer call I got an exception!
I think this is a serious issue and should be looked at ASAP.
Comment #3 by clugdbug — 2011-02-16T12:22:07Z
Top priority for this one.
Comment #4 by bugzilla — 2011-02-17T13:23:59Z
This is a misunderstanding of the calling conventions. Examination of the asm produced by the compiler shows that, indeed, EBX is preserved across all three functions.
However, the compiler is not obliged to preserve the contents of EBX from statement to statement within a function, which is what your code is expecting.
This is not a bug.
Comment #5 by hypothermia.frost — 2011-02-17T13:53:00Z
Ok, but as I said earlier this bug happened to me when the pointer call happened in a member function call. The EBX register contained a 'this' pointer and the compiler used it to load the function pointer. But after calling this function pointer the compiler still thought EBX contained 'this' and I got an Access Violation exception thrown when I tried to access a field from 'this'. I saw this all using debugger and I traced the EBX change to the call of the function pointer. And then how is this not a bug If the compiler thinks that after using the EBX register, It has a value from previous use? Maybe Its hard for you to understand what I'm trying to explain, but I will try to make an example code to prove my point.
Comment #6 by bugzilla — 2011-02-17T14:08:30Z
If you can post a complete code snippet, and the obj2asm output of the asm produced, that shows that EBX is not preserved across function calls, I can fix it.