Bug 3409 – stdio.File.seek() doesn't work for files >2GB
Status
RESOLVED
Resolution
FIXED
Severity
major
Priority
P2
Component
phobos
Product
D
Version
D2
Platform
Other
OS
Windows
Creation time
2009-10-16T23:01:00Z
Last change time
2016-04-13T22:03:51Z
Assigned to
nobody
Creator
dsimcha
Comments
Comment #0 by dsimcha — 2009-10-16T23:01:40Z
stdio.File.seek() doesn't work for large files. This is because fseek() is using an int to represent the offset because it's crufty old C code from the Stone Age. Instead, it throws a ConvOverflowError exception on trying to seek a long distance (> int.max). The proper fix would be to use the C function that takes a long instead, but this is apparently not available on all platforms and not part of the C standard.
I've attached a file with an implementation of a kludge to work around this issue by seeking incrementally. It's ugly but it works. It's implemented in terms of the current seek() implementation, and I haven't integrated it into the File struct, but the basic code is there.
Comment #1 by dsimcha — 2009-10-16T23:18:26Z
Oops, looks like I forgot to attach the attachment. That's ok, on further testing, it only solved the problem up to 4GB anyhow. I guess what *really* needs to happen is LFS support.
Comment #2 by dsimcha — 2009-10-17T08:45:01Z
Note that std.stream gets this right on Win32 but not Linux by using the Win32 API directly.
Comment #3 by andrei — 2009-10-17T09:00:27Z
The bug is in druntime. It defines fseek and ftell in core.stdc.stdio like this:
int fseek(FILE* stream, long offset, int whence);
c_long ftell(FILE* stream);
And it defines c_long in core.stdc.config like this:
version( Windows )
{
alias int c_long;
alias uint c_ulong;
}
else
{
static if( (void*).sizeof > int.sizeof )
{
alias long c_long;
alias ulong c_ulong;
}
else
{
alias int c_long;
alias uint c_ulong;
}
}
I don't know why under Windows c_long is actually 32 bit.
Then, in my code I had this note:
void seek(long offset, int origin = SEEK_SET)
{
enforce(p && p.handle,
"Attempting to seek() in an unopened file");
// @@@ Dubious: why is fseek in std.c.stdio taking an int???
errnoEnforce(core.stdc.stdio.fseek(
p.handle, to!int(offset), origin) == 0,
"Could not seek in file `"~p.name~"'");
}
So it seeme like there was something fishy going on.
I emailed Sean about the problem. He needs to fix druntime, then I will fix stdio, and then we can close this.
Comment #4 by dsimcha — 2009-10-17T09:41:44Z
Could be wrong, but I think the way druntime defines these functions is because that's the way they're defined in the C std lib API. I think to do better you need to use OS-specific APIs directly.
Comment #5 by dfj1esp02 — 2009-10-19T02:08:08Z
> I don't know why under Windows c_long is actually 32 bit.
because msvc compiler is LLP64.
Comment #6 by dfj1esp02 — 2009-10-19T02:13:29Z
(In reply to comment #4)
> Could be wrong, but I think the way druntime defines these functions is because
> that's the way they're defined in the C std lib API.
Yes, but that's *C* API, so the type should be c_long.
Comment #7 by dfj1esp02 — 2009-10-19T02:20:19Z
Strange, in 2.020 it was
int fseek(FILE* stream, c_long offset, int whence);
so this is regression.