Bug 18452 – std.zip has size limit of 2 GB

Status
RESOLVED
Resolution
FIXED
Severity
normal
Priority
P1
Component
phobos
Product
D
Version
D2
Platform
All
OS
All
Creation time
2018-02-16T19:51:28Z
Last change time
2018-04-03T09:35:17Z
Assigned to
Andrei Vasile
Creator
Andre

Comments

Comment #0 by andre — 2018-02-16T19:51:28Z
I just noticed that std.zip will throw an exception if the source files exceeds 2 GB. It seems an issue with std.zip as zlib has a limit of 4 GB. Windows 10 with x86_64 architecture. core.exception.RangeError@std\zip.d(808): Range violation ---------------- 0x00007FF7C9B1705C in d_arrayboundsp 0x00007FF7C9B301FF in @safe void std.zip.ZipArchive.putUshort(int, ushort) 0x00007FF7C9B2E634 in void[] std.zip.ZipArchive.build() void zipFolder(string archiveFilePath, string folderPath) { import std.zip, std.file; ZipArchive zip = new ZipArchive(); string folderName = folderPath.baseName; foreach(entry; dirEntries(folderPath, SpanMode.depth)) { if (!entry.isFile) continue; ArchiveMember am = new ArchiveMember(); am.name = entry.name[folderPath.length + 1..$]; am.expandedData(cast(ubyte[]) read(entry.name)); zip.addMember(am); } void[] compressed_data = zip.build(); // zip.build() will throw write(archiveFilePath, compressed_data); } Comment from Steven Schveighoffer: ...And you are right. I looked into it a bit, this has nothing to do (superficially) with zlib, it has to do with std.zip: https://github.com/dlang/phobos/blob/0107a6ee09072bda9e486a12caa148dc7af7bb08/std/zip.d#L806 Really, i should be size_t in all places, I can't see why it should ever be int. -- And Stefan Koch: ...It was partially changed in this PR: https://github.com/dlang/phobos/pull/2914/files The the put methods where left at int must have been an oversight. -- https://forum.dlang.org/post/[email protected]
Comment #1 by github-bugzilla — 2018-04-03T09:35:16Z
Commits pushed to master at https://github.com/dlang/phobos https://github.com/dlang/phobos/commit/0dc7ea22b08b317e4e5bda07d298996ecb9efa0f Fix Issue 18452 - std.zip has size limit of 2 GB https://github.com/dlang/phobos/commit/861e65ac5f590dd8bc12c99d07492728573a5d91 Merge pull request #6378 from johnsilver97/Issue_18452 Fix Issue 18452 - std.zip has size limit of 2 GB