Bug 12853 – std.encoding EncodingSchemeUtf16Native and EncodingSchemeUtf32Native decode() and SafeDecode() wrong stripping length

Created attachment 1360 bugfix, public domain std.encoding EncodingSchemeUtf16Native and EncodingSchemeUtf32Native decode() and SafeDecode() are wrong at stripping decoded symbols. they should take wchar/dchar sizes into account when stripping decoded bytes from input buffer. fix attached.

test case: import std.encoding; void testUTF16 () { version(LittleEndian) { auto efrom = EncodingScheme.create("utf-16le"); ubyte[6] sample = [154,1, 155,1, 156,1]; } version(BigEndian) { auto efrom = EncodingScheme.create("utf-16be"); ubyte[6] sample = [1,154, 1,155, 1,156]; } const(ubyte)[] ub = cast(const(ubyte)[])sample; dchar dc = efrom.safeDecode(ub); assert(dc == 410); assert(ub.length == 4); } void testUTF32 () { version(LittleEndian) { auto efrom = EncodingScheme.create("utf-32le"); ubyte[12] sample = [154,1,0,0, 155,1,0,0, 156,1,0,0]; } version(BigEndian) { auto efrom = EncodingScheme.create("utf-32be"); ubyte[12] sample = [0,0,1,154, 0,0,1,155, 0,0,1,156]; } const(ubyte)[] ub = cast(const(ubyte)[])sample; dchar dc = efrom.safeDecode(ub); assert(dc == 410); assert(ub.length == 8); } void main () { testUTF16(); testUTF32(); }

Could create a pull request? Pull requests help in reviewing and receiving feedback as well as verification of the patch. Ultimately your issue will be resolve quicker.

sorry, i have no github account and not planning to get one in nearest millenia. if this bugzilla is "second place citizen", it should be removed altogether i believe.

Commits pushed to master at https://github.com/D-Programming-Language/phobos https://github.com/D-Programming-Language/phobos/commit/6d3f59fa7b5b4b4eb43ff2d4cbe56905077a6ec6 fix Issue 12853 - std.encoding EncodingSchemeUtf16Native and EncodingSchemeUtf32Native decode() and SafeDecode() wrong stripping length https://github.com/D-Programming-Language/phobos/commit/a6c5488910d7746aada8bc378e8848b39f5e8aaf fix Issue 12853 - std.encoding EncodingSchemeUtf16Native and EncodingSchemeUtf32Native decode() and SafeDecode() wrong stripping length https://github.com/D-Programming-Language/phobos/commit/1f398230b1791e771486cf5489e895c14d77134d Merge pull request #2336 from AndrewEdwards/master fix Issue 12853 - std.encoding EncodingSchemeUtf16Native and EncodingSch...