Bug 21131 – Appender with string does not process UTF input ranges properly
Status
RESOLVED
Resolution
FIXED
Severity
enhancement
Priority
P1
Component
phobos
Product
D
Version
D2
Platform
All
OS
All
Creation time
2020-08-07T00:37:56Z
Last change time
2020-08-07T11:03:32Z
Keywords
pull
Assigned to
No Owner
Creator
Steven Schveighoffer
Comments
Comment #0 by schveiguy — 2020-08-07T00:37:56Z
When putting ranges of code units into an appender of a different width string, Appender uses integer promotion instead of decoding and reencoding to append to the string.
Example:
auto str = "ウェブサイト";
auto wstr = appender!wstring();
put(wstr, str.byCodeUnit);
writeln(wstr.data);
This outputs:
ã¦ã§ããµã¤ã
The fix is to detect this unique situation and decode the data into dchars.
Comment #1 by dlang-bot — 2020-08-07T00:41:12Z
@schveiguy created dlang/phobos pull request #7583 "Fix 21131 -- proper handling of non-dchar appending to Appender!wstring and Appender!dstring" fixing this issue:
- Fix 21131. Detect certain string appending situations and properly
handle them.
https://github.com/dlang/phobos/pull/7583
Comment #2 by dlang-bot — 2020-08-07T11:03:32Z
dlang/phobos pull request #7583 "Fix 21131 -- proper handling of non-dchar appending to Appender!wstring and Appender!dstring" was merged into master:
- 9c64fd76ffc2c78981a48d6f9201f52bfbf23760 by Steven Schveighoffer:
Fix 21131. Detect certain string appending situations and properly
handle them.
https://github.com/dlang/phobos/pull/7583