Bug 11017 – std.string/uni.toLower is very slow

char[] s = new char[10_000_000]; s[] = 'A'; auto s2 = s.toLower; This takes 4.3 seconds on my machine. char[] s = new char[10_000_000]; s[] = 'A'; auto s2 = s.map!toLower.to!string; This only takes 1.1 seconds. Looking at the code for std.uni.toLower, it appears the string is constructed using repeated ~=. It should use an Appender of some sort.

(In reply to comment #0) > char[] s = new char[10_000_000]; > s[] = 'A'; > auto s2 = s.toLower; > > This takes 4.3 seconds on my machine. > > > char[] s = new char[10_000_000]; > s[] = 'A'; > auto s2 = s.map!toLower.to!string; > > This only takes 1.1 seconds. > There 2 things here to consider - first the 2nd one is not correct in general (1 codepoint can map to many e.g. german sharp S). > Looking at the code for std.uni.toLower, it appears the string is constructed > using repeated ~=. It should use an Appender of some sort. This indeed could be fixed I do suspect put an optimisitc reserve(original.length) there would work even better. See also issue 10864: http://d.puremagic.com/issues/show_bug.cgi?id=10864

(In reply to comment #1) > There 2 things here to consider - first the 2nd one is not correct in general > (1 codepoint can map to many e.g. german sharp S). Good point, although std.uni.toUpper doesn't handle it either :-) assert("ß".toUpper == "ß"); // passes

(In reply to comment #2) > (In reply to comment #1) > > There 2 things here to consider - first the 2nd one is not correct in general > > (1 codepoint can map to many e.g. german sharp S). > > Good point, although std.uni.toUpper doesn't handle it either :-) > > assert("ß".toUpper == "ß"); // passes To Lower will do. Sharp S is capital ;)

(In reply to comment #3) > To Lower will do. Sharp S is capital ;) assert("ß".toLower == "ß"); assert("ß".toUpper == "ß"); Both pass.

(In reply to comment #4) > (In reply to comment #3) > > To Lower will do. Sharp S is capital ;) > > assert("ß".toLower == "ß"); > assert("ß".toUpper == "ß"); > > Both pass. Something wicked have happend. I see that I've messed up toUpper in table generator while introducing toTitleCase (that isn't even yet exposed!). toLower is fine, toUpper is broken in half of cases apparently. How I missed that I've no idea ... gotta expand the test coverage around toLower/toUpper.

(In reply to comment #5) > (In reply to comment #4) > > (In reply to comment #3) > > > To Lower will do. Sharp S is capital ;) > > > > assert("ß".toLower == "ß"); > > assert("ß".toUpper == "ß"); > > > > Both pass. > > Something wicked have happend. > I see that I've messed up toUpper in table generator while introducing > toTitleCase (that isn't even yet exposed!). toLower is fine, toUpper is broken > in half of cases apparently. > How I missed that I've no idea ... gotta expand the test coverage around > toLower/toUpper. P.S. And there are both kinds of sharp s ... \u1E9E and \u00df