Bug 18692 – assignment of std.regex.Captures reads freed memory from 2.072.0 to 2.078.3 inclusive

When I was working on: Issue 18691: assigning a std.regex.Captures with 3 or more groups causes double free ... I couldn't understand how this change: https://github.com/dlang/phobos/commit/872673d5570f1ee79df4b9e47d8f3d2cf4e49536 ... the "Use ref-counting for Captures struct" from @DmitryOlshansky, could be right. How can he get away with having the _refcount that applies to big_matches not stored in an object with the same lifetime as big_matches? I have about two days' familiarity with D, so I assume I'm missing something. That said, here's a program that goes wrong with the first release of dmd that included that change. First, it working on the preceding version. martind@swiftboat:~/tmp/D134366$ cat two_years.d import std.regex; void main() { auto rx = regex("()()()"); auto ma = "".matchFirst(rx); auto ma2 = ma; ma = ma2; ma[1]; } martind@swiftboat:~/tmp/D134366$ dmd --version DMD64 D Compiler v2.071.0 Copyright (c) 1999-2015 by Digital Mars written by Walter Bright martind@swiftboat:~/tmp/D134366$ dmd two_years.d && MALLOC_PERTURB_=1 ./two_years martind@swiftboat:~/tmp/D134366$ 2.072.0 was afflicted by https://digitalmars.com/d/archives/digitalmars/D/dmd_or_phobos_were_broken_in_ubuntu_16.10_d-apt_294148.html, hence the extra shared library parameters, which make no difference on 2.071.0. martind@balance:~/tmp/D134366$ dmd --version DMD64 D Compiler v2.072.0 Copyright (c) 1999-2016 by Digital Mars written by Walter Bright martind@balance:~/tmp/D134366$ dmd -defaultlib=libphobos2.so -fPIC two_years.d && MALLOC_PERTURB_=1 ./two_years core.exception.RangeError@std/regex/package.d(565): Range violation ---------------- ??:? _d_arraybounds [0xf693d091] ??:? std.regex.__array [0xf68f6f1e] ??:? std.regex.Captures!(immutable(char)[], ulong).Captures.opIndex!().opIndexinout(pure nothrow @trusted inout(immutable(char)[]) function(ulong)) [0xb4f37081] ??:? _Dmain [0xb4f0f60f] ??:? _D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv [0xf696d68f] ??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0xf696d5bb] ??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() [0xf696d634] ??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0xf696d5bb] ??:? _d_run_main [0xf696d51f] ??:? main [0xb4f3a67f] ??:? __libc_start_main [0xf55f52b0] martind@balance:~/tmp/D134366$ valgrind is unhappy too and more verbosely, making it explicit that the problem is reading from freed memory. The problems continue up to and including 2.078.3 but disappear for 2.079.0 and are absent in master. I suspect that the @MartinNowak change I finger in Issue 18691 fixed this particular reference counting issue at the expense of introducing that one.

> ... the "Use ref-counting for Captures struct" from @DmitryOlshansky, could be right. How can he get away with having the _refcount that applies to big_matches not stored in an object with the same lifetime as big_matches? I have about two days' familiarity with D, so I assume I'm missing something. That said, here's a program that goes wrong with the first release of dmd that included that change. First, it working on the preceding version. Indeed looking at it now, it doesn't seem right. Awfully so. It likely passed though because captures is typically a temporary. Instead it should allocate and equivalent of struct Payload { uint refcount; Group[0] matches; } Thanks for reporting!

*** This issue has been marked as a duplicate of issue 18691 ***