Bug 18103 – test17868 failing often but not every time

Not sure when it started exactly, but for a while test17868 in the dmd test suite has been failing. Seems to be limited to osx, https://auto-tester.puremagic.com/platform-history.ghtml?projectid=1&os=Darwin_64_64 https://auto-tester.puremagic.com/platform-history.ghtml?projectid=1&os=Darwin_64_32 It's not tied to a single host, all three hosts seem to have a mix of passing and failing.

Looks like its a race condition. The failure line says this: diff: generated/runnable/test17868.d.out.2: No such file or directory Note that the postscript does something with the output, copies it to a ".out.2" and then diffs it against the expected output. But I see no "release" or "debug" in that path. The appropriate line is this: diff --strip-trailing-cr runnable/extra-files/test17868.d.out ${RESULTS_DIR}/runnable/test17868.d.out.2 So clearly, RESULTS_DIR just contains "generated". That's not enough to avoid races. I actually see a lot of cases in the postscript files that are subject to possible races. CCing Martin, as he seems to be the one that set up these tests.

That test doesn't use a POST_SCRIPT anymore.