Bug 14381 – It is too difficult to contribute to the auto-tester

There is a number of things the D autotester could be improved to do: - Provide nightly build downloads - Perform coverage analysis and provide a visualization of which lines are covered - Render documentation and provide preview links (or rendered diffs, like on GitHub) - Measure memory usage, build times, object/executable file sizes, and trends/regressions thereof - Provide an API which programs such as Digger can consume (Currently Digger is forced to use a caching proxy over the GitHub API[1], as the latter is rate-limited). Unfortunately, it is currently very difficult (if not impossible) to contribute to the auto-tester. The current auto-tester implementation uses proprietary components and makes assumptions about its environment[2], which make it impossible to test any improvements. Additionally, the auto-tester's creator and maintainer (Brad Roberts) is not always available to review and integrate improvements, which cause improvements in other areas of D to be blocked, sometimes for very long periods of time[3][4]. I think we should prioritize making it possible for more people to maintain, contribute to, and host additional (backup or test) instances of the auto-tester. This may include: - Replacing the proprietary parts (this may require a clean-room implementation by someone other than Brad) - Improving the project's documentation - Moving the project's code under the D-Programming-Language GitHub repository - Granting access to several key D personnel to the existing auto-tester infrastructure [1]: https://github.com/CyberShadow/GHDaemon [2]: https://github.com/braddr/d-tester/blob/master/README [3]: http://forum.dlang.org/post/[email protected] [4]: https://github.com/D-Programming-Language/druntime/pull/960

My major comment is to stop thinking of the auto-tester as the D auto-tester. Think of it as a tool that the D community uses that is owned and operated by me. I agree that it's currently and for the past year poorly owned. But it's not specific to D and I don't intend to add capabilities to it that are specific to D or the D development process. Some of the feature work you suggest below can fit within either the build or the test target without requiring changes to the auto-tester. Some of it is general enough to work for more than D.

(In reply to Brad Roberts from comment #1) > My major comment is to stop thinking of the auto-tester as the D > auto-tester. Think of it as a tool that the D community uses that is owned > and operated by me. I agree that it's currently and for the past year > poorly owned. But it's not specific to D and I don't intend to add > capabilities to it that are specific to D or the D development process. > > Some of the feature work you suggest below can fit within either the build > or the test target without requiring changes to the auto-tester. > > Some of it is general enough to work for more than D. So what is the action item here, Brad? Do you plan to open things up or do we need to duplicate the effort? I said this and I maintain: we need to move away the setup in which only one person is the bottleneck for everything about D vitals. Please let us know if and when you are willing to open the system. Thanks.

Brad Roberts has done a great work with his platform, but have we ever considered to move to a standardized, complete and easily extensible solution such as Jenkins ? At this point, IMHO the option should be considered.

(In reply to Andrei Alexandrescu from comment #2) > (In reply to Brad Roberts from comment #1) > > My major comment is to stop thinking of the auto-tester as the D > > auto-tester. Think of it as a tool that the D community uses that is owned > > and operated by me. I agree that it's currently and for the past year > > poorly owned. But it's not specific to D and I don't intend to add > > capabilities to it that are specific to D or the D development process. > > > > Some of the feature work you suggest below can fit within either the build > > or the test target without requiring changes to the auto-tester. > > > > Some of it is general enough to work for more than D. > > So what is the action item here, Brad? Do you plan to open things up Depends on what you mean by open things up? I've always accepted pull requests, though most have been reworked before being merged as they're typically not workable as is. Do I plan on allowing anyone other than me to have merge / pull permissions? Not at this time. Do I plan on allowing anyone other than me to have login access to the client build machines? Not blanketly no. If there's some extraordinary need for debugging purposes, maybe. That said, there's almost always going to be a better way to do that.. like a VM for instance. > or do we need to duplicate the effort? I hope not, it'd be a pretty large duplication of effort. > I said this and I maintain: we need to move away the setup in which only one > person is the bottleneck for everything about D vitals. That's a huge exaggeration over the current situation. Me for the auto-tester is far from a bottleneck for everything about D vitals. I'm a bottleneck for one key tool, certainly. > Please let us know if and when you are willing to open the system. Thanks. I'm open to discussion and feature requests. I'm open to accepting contributions, but only in directions that I believe are right for the tool. I'm going to say no to things that I don't believe are a good fit.

Let's take a concrete example. I'm trying to get dlang.org/documentation building integrated into or CI since 4 month, because it breaks too often and regularly stalls releases. https://github.com/braddr/d-tester/issues/41 I tried various ways (email, newsgroup, github) to get a bit of your attention on that topic. I also tried to implement it myself, but couldn't get the client script to run on my machine. This is asking for 10-20 minutes of your time for an important feature, it is mainly a communication issue though.

We could move the build/test scripts into the DMD/druntime/phobos repos, that would help with things such as this. https://github.com/D-Programming-Language/druntime/pull/1206

Ldc now uses buildbot to handle their ci. https://forum.dlang.org/post/[email protected] Basically every user can contribute and run his own client. Seems to be a mature project with 2k stars on github.

*** Issue 16608 has been marked as a duplicate of this issue. ***

The Auto-tester itself has been taken down.