Bug 14427 – Regression: navigation for phobos documentation has disappeared

http://dlang.org/phobos/index.html Note the navigation list of all phobos modules in the left panel has disappeared. This is a critical bug.

The following sequence of events has happened: * After Vladimir got an account, he uploaded the site as user "vladimir". * I upload files as user "d-programming". * There is a mismatch in groups and permissions of these two usernames that makes it impossible for me to upload the site. I get errors and freezes in rsync. * This problem has been known for a while, and Martin tried to help it: https://github.com/D-Programming-Language/dlang.org/pull/959 * This afternoon I tried to update the site with the rsync command as fixed by Martin. Things still didn't work. * Assuming at least I didn't upload some corrupted files, I gave up and sent emails to Martin, Vladimir, and Sönke asking for help. Looking into it further I think the issue here is d-programming is not a member of the group dlang.org. I emailed about it to Jan. The core issue here is twofold: (a) we have no centralized web czar so everybody believes a failure is someone else's problem; (b) our provider (Jan) is external so we depend on him being around to get stuff done. In the meantime I manually synchronized /phobos/. Will keep this open until the rsync issue gets fixed.

I think we need a staging area of sorts - a controlled environment which pulls strictly the website from git, performs a clean build, allows previewing, and only then uploads it to dlang.org. This would solve a few issues: - Removes requirement to install software to fully build dlang.org (e.g. kindlegen) - Removes requirement to use a Posix OS (completely building the website is currently impossible on Windows) - Removes issues due to environment (wrong/incompatible software versions, e.g. rsync) - Removes bottlenecks on Jan to resolve permission issues - Removes upload bandwidth bottleneck for slow connections - Fixes users having to upload the entire site because timestamps don't match between users' local copies I have something like this on dlang.k3.1azy.net (same server which hosts forum.dlang.org). Andrei has access already. The workflow is: 1. If there have been some significant changes in the build infrastructure which might cause a dirty build, run: `ssh [email protected] run/clean`. This will delete all intermediary files and force clean all repositories (incl. dub stuff). 2. Run `ssh [email protected] run/update` 3. Browse to http://dlang.k3.1azy.net/ and make sure there are no glaring problems. 4. Run `ssh [email protected] run/upload`

The rsync issue has been resolved by now (was a permission problem). Can we close this?

yes thx

@Vladimir that's pretty rad! I think what would work best is a "view deltas" capability, i.e. detect and make available for previewing the pages that have been changed since the last upload. That way instead of browsing the entire site, one needs only to glance at the pages that actually changed. Of course, at times the changed list will be bulky - every time the navigation menu changes, a bunch of html pages change as well. But other times only one or a few pages get updated, and it's nice to be able to distinguish those. In the future it would be even nicer to have an online diff between the live site html and the preview html. Any chance of that being reasonably easy to implement?

Yeah, that shouldn't be difficult to put together. Just cache uploaded files to a local copy and add a diff Makefile target that diffs built vs. last-uploaded files. I'll add it to my list but not sure when I'll get to this.

The pinnacle of your idea is a GitHub bot that pastes diff links on each dlang.org pull request, BTW.

(In reply to Vladimir Panteleev from comment #7) > The pinnacle of your idea is a GitHub bot that pastes diff links on each > dlang.org pull request, BTW. Well it's diffs between built files, not sources.

That's what I meant.

@Cybershadow: before I archive this, you may want to document this as a tool if you want this to become our standard way of changing the website. Otherwise it will be forgotten.

Document it where? It's still an internal tool to which a few people need to have access.

E.g. the wiki. This page is definitely not the place :o).

(In reply to Vladimir Panteleev from comment #13) > http://wiki.dlang.org/User:Vladimir_Panteleev/Website_staging Well I'll be. It worked. Thanks for a nice tool!