Le weblog entièrement nu

This is getting old news, and others have blogged about them before I did, but here's my summary of the recent activity in and around FusionForge.

The early February meeting was a success, and gathered about twenty people on the first day and a dozen or so on the second day (not planned initially). My impression is that there was a healthy mix of FusionForge hackers, FusionForge users, and people from other forge communities (Codendi, NovaForge, and even one representative from nFORGE, from South Korea). I'm not going to repeat all that was said then, especially since the proceedings are online. Beyond the technical points, I'll just advertise PlanetForge again, since everyone present agreed we had lots to share and that this site would be a good and relatively neutral place. If you're into forges, I recommend joining us in that community.

On the purely FusionForge front, news are good too. Most of the major pieces we want to see in the next release (which is probably going to be called 5.0) are in place. The last blocker we had was the merge of the rework of the default theme for better accessibility and easier maintenance and customisability (most of the theming now happens in CSS). This merge has been completed this week, and although there are still a few rough edges, it's mostly done. We'll try to fix most of these rough edges soonish, then start a stabilisation branch towards 5.0, so more experimental work can start again on trunk. For the impatient and the curious, there's a list of new features on the fusionforge.org homepage, and the site is now running code from trunk.

Of course, we're eager to get testers for that, which is why I prepared snapshot packages. They are currently stuck in NEW on their way to the official Debian experimental repository due to the renaming of the source package and the introduction of plenty of new binary packages, but they can already be obtained from my unofficial repository at people.debian.org. The packages are built for Debian unstable, but they seem to run just fine on Lenny if you grab mediawiki from backports.org (only required for the Mediawiki plugin, of course), and libnusoap-php and php-htmlpurifier from Debian testing (they don't drag any extra dependencies).

I'll end this note by reminding people of the announcement I did three months ago: as of this week, Debian Etch is no longer officially supported security-wise, and so neither is GForge 4.5. As far as I know, I was the last person doing that, and my incentives have gone away on the day Etch ceased to be supported, since it was also the day the Adullact forge finally migrated from Etch with GForge 4.5 to Lenny with FusionForge 4.8. If you're still using 4.5, well… I think you should be aware of that.

That more or less wraps it up for now. The next announcement is likely to be about a release candidate…

I normally don't relay security announces for GForge or FusionForge on this blog, but I will make an exception this time: Alain Peyrat found several places in the code with insufficient input sanitizing, which can cause cross-site scripting vulnerabilities (CVE-2009-3303). It's been fixed in the 4.7 and 4.8 branches as well as the trunk of FusionForge (and in Debian Sid and Squeeze), and updated Debian packages for GForge 4.5 and 4.7rc2 have been released for users of the Etch and Lenny distributions.

The reason I make an exception for announcing this here is to remind people that I appear to be the only one maintaining code for GForge 4.5. I do that for two reasons: first, because I'm the maintainer of the package in Debian, and Debian Etch has GForge 4.5, and Etch is supported for security fixes; second, because I also admin/maintain an instance for a client of mine, so I need to backport the fixes anyway, and making them public is no bother. Both of these reasons are going to vanish sometime in the not too distant future: security support for Etch will end in February, 2010, and I hope to have migrated my client's forge to FusionForge 4.8 by then too. A direct consequence is that I will probably stop maintenance for GForge 4.5 in the coming months (at least I'll stop doing it in my free time).

So if you're still using GForge 4.5, you should really consider upgrading to something supported, either GForge AS (free download from the GForge Group) or FusionForge (free as in Free Software). Both have an upgrade path. Obviously I think FusionForge is a better choice, but my position is probably biased.

Executive summary

To avoid confusion with the proprietary versions of GForge (known as GForge Advanced Server, GForge Express Edition and GForge Community Edition), the free/libre/opensource codebase will from now on be separately maintained under the name FusionForge by the main developers of the free GForge 4.x codebase. Since this is mostly a renaming, the migration path for existing users will be smooth.

Longer version, with details

After the initial forking from the Sourceforge codebase, the development of GForge has long been hosted, and many enhancements directly developed, by the GForge Group (GForge, LLC), with regular contributions from outsiders. The results of these evolutions were public and free, subject to the GNU GPL.

In parallel, the GForge Group wrote a proprietary re-implementation of GForge, which it sold under the name "GForge Advanced Server", or "GForge AS" for short. This re-implementation added some features for "the enterprise", but was not contributed wholesale to the GForge codebase under a free license. Although some of the features were contributed to the public, the GForge Group concentrated its efforts on its (proprietary software) business model, with more versions appearing, such as "GForge Express Edition" and more recently "GForge Community Edition". As a result, it became increasingly harder for the public to know which version was which without doing extensive research (indeed, some users mistakenly installed one version instead of the other). A consequence was that the free software codebase suffered from a loss in visibility, which lowered its momentum to the point that there haven't been any moderately important releases since the (currently stable) 4.5.x series was announced in late 2005.

So, in order to clarify things, avoid further confusion, and regain some of the lost momentum, it was decided by a group of leading contributors that the free software version of the GForge codebase would from now on be developed under the FusionForge name, and its development would be hosted on FusionForge.org.

So is this a fork?

Well, we don't know yet. It could arguably be called one, since we're taking the code and running away with it under a new name. However, we believe it's not a fork unless both roads continue their own way (more of a oddly-shaped bend). What happens to the GForge codebase developed by the GForge Group at gforge.org remains to be seen, although for the sake of our users we will backport security fixes to the gforge.org Subversion repository (at least for the 4.5.x series and the unreleased 4.6 and 4.7 pre-series) for some time. The bulk of the development will move on to FusionForge and the repositories at FusionForge.org, though, and users are encouraged to migrate at their own pace. Since we're basically continuing the evolution rather than starting from scratch, the migration path should be rather smooth.

So why the FusionForge name?

Because there were actually lots of locally-patched versions of GForge (and Sourceforge), and we felt it was a waste of resources that should be fixed. It seems many people and organisations took these codebases at some point in time and evolved them for their own needs. Sadly, many of the changes were not contributed back or even published, so lots of efforts were duplicated. Fortunately, many of the people managing these locally-patched forges are now realising that "out-of-tree" patches and features require quite some manpower to maintain. Some formal inter-project discussion is already taking place, and we hope to achieve actual merging of most of the interesting features that have been developed here and there into a common base that can be reused locally with minimal changes. We'd like to "un-fork" as much as possible.

We also expect that, by using standard components and tools, we'll facilitate the work of potential contributors, thereby reducing the risk of a new era of fragmentation.

And who are we anyway?

We're Christian Bayle, Roland Mas and Alain Peyrat, long-time contributors to GForge and responsible for over 95% of the commits over the past two years, as well as a few relative newcomers. Christian and Roland have been maintaining the Debian packaging since the "Debian-SF" era, and Alain has been focusing on code quality. The three of us have, for various reasons, a vested interest in maintaining a lively codebase in a healthy ecosystem.

What are our plans?

Our short-term goals, as currently planned, include:

• pushing a stable FusionForge release out of the door;
• cleaning up and auditing the database schema to ensure consistency and increased performance;
• merging in new plugins, in particular for new version control systems;
• continuing streamlining the installation process to make it more portable across distributions.

Longer term goals are less well defined, but we're thinking about the following:

• increasing code quality by the use of modern automated tools;
• encouraging a lively stream of external contributions, to reduce the gap between the official version and the locally-patched ones;
• defining an explicit governance model and release process for the FusionForge project;
• as a consequence, a more frequent and predictable release schedule;
• increasing data portability and interoperability with other forges, to reduce lock-in for users and projects.

Some of these items should be facilitated by our switch to a distributed version control system and a new coordinated workflow. Also, the Debian i18n team has been kind enough to offer to host our translation effort on their Pootle server, which means translators will have a much easier time doing their job.

We hope to hear from users and contributors alike in the near future.

For more information, we can be reached via our fusionforge-general mailing-list (see our lists), which is also suitable for general discussions. We can also be found on IRC (#fusionforge on the Freenode network).

Stuff happens quietly on the GForge front, but after some time we decided we're getting bored with not releasing. Since we seem to have run out of major problems in the codebase, the long-awaited GForge 4.7 release is probably round the corner.

And so, since GForge migrated from its own in-house translation system to the more conventional gettext API, I'd like to take the opportunity to issue a call for translations, knowing that potential translators won't be too disturbed by unusual tools and formats.

You can grab the current state of the translations from the GForge repository browser. Or, for more long-term involvement, checkout the code through Subversion or through Bzr (my gateway branch is available from bzr.debian.org. Current statistics are as follows:

• bg.po: 382 translated messages, 174 fuzzy translations, 1813 untranslated messages.
• ca.po: 1592 translated messages, 281 fuzzy translations, 496 untranslated messages.
• de.po: 1627 translated messages, 272 fuzzy translations, 470 untranslated messages.
• el.po: 0 translated messages, 2369 untranslated messages.
• eo.po: 0 translated messages, 2369 untranslated messages.
• es.po: 1600 translated messages, 236 fuzzy translations, 533 untranslated messages.
• eu.po: 1392 translated messages, 272 fuzzy translations, 705 untranslated messages.
• fr.po: 2368 translated messages, 1 untranslated message.
• he.po: 0 translated messages, 2369 untranslated messages.
• id.po: 0 translated messages, 2369 untranslated messages.
• it.po: 1781 translated messages, 303 fuzzy translations, 285 untranslated messages.
• ja.po: 246 translated messages, 118 fuzzy translations, 2005 untranslated messages.
• ko.po: 1292 translated messages, 264 fuzzy translations, 813 untranslated messages.
• la.po: 0 translated messages, 2369 untranslated messages.
• nb.po: 0 translated messages, 2369 untranslated messages.
• nl.po: 1621 translated messages, 287 fuzzy translations, 461 untranslated messages.
• pl.po: 0 translated messages, 2369 untranslated messages.
• pt_BR.po: 1292 translated messages, 262 fuzzy translations, 815 untranslated messages.
• pt.po: 0 translated messages, 2369 untranslated messages.
• ru.po: 302 translated messages, 142 fuzzy translations, 1925 untranslated messages.
• sv.po: 1289 translated messages, 261 fuzzy translations, 819 untranslated messages.
• th.po: 0 translated messages, 2369 untranslated messages.
• zh_CN.po: 1691 translated messages, 300 fuzzy translations, 378 untranslated messages.
• zh_TW.po: 1664 translated messages, 289 fuzzy translations, 416 untranslated messages.

Results as patches to our patch tracker or the gforge-devel ML please.

(Note to Debian-related readers: this translation work will be directly useful on Alioth when we upgrade it.)

Quick status update: not much happened due to a variety of reasons, but there is still some progress to report.

The most important piece of news is that the Mediawiki plugin should be on its way to Debian sid by the time you read this, as the new gforge-plugin-mediawiki binary package (it'll have to go through NEW, but that seems to be rather fast these days). Testing and reporting and bugfixing are most welcome, of course.

I also went through a round of cleanups in the packaging. No more Lintian overrides, far fewer Lintian errors and warnings, and some fixes for PostgreSQL 8.3 compatibility.

First, and most important: while researching a functional bug for a client, I found a rather important security problem in GForge. All versions (starting from 3.1) are vulnerable to an SQL injection problem due to missing input sanitisation. Debian packages have already been fixed and released, and the patches have been committed to the upstream Subversion repository, so non-Debian users are encouraged to grab the patches from there. For instance, the patches for the 4.5.* branch can be obtained from the ViewVC page. For reference, the CVE ID for this problem is CVE-2008-0173.

Secondly, there's a new "gforge" tag on this blog, to filter posts that relate to GForge. I mainly created it in response to the existence of a feed aggregator focusing on forges and variants, but you can also subscribe to it directly if you only want to hear about Gforge and not about my other Free Software activities. I'll also use it to announce security patches like this one.

I'm on a roll...

• Gettext transition is fully done now (old functions have even been removed).

• French translation 100% complete! Woo!

• Packages now use ucf. So there.

• Installation process streamlined: if you just want the web interface, you'll only need to aptitude install gforge-web-apache2, and the only interactions it'll require are to type the admin password (twice) and to accept (or integrate by hand) a change in the PostgreSQL config file.

• It even works on Debian GNU/kFreeBSD!

Plans for the near future include continuing to clean up upstream code and maintainer scripts, making sure the installation process is as simple as possible (even for other subpackages), splitting out a few plugins into their own packages. And the big placeholders-in-prepared-SQL-queries audit I mentioned last time, but it may happen progressively rather than in one big go.

Apparently some people worry that Gforge may be abandoned, or on the verge of being superseded with the proprietary "Gforge Advanced Server" rewrite. Let it be known that I for one have no plans to switch to Gforge AS, for all the reasons you'd expect from a free software user and advocate: I don't have access to the source code (it's made illegible by some sort of industrial PHP obfuscator), I can't hack it because the license doesn't allow me to, I can't audit it for security flaws, I can't adapt it to particular needs, etc. Since a significant part of my income comes from maintaining Gforge instances for clients, with local modifications for their particular needs, it's an economical necessity that Gforge stays free. And evolving.

Right. Having said that, I guess I have to show concrete evolutions in addition to principles and ideals. So what changed recently?

• I believe the Gettext transition is now more or less complete. The initial bulk of it was done during the summer, but a few odds and ends were still dangling, such as plural forms here and there. The translations should now also be used in the appropriate place (no more receiving an email in Dutch even though you're French, simply because the person who clicked on the button that triggered the email was Dutch). I'm trying to bring the French translation to completion, but it's a long and boring task, so it'll happen gradually. As for other languages... "patches welcome".

• I've started using IPv6 at home, with a dual IPv4+IPv6 stack and hostnames that resolve to both kinds of addresses. The problem is that neither Gforge nor its packaging were designed with IPv6 in mind, so the Apache virtual host didn't listen on the IPv6 address, and even when it did, Gforge wouldn't understand IPv6 addresses in its session code. All that has been fixed, and you can now run Gforge on a dual-stack server.

• Lots of "undefined variable" PHP warnings have been fixed (not only by me, by the way). They probably weren't harmful by themselves, but they were signs of sloppy coding.

• The FCKeditor plugin now uses FCKeditor as provided by the Debian package, rather than its own (outdated and unmaintained) copy.

All this has committed and uploaded to Debian. As usual, please test and report failures.

Plans for my foreseeable future include:

• Making the clean-and-working Mediawiki integration plugin public (the delay is not due to technical reasons).

• Using placeholders in SQL queries, to reduce the risk of SQL injections. This probably means a full audit of all the places in the code where queries are made, so it may take some time.

• Maybe a full audit for HTML injections and cross-site scripting vulnerabilities, too. Probably a long and boring task too.

• Switching to ucf for management of config files.

• I'm still toying with the idea of switching to dbconfig-common, too. But since I'm not sure it's going to be good enough, that's not very high on my priority list.

• I'm pondering whether I should try to make a live-CD with an installed Gforge. Or maybe a fully automated installation with debian-installer and preseeding. That may debunk the misconception that Gforge is hard to install.

Places where help would be most welcome:

• Translations! I can do French, I fixed a few strings in Spanish and Japanese, maybe I can do some minor fixes in German too, but that's not good enough. Of course, translators should be aware this is not a small task: gforge.pot contains about 2200 strings. But even reading and fixing (or removing) the obviously incorrect translations would help.

• Packaging for RPM-based systems. I fear that aspect of Gforge has been all but abandoned, and I don't think there are any working RPMs around.

• Testing, testing, testing. Gforge is rather versatile, and a big piece of software. I don't use all the features myself, so bugs are likely to lurk in those places I don't see often enough.

• I know some of you out there have a locally patched Gforge installation. You know you'll have a hell of a time porting those patches to the next version when you upgrade. Why not share them, so everyone profits from them (and you can then profit from the patches from the other users)?

There. That was the news. Now for a bit of trivia: it's amazing how having a metric of one's productivity gives an incentive to increase it. I found mine on Ohloh, which provides code and license analyses and statistics on free software projects, as well as statistics on commits by contributors. They even have a shiny widget with a scrollable timeline showing commits over time, as well as comparative commit graphs. My obvious personal goal is to get up to the first position among the contributors, but of course I wouldn't complain if the current top committer stayed ahead by springing back into activity.

As I type this, debs for a current snapshot of upstream Subversion are on their way to Debian Sid. They are taken from the Subversion trunk and not from the yet-to-be-released 4.6 branch, because a few important changes have taken place on the trunk only, and the 4.6 branch is merged anyway. Hence the version number, 4.6.99+svn6078-1. These packages did spend some time in experimental, and I didn't get any bug reports, but that doesn't mean they're bug-free. Use with caution.

One of the big changes is that Gforge now uses Gettext rather than its home-made internationalisation system. Which means probably fewer problems, and a more standard system allowing more people to get involved. Can you guess what I'm hinting at? Yes, it's a call for translations! Anyone interested in getting involved should bzr branch http://alioth.debian.org/~lolando/bzr/gforge/upstream-svn/trunk/, and start poking the existing *.po files (or creating new ones!).

Why the private repo and not the upstream Subversion repository? Rationale follows.

My job as a freelancer as well as my role as an Alioth admin involve maintaining separate branches of the Gforge code (different clients have different needs and patches). In order to facilitate patch migration, I therefore need a distributed VCS, and I've been using Bazaar for a few years, with a manual gatewaying between upstream CVS first, then Subversion, and a few private Bazaar branches. Now that Bazaar (as in Arch) is dead, I'm using Bazaar (as in Bazaar-NG, Bazaar-2, or bzr), which seems to be approaching a 1.0 release, and which provides a plugin to interoperate easily with Subversion repositories.

And since the upstream Subversion repository is not accessible anonymously anyway, I decided to publish my gateway branch. I'll probably publish more branches as time passes (probably the Alioth branch, and quite possibly feature branches too).

Note: if you want to share a bzr repository of a project containing PHP scripts with Apache, you may encounter a problem, because the bzr repository contains files named *.php.knit and *.php.kndx. And Apache will happily give these files to the PHP interpreter when serving them, and that's not what you want. My trick to fix that is to add a .htaccess file somewhere where the repository is stored, with the following contents:

AddHandler None .knit .kndx

This will ensure that these files will be sent straight to the HTTP client, and not through the PHP interpreter.

Following about a month of irregular activity, I just uploaded three source packages (along with related binary packages, of course) to Debian unstable: gforge (4.5.14-9), gforge-plugin-scmcvs (4.5.14-3), and gforge-plugin-scmsvn (4.5.14-2). I hope unstable will give them more exposure than experimental did. Plans for the future: I’ll focus on getting these packages into testing, without too many changes if possible. I haven’t managed to get dbconfig-common to work reliably (or at all) with PostgreSQL so far, so I’ll consider the migration to its infrastructure as a nice option, but not a blocker. The problem is that it would make the install scripts simpler and (hopefully) more reliable if it worked. Again, help would be very much appreciated. (And yes, I will be submitting bug reports when I get some time to do some more thorough testing of dbconfig-common.)