Le weblog entièrement nu

Just an update about sgeps, because it seems to have made a small stir (which is more than I expected).

• Yes, I know about emacs foo.gpg. Admittedly I found out while I was “developing” sgeps, but I kept on my track anyway. The real reason was that I was having fun, but I could also mention that sgeps doesn't store data unencrypted on disk, not even temporarily. I'm not sure about vim foo.gpg. (Update: Joey Hess telle me it does the right thing.) Anyway, I don't want to fire up an editor (or switch to an already opened one) just to get a password.
• I had also found out about pwman too. My script started as pwman.pl but it was renamed later. I like the simplicity of sgeps better, especially the lack of any UI besides the CLI, but pwman is probably good in its own way, but it doesn't fit my usage pattern.
• It seems the Gentoo people don't share my qualms about making very small packages, and apparently sgeps is now packaged for Gentoo Linux.
• Mehdi Dogguy already contributed suggestions (including better error handling) and even a patch implementing sgeps --delete. Thanks to him!
• An anonymous commenter suggests that sgeps should be able to store notes as well as passwords. That wasn't a requirement I initially had, but I won't reject the patch if it comes… He also argues that it should be able to push the password into the X11 paste buffer. Again, why not, if it doesn't break anything.

I've been accumulating passwords recently. More than I could remember all in one go. I even got worried that I'd locked myself out of one of my own servers recently. So I decided to play it safe and store the passwords somewhere. However, plain text files, even on an encrypted disk, aren't the most secure plan, so I tried to go shopping for a tool that would store passwords in encrypted files and wouldn't be too inconvenient to use. I found a few (pwsafe, keysafe, keepassx, yapet and so on), but they all seem to be either graphical or using their own encryption scheme and (presumably) storage format. Being rather nervous about long-term data accessibility, I thus decided to roll my own script, that would be as simple as possible while doing just the required amount of work.

I call the result sgeps, for “simple GnuPG-encrypted password store”. Note the initial s: I didn't invent any wheel.

• Data model: a list of key/value pairs (each being a string);
• Storage: serialisation using Perl's built-in Storable module;
• Encryption: the serialised data is GnuPG-encrypted;
• Hopefully secure: no password stored in plaintext files at any time.

The code comments should give an idea of the capabilities of sgeps:

  # Usage: sgeps --create                     to create the store
  #        sgeps --add <key>                  to add a key/value to the store
  #        sgeps --list                       to list existing keys
  #        sgeps --add --overwrite <key>      to replace a key/value

I trust both GnuPG and Perl to stay around for quite some time, so hopefully I can forget even the passwords I use very rarely and still be able to recover them later. Even in the event of a hard drive dying, since the encrypted store can now be backed up and burnt on DVDs. I “just” need to be careful about my GnuPG key.

Interested people can grab sgeps from its Bazaar branch with bzr branch http://bzr.debian.org/users/lolando/sgeps/trunk/ or browse it on the web interface. I don't plan to make a Debian package for a hundred lines of Perl code, but if anyone is interested, feel free to include it in an existing package (moreutils maybe?).

This month hasn't seen many big changes happen in FusionForge. Notable improvements include an initial search engine for Word files, fixes to the automated builds and tests, and lots of bugfixes.

The biggest news is probably the start of the Coclico project, an initiative bringing together developers and users of several existing forges in order to reduce the gap (and ideally unify the codebase across the forks) and work together in some fields where cooperation is important. Subjects include a generalisation of the current identity/permission/authentication models and systems, data exchange and migration, interoperability, integration of agile development methods inside the forge, and better integration with the desktop applications such as IDEs. The participants include NovaForge, Codendi, and of course FusionForge. The project only officially started early this month, but we hope to be able to demonstrate results soon.

Business as usual apart from that.

Here's another round of the semi-regular bulletin about FusionForge.

First item: FusionForge 4.8.1 was released this week. It's not exactly an important update, but the 4.8 branch had been accumulating fixes over time and we felt that it would be good to push these fixes out. If you don't encounter particular problems, there's probably no need to upgrade in a hurry.

A follow-up for the rewrite of the SCM subsystem: I now consider the Bazaar and Git plugins complete. The missing part, in both cases, was a proper integration of a repository browser and the collection of commit statistics; since one of my clients wants to use Bazaar and another one wants Git, both features have been completed recently. The code still lives on a branch based off 4.8 (for people who need a 4.8-based instance), but it's also been pushed into trunk so the next release will have it natively.

Another branch I've been working on (for clients) was about making the Mediawiki plugin able to handle one wiki per project rather than one shared wiki. This is now possible with yet another 4.8-based branch, where the wiki creation is completely automated. A nice feature is that the FusionForge identification is used as a basis for Mediawiki, with different groups on the wiki depending on project membership and role in the forge. That allows specifying wiki permissions in a simple way, for instance to say that only project members can create new pages, authenticated users can only edit existing pages, and non-authenticated users are read-only. This code will be pushed to trunk in the coming weeks.

Thanks to Alain Peyrat, we now have a buildbot running Hudson for unit tests and a few other things. The coverage isn't complete yet, but we hope to increase it as time passes. It's already proven useful, by ensuring at least correctness of PHP syntax, encoding and line-endings.

I think that's about it for this time. Business as usual.

I have a problem with Debconf, but it's far from specific to Debconf. If anything, it's specific to me. I have a short term memory for people, and I tend to forget faces, names and nicknames. And their mappings. Which means that people I haven't seen in a few years tend to get blank looks, puzzled frowns and/or awkward greetings from me. Sometimes I know the face but it takes me a while to put a name on it, sometimes a nickname pops up, sometimes just a feeling that I've met a person for such-and-such occasion in such-and-such place. Sometimes I get it all rushing back at me after a few minutes.

This has happened a few times already at this Debconf, and is likely to happen again, except for people who'd be hard to forget (colourful shirts, memorable hairstyles and exuberant personalities tend to stick in my memory). This is bothering me as much as you, and I would like to apologise to all people I'm likely to offend. Sorry, all of you. Whoever you are.

Welcome to this month's FusionForge news batch.

I did a presentation of FusionForge at the Libre Software Meeting (Rencontres Mondiales du Logiciel Libre, in French) earlier this month, to explain where we come from and where we hope to go. Many people attended despite the talk being early on the morning following the formal dinner, and the questions showed interest, which is encouraging for the project as a whole. I don't think the talk has been recorded, but the summary and slides are available on the RMLL website.

The big news, though, is that I'm currently at the Debian Conference, Debconf, and that I also attended Debcamp before that. Debcamp is a very productive get-together of developers from all across Debian, and I took the opportunity to get help from them. I spent the first few days refactoring some of the code that was duplicated between the CVS and Subversion plugins, and the result is that version control plugins are now much easier to implement. Case in point: I managed to get the attention of a few users of other tools, and since they only had to implement small specific parts, we now have almost complete plugins for Bazaar, Darcs and Git, and Mercurial will probably follow. CPOLD was done too, but mostly as a proof of concept. If you're around, come and see me, we'll finish the support for your favourite tool together. Or even start it (I haven't started on Arch and Monotone for lack of perceived interest, but I'm quite open to these tools too). In both cases, I promise it won't take long.

This code currently only lives on a temporary branch based off FusionForge 4.8, but I'll port that to trunk and commit it in the coming weeks.

Quick heads-up about FusionForge. The main news of course is that 4.8 has been released upstream (and uploaded to Debian experimental). We'll keep fixing major bugs on that branch of course, but our focus is now on trunk.

We're finding it tedious to deal with legacy code, so one of the goals we have now is to clean up the codebase to bring it more in line with good practice. That's going to take some time, though, because there's lots of code. Some of that code, however, seems unused (it's been broken for some time without anyone complaining), so it's likely that we'll deprecate and/or remove bits of code unless someone steps forward to maintain it (or at least bring it into shape). In particular, we're looking at the MySQL support (which hasn't been maintained for years) and some of the old visual themes which are going to require some work to keep working with some changes we're planning in the way the pages are displayed.

This should make maintenance easier for the implementation or integration of new features down the line. Which will be the subject of a future post, when a currently undercover French Forge Cabal actually starts producing concrete results. Watch this space.

It's been too long since the last blog post about FusionForge, but we haven't sat on our hands and lots of things happened. This is a brief summary:

• About 900 commits in total over the trunk and the two currently active branches.
• Three minor bugfix versions were released, and the currently stable version is 4.7.3.
• A branch has been started to stabilise the code in prevision for the upcoming 4.8 release. Two release candidates have been published already, the final version should happen in the coming weeks.
• Feature-wise, 4.8 will add a tag cloud for project classification, better reporting for downloads, and a new version of the phpwiki plugin. It will also include the beginnings of an automated test suite.
• 4.7.3 is in Debian unstable, 4.7.2 in testing, 4.8rc1 is in experimental. Don't be fooled by the packages still being called gforge, it's just that they haven't been renamed to fusionforge yet.

Not bad for a start, eh? But we've also kept true to our promise and started merging lots of things from various local branches and patches. Among what's been committed to trunk so far are the following features, coming from various existing or upcoming instances of GForge/FusionForge.

• A script for migrating data from an existing GForge AS instance to FusionForge. It's not a turn-key solution, and it requires some manual intervention for working, but it does most of the job.
• New extratabs plugin, allowing project administrators to define new tabs for their projects, pointing at arbitrary URLs.
• New projectlabels plugin, allowing site administrators to define labels as chunks of arbitrary HTML and stick that HTML onto some projects (useful for "project of the month" stuff).
• Refreshed globalsearch plugin, allowing cross-forge project searches.
• Command-line data injectors to populate a FusionForge database with data coming from another system.
• A more consistent automated self-test suite using Selenium.

Other great things are afoot, and they'll be described here in due time.

Hot from the oven: FusionForge 4.7 was just released. The release notes follow. If you have local enhancements based on a previous version of GForge, now is the time to port them to the 4.7 codebase and submit them, so they can be merged in time for the next version!

Release notes for FusionForge 4.7

This is the first public release of FusionForge. FusionForge is based on GForge, and started as an identical copy, with only a name change to avoid confusion with the proprietary versions of GForge (known as GForge Advanced Server or GForge AS). As such, it benefits from mature code and known-good infrastructure, and builds on it for the future.

This 4.7 release is focused on bringing the recent evolutions out to the community in an official stable release. This should provide a solid base as a starting point for community-based development, making it easier for enhancements to be maintained. The FusionForge name was chosen to reflect this: this is a community effort, and we hope to hear about your improvements. Contributing these improvements would make their future long-term maintenance easier for everyone.

Major changes since previous versions (of GForge) include:

• Support for PHP5.
• Support for PostgreSQL 8.x.
• Translations are now managed by gettext.
• Support for several configurations running on the same code.
• Improved security, no need for PHP register_globals.
• Available as full install CD.
• New wiki plugins (using MediaWiki or phpWiki).
• New online_help plugin.
• New phpwebcalendar plugin.
• New project hierarchy plugin.

Things to keep in mind when installing:

• FusionForge is based on GForge, and the renaming is quite recent. So the code still contains lots of references to GForge. This will be fixed as time passes.
• Full text search using the primitives provided by PostgreSQL 8.3 isn't quite complete yet.
• Not all plugins are packaged for all distributions yet.

Things to keep in mind when upgrading:

• Since internationalisation was changed from a hand-made system to standard gettext, locally customised translations will no longer override standard ones. This will be addressed in a future release.

For more up-to-date information, please visit http://fusionforge.org/ or http://fusionforge.fusionforge.org/ -- you can even join us on IRC from there!

-- The FusionForge development team

Executive summary

To avoid confusion with the proprietary versions of GForge (known as GForge Advanced Server, GForge Express Edition and GForge Community Edition), the free/libre/opensource codebase will from now on be separately maintained under the name FusionForge by the main developers of the free GForge 4.x codebase. Since this is mostly a renaming, the migration path for existing users will be smooth.

Longer version, with details

After the initial forking from the Sourceforge codebase, the development of GForge has long been hosted, and many enhancements directly developed, by the GForge Group (GForge, LLC), with regular contributions from outsiders. The results of these evolutions were public and free, subject to the GNU GPL.

In parallel, the GForge Group wrote a proprietary re-implementation of GForge, which it sold under the name "GForge Advanced Server", or "GForge AS" for short. This re-implementation added some features for "the enterprise", but was not contributed wholesale to the GForge codebase under a free license. Although some of the features were contributed to the public, the GForge Group concentrated its efforts on its (proprietary software) business model, with more versions appearing, such as "GForge Express Edition" and more recently "GForge Community Edition". As a result, it became increasingly harder for the public to know which version was which without doing extensive research (indeed, some users mistakenly installed one version instead of the other). A consequence was that the free software codebase suffered from a loss in visibility, which lowered its momentum to the point that there haven't been any moderately important releases since the (currently stable) 4.5.x series was announced in late 2005.

So, in order to clarify things, avoid further confusion, and regain some of the lost momentum, it was decided by a group of leading contributors that the free software version of the GForge codebase would from now on be developed under the FusionForge name, and its development would be hosted on FusionForge.org.

So is this a fork?

Well, we don't know yet. It could arguably be called one, since we're taking the code and running away with it under a new name. However, we believe it's not a fork unless both roads continue their own way (more of a oddly-shaped bend). What happens to the GForge codebase developed by the GForge Group at gforge.org remains to be seen, although for the sake of our users we will backport security fixes to the gforge.org Subversion repository (at least for the 4.5.x series and the unreleased 4.6 and 4.7 pre-series) for some time. The bulk of the development will move on to FusionForge and the repositories at FusionForge.org, though, and users are encouraged to migrate at their own pace. Since we're basically continuing the evolution rather than starting from scratch, the migration path should be rather smooth.

So why the FusionForge name?

Because there were actually lots of locally-patched versions of GForge (and Sourceforge), and we felt it was a waste of resources that should be fixed. It seems many people and organisations took these codebases at some point in time and evolved them for their own needs. Sadly, many of the changes were not contributed back or even published, so lots of efforts were duplicated. Fortunately, many of the people managing these locally-patched forges are now realising that "out-of-tree" patches and features require quite some manpower to maintain. Some formal inter-project discussion is already taking place, and we hope to achieve actual merging of most of the interesting features that have been developed here and there into a common base that can be reused locally with minimal changes. We'd like to "un-fork" as much as possible.

We also expect that, by using standard components and tools, we'll facilitate the work of potential contributors, thereby reducing the risk of a new era of fragmentation.

And who are we anyway?

We're Christian Bayle, Roland Mas and Alain Peyrat, long-time contributors to GForge and responsible for over 95% of the commits over the past two years, as well as a few relative newcomers. Christian and Roland have been maintaining the Debian packaging since the "Debian-SF" era, and Alain has been focusing on code quality. The three of us have, for various reasons, a vested interest in maintaining a lively codebase in a healthy ecosystem.

What are our plans?

Our short-term goals, as currently planned, include:

• pushing a stable FusionForge release out of the door;
• cleaning up and auditing the database schema to ensure consistency and increased performance;
• merging in new plugins, in particular for new version control systems;
• continuing streamlining the installation process to make it more portable across distributions.

Longer term goals are less well defined, but we're thinking about the following:

• increasing code quality by the use of modern automated tools;
• encouraging a lively stream of external contributions, to reduce the gap between the official version and the locally-patched ones;
• defining an explicit governance model and release process for the FusionForge project;
• as a consequence, a more frequent and predictable release schedule;
• increasing data portability and interoperability with other forges, to reduce lock-in for users and projects.

Some of these items should be facilitated by our switch to a distributed version control system and a new coordinated workflow. Also, the Debian i18n team has been kind enough to offer to host our translation effort on their Pootle server, which means translators will have a much easier time doing their job.

We hope to hear from users and contributors alike in the near future.

For more information, we can be reached via our fusionforge-general mailing-list (see our lists), which is also suitable for general discussions. We can also be found on IRC (#fusionforge on the Freenode network).