Salsa webhooks and integrated services

Since many years now, Debian is facing an issue with one of its most important services: alioth.debian.org (Debian's forge). It is used by most the teams and hosts thousands of repositories (of all sorts) and mailing-lists. The service was stable (and still is), but not maintained. So it became increasingly important to find its replacement.

Recently, a team for volunteers organized a sprint to work on the replacement of Alioth. I was very skeptical about the status of this new project until... tada! An announcement was sent out about the beta release of this new service: salsa.debian.org (a GitLab CE instance). Of course, Salsa hosts only Git repositories and doesn't deal with other {D,}VCSes used on Alioth (like Darcs, Svn, CVS, Bazaar and Mercurial) but it is a huge step forward!

I must say that I absolutely love this new service which brings fresh air to Debian developers. We all owe a debt of gratitude to all those who made this possible. Thank you very much for working on this!

Alas, no automatic migration was done between the two services (for good reasons). The migration is left to the maintainers. It might be an easy task for some who maintain a few packages, but it is a depressing task for bigger teams.

To make this easy, Christoph Berg wrote a batch import script to import a Git repository in a single step. Unfortunately, GitLab is what it is... and it is not possible to set team-wide parameters to use in each repository. Salsa's documentation describes how to configure that for each repository (project, in GitLab's jargon) but this click-monkey-work is really not for many of us. Fortunately, GitLab has a nice API and all this is scriptable. So I wrote some scripts to:

• Import a Git repo (mainly Christoph's script with an enhancement)
• Set up IRC notifications
• Configure email pushes on commits
• Enable webhooks to automatically tag 'pending' or 'close' Debian BTS bugs depending on mentions in commits messages.

I published those scripts here: salsa-scripts. They are not meant to be beautiful, but only to make your life a little less miserable. I hope you find them useful. Personally, this first step was a prerequisite for the migration of my personal and team repositories over to Salsa. If more people want to contribute to those scripts, I can move the repository into the Debian group.

Debian from 10,000 feet

Many of you are big fans of S.W.O.T analysis, I am sure of that! :-) Technical competence is our strongest suit, but we have reached a size and sphere of influence which requires an increase in organisation.

We all love our project and want to make sure Debian still shines in the next decades (and centuries!). One way to secure that goal is to identify elements/events/things which could put that goal at risk. To this end, we've organized a short S.W.O.T analysis session at DebConf16. Minutes of the meeting can be found here. I believe it is an interesting read and is useful for Debian old-timers as well as newcomers. It helps to convey a better understanding of the project's status. For each item, we've tried to identify an action.

Here are a few things we've worked on:

• Identify new potential contributors by attending and speaking at conferences where Free and Open Sources software are still not very well-known, or where we have too few contributors.
  
  Each Debian developer is encouraged to identify events where we can promote FOSS and Debian. As DPL, I'd be happy to cover expenses to attend such events.
• Our average age is also growing over the years. It is true that we could attract more new contributors than we already do.
  
  We can organize short internships. We should not wait for students to come to us. We can get in touch with universities and engineering schools and work together on a list of topics. It is easy and will give us the opportunity to reach out to more students.
  
  It is true that we have tried in the past to do that. We may organize a sprint with interested people and share our experience on trying to do internships on Debian-related subjects. If you have successfully done that in the past and managed to attract new contributors that way, please share your experience with us!
  
  If you see other ways to attract new contributors, please get in touch so that we can discuss!
• Not easy to get started in the project.
  
  It could be argued that all the information is available, but rather than being easily findable from on starting point, it is scattered over several places (documentation on our website, wiki, metadata on bug reports, etc…).
  
  Fedora and Mozilla both worked on this subject and did build a nice web application to make this easier and nicer. The result of this is asknot-ng.
  
  A whatcanidofor.debian.org would be wonderful! Any takers? We can help by providing a virtual machine to build this. Being a DD is not mandatory. Everyone is welcome!
• Cloud images for Debian.
  
  This is a very important point since cloud providers are now major distributions consumers. We have to ensure that Debian is correctly integrated in the cloud, without making compromises on our values and philosophy.
  
  I believe this item has been worked on during the last Debian Cloud sprint. I am looking forward to seeing the positive effects of this sprint in the long term. I believe it does help us to build a stronger relationship with cloud providers and gives us a nice opportunity to work with them on a shared set of goals!

During next DebConf, we can review the progress that has been made on each item and discuss new ones. In addition to this session acting as a health check, I see it as a way for the DPL to discuss, openly and publicly, about the important changes that should be implemented in the project and imagine together a better future.

In the meantime, everyone should feel free to pick one item from the list and work on it. :-)

Newmaint — Call for help

The process leading to acceptation of new Debian Maintainers is mainly administrative today and is handled by the Newmaint team. In order to simplify this process further, the team wants to integrate their workflow into nm.debian.org's interface so that prospective maintainers can send their application online and the Newmaint team review it from within the website.

We need your help to implement the missing pieces into nm.debian.org. It is written in Python and using Django. If you have some experience with that, you should definitely join the newmaint-site mailing list and ask for the details. Enrico or someone else in the list will do their best to share their vision and explain the needed work in order to get this properly implemented!

It doesn't matter if you're already a Debian Developer to be able to contribute to this project. Anyone can step up and help!

2016 DPL election

It is time of the year where Debian project members should elect a new project leader. This year, only one candidate has stepped up, yours truly. As a reminder, my platform is published here. The campaign has been quite calm, comparing to past editions, which is not surprising given the number of candidates. There have been some interesting questions though as detailed below:

• Thoughts about quantity of DPL candidates? (reply)
• Thoughts about liw's non-platform (reply)
• Broader vision on Debian and its role in the FOSS/IT ecosystem (reply)
• DPL job profile (reply)

There are not many questions. It will take little time to read, if you haven't already.

Last but not least, please do vote! You have until 2016-04-16 23:59:59 UTC to vote (as announced here). It is very important to share your opinion and take part in the reflection around the future of the project. That every project member takes part in the evolutions that should be implemented to stay relevant and innovative in what we do.

In memoriam: Ian Murdock

It is with great sadness that I learned of the passing of Ian Murdock. I have never had the chance to meet him. Several persons testify for his kindness and talent. We will always remember him. His legacy influences our lives everyday!

Looking at his latest blog posts, he seemed a bit nostalgic about Debian and still very proud of it.

Later, he wrote about how he came to find Linux and the importance of telling the story of hackers of his generation. In his memory, I'll reread Stephen Levy’s Hackers for the nth time too.

When (if?) his webserver will shut down, you will still be able to read his past blog posts using the archived version of his website or a static mirror that has been set up.

RIP Ian.

3rd annual Aaron Swartz Day, November 7-8

This weekend is organized the Aaron Swartz Day across the world. There are events organized in many cities and video streams available. It is important that we remember Aaron's projects and fights. If you want to know more about Aaron Swartz, you may start by watching the excellent documentary The Internet's Own Boy : The Story of Aaron Swartz. His work was very inspirational and should not be forgotten!

DPL campaign 2015

This year's DPL campaign is over and voting period is also almost over. Many did not vote yet and they really should consider doing so. This is meant as a reminder for them. If you didn't have time to dig into debian-vote's archives and read questions/answers, here is the list with links to candidates' replies:

• How would you spend Debian's money? (mehdi, neilm, algernon)
• Would you authorize to use regular Debian funds to sponsor Debian participation into Outreachy? (mehdi, neilm, algernon)
• What do you think about outsourcing some of the gruntwork related to accounting and treasury to professional agencies? (mehdi, neilm, algernon)
• Will you revoke <20131008134615.GA19150@xanadu.blop.info> or do you think this authorization is useful? (mehdi, neilm, algernon)
• SWOT analysis (mehdi, neilm, algernon)
• A bundle of long questions from ajt (mehdi, neilm, algernon)
• What is your perception of DebConf and its organisation? (mehdi, neilm, algernon)
• What is your perception of fundraising in and around Debian? (neilm, algernon). I didn't answer to this question explicitly as I mainly agree with algernon and a similar topic has been discussed/explained elsewhere (platforms, other questions, ...)
• Dropping §5 from our social contract (mehdi, neilm, algernon)
• Debian in five years? (mehdi, neilm, algernon)
• PPA (mehdi, neilm, algernon)
• Have you considered working with a "DPL team" (mehdi, neilm, algernon)
  
• More women in key positions? (mehdi, algernon)
• Also, as a reminder, you can find candidates platforms here.

Compared to past years, we had a comparable number of questions. All questions did not start big threads as it used to be the case sometimes in the past :-) The good side of this is that we are trolling DPL candidates less than we used to do :-P

Now, if you still didn't vote, it is really time to do so. The voting period ends on Tuesday, April 14^th, 23:59:59 UTC, 2015. You have only a few hours left!

Running for DPL

Every year, Debian organizes a DPL election. Around end of March, one waits for the beginning of the DPL campaign. Everyone can ask questions to nominated candidates on debian-vote. This year, and for the first time, I nominated myself as a candidate for the 2015 DPL election. You can read my platform here.

Over the past few years, I've followed DPL campaigns on debian-vote reading questions and replies from candidates. It didn't seem easy to keep up with flood of questions and find the right wording while replying. Intuitively, you may think that a question is the first mail of every thread and replies follow... but, not at all :-) Questions can be asked in any mail. So candidates have to read every single mail posted to the list :-) The campaign ends within a week (or so) and it is still time to ask more questions.

Following discussions on debian-vote is a very good opportunity for newcomers to understand, for example, how Debian works and where help is needed. It is also a good place to see what are the main current issues (as perceived by contributors) and read a list of proposals to fix them. I invite anyone interested in Debian in reading debian-vote's archives.

While preparing my platform, I've also realized how much writing down thoughts and ideas was important. It really helps to put things into perspective and re-evaluate priorities. It may sound obvious but I think we are not used to do this often. I really recommend everyone to do this as an exercise, and for any perimeter (personal, team, project-wide, ...).

Last but not least, I'd like to thank all those who helped me to polish my platform and to prepare my candidacy. I am sure they will recognize themselves :-) (whatever the outcome of the election may be)

Clickable DepWaits and Edos results in pgstatus

A long time ago, Stéphane Glondu implemented an alternative to the Buildd status pages. It was implemented in OCaml and had some unique features. I think that my favorite one was the ability to click on package names appearing in DepWaits.

What you see on the image below is a simple example where the package gnome-shell Dep-Waits on a single package. This prevents buildd daemons to start the job as long as the specific version of the specified package or some new package are available. So, each package name in that field became now clickable and points to its buildd status page.

Similarly to Dep-Waits, results of edos-debcheck, which is used to spot installability issues of build dependencies, are now clickable too.

If you are a porter or if you are tracking down a build issue, I am sure you'll appreciate these two new features. Enjoy! As usual, if you notice some bugs or have some feature requests, please let me know.

Wheezy is out!

Wheezy is finally out! Thanks to everyone who contributed to this release!

[Image from: http://devopsreactions.tumblr.com/post/49640679398/debian-wheezy-released]

DPL Game

For once, I think we are going to have a funny DPL campaign thanks to Francesca who brought to us a DPL Game :) So here is my little list of fantastic four:

• Gregor Herrmann
• Lucas Nussbaum
• Russ Allbery
• Enrico Zini (sorry, you've been on my list since the beginning).

(in random order)

Each one of them has super powers (lot of energy, wisdom, impartiality, ability to sort out complex situations) and can do a great DPL!

My fantastic four, please consider running for DPL!

Android's Calendar app

http://code.google.com/p/android/issues/detail?id=5484

Such a basic/important app and such a stupid bug... 3 years and not fixed yet? Seriously? and people claim Android is ready for lambda users?

</rant>

Adding useful links on bugreports pages

I'm a heavy user of Debian's Bug Tracking System (especially during this period of the year ;)) and I quite like it. And as a Debian developer, I have to check multiple sources when looking at some source package. Namely, I mainly check the following sources:

• Packages Tracking System — gives almost all information concerning a source package (maintainer, available versions, number of bugs, etc…)
• Buildd Status Pages — build status for the package on each architecture
• and sometimes Popularity Contest numbers

All this is great since you can move from the first one to the second one (and vice-versa) just by clicking with your mouse. It becomes less handy when you are looking at any bug report. In fact, the BTS doesn't link back to the previously mentioned services [1]. So, you have to copy/paste the name of the package or start a new tab and write down the address of the desired service. All this is not very nice for a lazy person like me :)

A few days ago, I remembered that I was already using GreaseMonkey and thought : "Hey, it would be nice to hack quickly something using GM to add the missing links". After a few tests, I've come up with a (very simple, yet extremely useful to me) user script that I called Debian BTS extra links [2]. It adds what I've been missing for years. If you are lazy and don't want to install right now, you can check the following pages to see what it adds:

• A regular bug report page
• A modified bug report page

The userscript will add links to PTS, Buildd status pages and Popcon near every pkgreport.cgi?src= link. Simple and useful. I hope you'll enjoy it as much as I do.

Of course, I've submitted a bug (#679282) against the pseudo package bugs.debian.org kindly asking its maintainer to add a link to (for now) the PTS. But I know that they are busy so I'm not expecting a result soon. Fow now, my simple and dumb userscript WFM.

[1] This is partly true since it links back to the PTS on main pages (pages that list all bug reports of a given set of packages).

[2] Only tested with a recent Iceweasel (from experimental) and Chromium (from unstable).

P.S.: If you do wonder, #4300 is the least bug number known to the BTS. Well, or at least, the result of running ls -1 /srv/bugs.debian.org/spool/archive/00/ | sort -h | head -n1 on bugs.debian.org.

Au revoir président !

La vidéo la plus regardée du moment :



Ça fait plaisir de respirer de l'air frais :)

I'm going to DebConf 2011

I'm really happy to be able to attend DebConf11. It will be my third DebConf. I hope it will be as productive and enjoyable as past editions :) I'll arrive on July 20th and stay until the end of DebConf.

See you all there soon!

First Tunisian Debian mirror

It seems that Tunisia hosts a new shiny complete Debian mirror since a few days. It's available at http://debian.mirror.tn. It even has all release architectures, which was a good surprise. Possibly, this new mirror will get its ftp.tn.debian.org some day if it implements all the requirements (which are listed here). As far as I know, this is the second complete Debian mirror in Africa. The first one is hosted in South Africa. Algeria hosts a Debian mirror too, but has i386 only. http://debian.mirror.tn had even a complete debian-cd/ directory when I had a look at it last Friday, but it disappeared over the week-end. I do wonder if it will come back at some point. It would make a lot of people happier, in my opinion.

This new mirror is hosted by ATI (Tunisian Internet Agency), a government agency which organizes anything that has something to do with Internet :) (IP addressing, interconnection of ISPs, mail gateways, main registrar of .tn domain extension, …). During Ben Ali's era, they were known for Internet censorship. But, it seems that censorship was controlled directly by the palace. Since the revolution in Tunisia, ATI tried to clean his name and start a "new life".

At the same location, they also host other mirrors among which we can find: Ubuntu, Mozilla, Eclipse, CPAN, Apache and Fedora. So, I think I'm not mistaken to say that there is a real intention to support Free Software. I hope they'll be able to go further and provide hosting for Free Software Clubs/Groups in Tunisia, or even help to organise FOSS events.

Anyway… It's nice to know that I'll have a decent Debian mirror to use this summer from Tunisia :)

The revolution continues… stay tuned!

Aubry en a marre !

Le prochain qui fera un scandale au Parti Socialiste aura affaire à Martine Aubry !

P.S.: Photo trouvée sur le site de 20minutes.fr. Elle montre Martine Aubry donnant le départ de la route du Louvre à Lille, le 15 mai 2011.

Oops!… I did it again

Well, I still don't feel ready enough to go back to (real) work. So, once more, I had to get my fingers on something to hack and I picked the Buildd Status Pages once more. It's not my favorite project, but I decided to implement the few remaining ideas and get (temporarily) done with it. So, here we are with new features (and a few bug fixes):

• Add option to show (co-)maintained packages (Closes: #559515, #507782). It takes as input two files (Maintainers and Uploaders) to find the list of packages. Unfortunately, they contain binary packages (and particularly architecture "all" packages) that wanna-build doesn't know anything about. On buildd.debian.org, I don't have enough data (yet) to filter out that list manually. Ideally, my input should be a list of sources packages with at least one binary non architecture "all" packages only. Anyway… now, if you enter an email address, the script should parse those files looking for your list. Then, it adds a link to DDPO using that email address and replaces it by the list of found packages in the form, so that you can manually filter it.
  Update: I ended up filtering manually the list of packages. I keep only packages that wanna-build knows something about, on at least one architecture.


• Escape special HTML characters when we fetch a log (Closes: #621739). No comment on this one please :) At least, it was very easy to fix.


• Add a "raw" option to fetch.php so that it outputs only what's in the log file, without any processing or escaping.


• Add a link to buildd.debian-ports.org, with the same list of selected packages and suite. Additionally, I've made some changes so that the deployment there becomes a bit easier.


• Try to produce valid XHTML code. I just went through errors I've seen and fixed them. The produced XHTML code is rather simple. So, I don't need a real library for that yet.


• Add support for tail of logs. If your package fails to build on some architecture, the script will show you the last 15 relevant lines below the status table.

In my opinion, this new set of features deprecates totally the old version, since all (and more) is now covered and working with this new version. So, I think we can remove pkg.cgi and friends and put some redirection to the new scripts. If you disagree, please tell us ! I'll submit a bugreport to QA folks asking them to change/remove links still present on the PTS and DDPO.

I think that this is the last set of changes before a while (except fixes). I hope that you'll enjoy these new features as much as I do. If I still have some energy, I'll try to spend it on -release stuff where there are several ongoing transitions (and many others waiting for acks, hints, analysis, etc…).

Buildd status pages: say goodbye to external dependencies

Last week, I had to put my daily (real) work off for a bit, to get some rest.  I took this occasion to implement a few ideas I had for the Buildd Status Pages. Mainly, I wanted to get rid of my external dependencies which are:

• https://buildd.debian.org/build.php which shows all build logs available for a package. It's nice and simple, but old and not easy to read. In my opinion, it's almost useless because it takes some time to realize (visually) what happened to the package (there are no colors to distinguish successful builds from failures, text is not aligned, etc…).


• https://buildd.debian.org/fetch.cgi which shows the content of a build log given a package, a version, an architecture and a time-stamp identifying the log file (you can see this as an example). The page works, there is no real need to rewrite it, modify it or whatever. It just does its job perfectly, but it was my last external dependency :)

Besides of getting rid of external dependencies, there were also a few feature requests (#618676, #612174, #518526) waiting for a while and it was good time to fix them. Hopefully, there are related to what I was fixing.

The final version is now deployed and can be used. The implemented changes are :

• From the regular package view, you have now a link to all logs available for that packages (click on Logs in the table), or all logs for a specific architecture (click on old in the table, for you preferred architecture).


• The listing of available build logs has now its own new page. The default listing shows everything, i.e. every version available and everywhere known architecture (just like the old page did before). But now, you get with that the build time and disk space consumed during the build, when available. Furthermore, it groups the logs by version so that it becomes easier to see that "all that group are logs for the same version". And you can also show build logs for a specific architecture by choosing one (links in rows) ; or restrict on versions, in a similar way. And, of course, there are some additional colors in each row to distinguish successful builds from those who failed.


• The page that fetches the log is now available from other pages. And this removes the last external dependency.


• All pages have an appropriate HTML title set (yeah, it's a shame that I didn't fix that since the first deployment).

I think that I tested well enough this new version and it works okay. Nevertheless, if you encounter a bug or a strange behavior, please don't hesitate to contact me (either by mail or on irc) or submit a bug report against buildd.debian.org.

The next steps (probably) are :

• add the last feature that pkg.cgi has which the ability to select a list of packages according to their maintainer (or co-maintainer).


• a better integration with P-A-S files.


• show tail of logs for build failures.


• ask wb-adm if build.cgi, bymaint.php, fetch.php, info.cgi are still needed/useful. If you do use use/need them, please speak up before they get removed.


• if you have any other idea, do submit a bugreport.

Asking questions, the Free Software way

Recently [1], Zack, from an original idea of Jorge, asked the project whether it makes sense to have a debian instance of Shapado (a free software web application for question and answer support for our users). The idea was well accepted and http://ask.debian.net showed up quite quickly[2].

More recently, I stumbled upon an article[3] which announces the birth of the pretty same service but for Ubuntu users (in partnership with Canonical, Ltd.). It was known for some time as http://ubuntu.stackexchange.com. Then, it was redesigned (I guess) and offered a new domain name. The service runs Stack Exchange which is like Shapado but closed-source. It even runs on Windows Server and .Net. Admittedly, there is also http://ubuntu.shapado.com but it doesn't seem very active (nor seems to be supported by Canonical).

In 2 days timeframe, both distributions gained the same service but they did it quite differently. I found this chain of events quite funny. And, really, I'm quite happy to be on Debian's side (as always) :)

[1] http://lists.debian.org/debian-project/2010/09/msg00123.html
[2] http://lists.debian.org/debian-project/2010/10/msg00028.html
[3] http://blog.stackoverflow.com/2010/10/ubuntu-stack-exchange-is-askubuntu-com/