Arch Linux

Please read this before reporting a bug:
https://wiki.archlinux.org/index.php/Reporting_Bug_Guidelines

Do NOT report bugs when a package is just outdated, or it is in Unsupported. Use the 'flag out of date' link on the package page, or the Mailing List.

REPEAT: Do NOT report bugs for outdated packages!
Tasklist

FS#24999 - {bugtracker} Migrate Flyspray to an alternative (Bugzilla/Roundup)

Attached to Project: Arch Linux
Opened by Dmitry Korzhevin (dkorzhevin) - Sunday, 03 July 2011, 14:32 GMT
Last edited by freswa (frederik) - Thursday, 20 February 2020, 21:08 GMT
Task Type Feature Request
Category Web Sites
Status Assigned
Assigned To Jelle van der Waa (jelly)
Sven-Hendrik Haase (Svenstaro)
Giancarlo Razzolini (grazzolini)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 17
Private No

Details

Description:

Feature request of migrating Arch Linux bug tracker to Roundup Issue Tracker, since Flyspray are not updated since 01 May 2009 and Flyspray project is dead.

Roundup Issue Tracker:
http://roundup.sourceforge.net/
This task depends upon

Comment by Dave Reisner (falconindy) - Sunday, 03 July 2011, 14:48 GMT
Perhaps you could detail the reasons why Roundup would make a suitable replacement for Flyspray, as well as devise a migration plan for our 25k bug reports that we'd like to keep as history.
Comment by Thomas Dziedzic (tomd123) - Sunday, 03 July 2011, 14:49 GMT
Never heard of roundup. Is there a specific reason to choose roundup?

Perhaps bugzilla might be a more feature rich bug tracker.
Also, mantis looks nice and simple if we want to go that route.

edit: Bugzilla has a browse feature? ok, I officially vote for bugzilla :)
Comment by Jelle van der Waa (jelly) - Sunday, 03 July 2011, 15:20 GMT
If we move, we want to move to a stable, actively developped bugtracker. It might be nice to have an api so we can automatically send emails to bughunters and keep track of old bugs.
Comment by Greg (dolby) - Sunday, 03 July 2011, 15:44 GMT
Are there any other projects using roundup? I cant find such a list on the roundup site.
Also if Arch was to switch to another bug tracking solution i would prefer one that doesnt make use of js so that is usable with all non graphical web browsers (links,lynx,w3m). Flyspray isnt.
Comment by Jens Adam (byte) - Sunday, 03 July 2011, 15:56 GMT
I'm using roundup on an internal project, it's okay, especially once you've integrated the email gateway... although I have no idea how to migrate stuff.
One example site I know of is http://bugs.dragonflybsd.org/
Comment by Dmitry Korzhevin (dkorzhevin) - Sunday, 03 July 2011, 17:47 GMT
For example, python uses roundup as bugtracking system:
http://bugs.python.org/

Roundup is open-source, has a command-line, web and e-mail interface. It is written in Python and highly customizable. Flyspray project is dead and have some XSS holes (can search at http://www.cvedetails.com/).

Bugzilla very heavy and it will be very dificult for users. Even some basic actions, like fill bug report need bugzilla knowledge and time.

Roundup is very light and actively developed since 2001 year. And as it is used as python main bugtracking system (migrated from bugzilla) says about its stability and developement. (roundup serves 1778410 bug tickets at bugs.python.org) And Roundup e-mail interface will be very useful for bughunters (also it has xml-rpc api: http://roundup.sourceforge.net/docs/xmlrpc.html#client-api)
Comment by Thomas Dziedzic (tomd123) - Sunday, 03 July 2011, 19:01 GMT
"Roundup is open-source, has a command-line, web and e-mail interface. It is written in Python and highly customizable. Flyspray project is dead and have some XSS holes (can search at http://www.cvedetails.com/).";

I would love to be able to access the bug tracker through the command line. Bugzilla has a similar client for the command line.

"Roundup is very light and actively developed since 2001 year. And as it is used as python main bugtracking system (migrated from bugzilla) says about its stability and developement."

If you want to use large projects as examples, note that redhat and linux use bugzilla as their bug tracker.
Comment by Dmitry Korzhevin (dkorzhevin) - Sunday, 03 July 2011, 20:34 GMT
2 Thomas Dziedzic (tomd123)

there's the xmlrpc interface
Comment by Sergej Pupykin (sergej) - Friday, 09 September 2011, 20:36 GMT
Latest bugzilla looks well, uses ajax and has nice feature - it tries to search duplicate bugs while you type summary.
Comment by Anonymous Submitter - Sunday, 11 September 2011, 10:16 GMT
Flyspray to Bugzilla migration perl script : http://www.bailopan.net/blog/?p=108 (found the link by chance).
Comment by Andrea Scarpino (BaSh) - Sunday, 11 September 2011, 11:21 GMT
@Keshav
Thanks, I was writing a python script my self. I like the idea to switch to Bugzilla, I'm proposing this to the others DEVs.
Comment by Jelle van der Waa (jelly) - Monday, 09 January 2012, 20:03 GMT
IIRC , bugzilla has products so you can assign stuff automatically if the maintainer is set .
Comment by Keshav Amburay (the.ridikulus.rat) - Tuesday, 10 January 2012, 07:17 GMT
Someone should change the title of this ticket to something like "Migrate Arch Linux Flyspray bug tracker to another software Bugzilla/Roundup" . Also is there any consensus and if so any timeframe to implement this?
Comment by Jelle van der Waa (jelly) - Tuesday, 10 January 2012, 18:01 GMT
The real issue is that we want to retain the information of the DB. So there should be an option to migrate these DB entries to a new issue tracker ( Bugzilla has such functionality, but you might have to write your own custom script) If someone tried this "migration from flyspray -> bugzilla" the devs could easily adopt it.
Comment by Alexander F. Rødseth (xyproto) - Sunday, 15 January 2012, 21:20 GMT
Mantis (http://www.mantisbt.org/) is also a possibility. Or rolling our own. Or forking flyspray.

Comment by Thomas Dziedzic (tomd123) - Sunday, 15 January 2012, 23:01 GMT
@trontonic, the last 2 options are probably the least favorable imo. We should use already existing solutions instead of looking for more work.
Comment by Keshav Amburay (the.ridikulus.rat) - Monday, 16 January 2012, 07:46 GMT
How about trac or http://thebuggenie.com/features (looks good) or http://sourceforge.net/p/bugvisor or http://www.redmine.org/projects/redmine? I vote for either thebuggenie or bugzilla (the former has better UI - my opinion, but the latter is the most widely used one).
Comment by Keshav Amburay (the.ridikulus.rat) - Monday, 16 January 2012, 07:46 GMT Comment by Jelle van der Waa (jelly) - Monday, 16 January 2012, 11:08 GMT
Whatever bugtracker you name, it needs to support some way of moving the DB and history to a new bugtracker. Moving to a currently not well known/not good supported tracker adds zero benefits.

What version of flyspray are we using now btw?
Comment by Florian Pritz (bluewind) - Monday, 16 January 2012, 11:48 GMT
> What version of flyspray are we using now btw?

I think it's a modified 0.9.9.4. You can take a look yourself https://projects.archlinux.org/vhosts/bugs.archlinux.org.git/tree
Comment by Jelle van der Waa (jelly) - Monday, 30 January 2012, 18:25 GMT
I have then both setup, the posted script doesn't really work, it will need more investigation work. Plus mailed the author of the script.
Comment by Alexander F. Rødseth (xyproto) - Thursday, 09 February 2012, 11:13 GMT Comment by Florian Pritz (bluewind) - Friday, 25 May 2012, 21:17 GMT
I've started on getting the script linked earlier to work for our installation. So far users, comments, closing reasons and attachments are migrated.

Points that haven't yet been addressed:
- Design
- Workflow
- Any important history items that are missing? Examples I don't consider important: changes of assignees, reopen requests, status updates
- Who will take care of bugzilla? (updates, configuration changes, ...)
- maybe more
Comment by Sergej Pupykin (sergej) - Monday, 28 May 2012, 18:36 GMT
Flyspray 0.9.9.7 - 28 May 2012 - released.

http://www.flyspray.org/changelog
Comment by Dmitry Korzhevin (dkorzhevin) - Tuesday, 29 May 2012, 10:15 GMT
2 Sergej:

3 years is too much time for minor release.
Comment by Alexander F. Rødseth (xyproto) - Sunday, 11 November 2012, 21:02 GMT
Perhaps the speed will pick up?
Comment by Greg (dolby) - Saturday, 17 November 2012, 11:44 GMT
Attaching scripts from http://www.bailopan.net/blog/?p=108 which is also mentioned above in case this actually gets through at some point.

"If anyone’s looking to convert from Flyspray to Bugzilla, you can have my hacked-up Perl script. It uses an intermediate table called “mvtab” (type varchar, old_id int, new_id int) as a temporary work table. You need to create this yourself. You also need to manually import products and components (components must have the same names as categories). It’s really a mess but it got the job done, so play with it at your own risk.

flyspray_to_bugzilla.pl
bz_import_mail_settings.pl (tack-on, since the original import didn’t get e-mail settings right)

And here’s the Bugzilla vBulletin binding: vBulletin.pm (goes in Bugzilla/Auth/Verify). You may need to change config files somewhere to use it."
Comment by trusktr (trusktr) - Monday, 25 February 2013, 07:58 GMT
Bugzilla has a new theme coming out soon. Check out the screenshot: http://bugzillaupdate.files.wordpress.com/2011/03/jwilde-bugzillapretty-bug-27-2-2011.png
It could be modified to have Arch colors...

Redmine is simple and clean too, and probably easier to theme for Arch than bugzilla. The Redmine layout seems more Arch-ish than Bugzilla.
Comment by trusktr (trusktr) - Monday, 25 February 2013, 08:17 GMT
What a coincidence, there's already an old Arch Linux theme for Redmine: http://www.redmine.org/projects/redmine/wiki/ThemeArchLinux
Comment by Stefan Tatschner (rumpelsepp) - Sunday, 26 May 2013, 18:26 GMT
Flyspray is not dead at all.
https://github.com/Flyspray/flyspray

It has also a bugtracker: https://bugs.flyspray.org/
Comment by Paul Gideon Dann (giddie) - Wednesday, 11 September 2013, 15:25 GMT
+1 for Redmine or Chili (a derivative). I set up Redmine at work, and it's been great. It feel more user-friendly to me than Bugzilla.
Comment by Pablo Lezaeta (Jristz) - Wednesday, 23 April 2014, 18:30 GMT
Looking the github page fryspray look near death, and I think is better or change bugtracker or fork fryspray as an arch project.
unless you not want become the uptream of a death project
Comment by Pablo Lezaeta (Jristz) - Wednesday, 16 September 2015, 04:13 GMT Comment by Jerome Leclanche (Adys) - Thursday, 17 September 2015, 21:09 GMT
There once was a tracker named flyspray
Their devs quickly ran out of money
They put in some ads
Made everyone sad
And all their users went away

Commit's been reverted. The discussion was ridiculous though. Looks like some serious incompetence.
Comment by Leonidas Spyropoulos (inglor) - Saturday, 21 October 2017, 07:41 GMT
I tested a clean install of bugzilla on a local VM (branch 5.0-stable) through apache. The siege output of the command:
$ siege -c 5 -b -t30s
...
Lifting the server siege...
Transactions: 15150 hits
Availability: 100.00 %
Elapsed time: 299.62 secs
Data transferred: 62.53 MB
Response time: 0.16 secs
Transaction rate: 50.56 trans/sec
Throughput: 0.21 MB/sec
Concurrency: 7.99
Successful transactions: 15158
Failed transactions: 0
Longest transaction: 1.06
Shortest transaction: 0.00

Comment by Jelle van der Waa (jelly) - Saturday, 21 October 2017, 10:21 GMT
Neat, but does it contain any data? ;-)

Currently I've been trying to migrate our current bugtracker data to bugzilla and somehow fit in our packages as products or components. Migrating and fitting bugzilla in our environment is what is withholding us at the moment.
Comment by Leonidas Spyropoulos (inglor) - Sunday, 22 October 2017, 00:04 GMT
And the results when running the 5.1.1 dev branch on a psgi app using plack:
$ siege -c 5 -b -t30s
...
Lifting the server siege...
Transactions: 94009 hits
Availability: 100.00 %
Elapsed time: 299.61 secs
Data transferred: 10755.75 MB
Response time: 0.03 secs
Transaction rate: 313.77 trans/sec
Throughput: 35.90 MB/sec
Concurrency: 7.98
Successful transactions: 94009
Failed transactions: 0
Longest transaction: 0.09
Shortest transaction: 0.00
Comment by Leonidas Spyropoulos (inglor) - Monday, 23 October 2017, 07:32 GMT
Finally manged to get it to work over nginx using plack as FCGI provider (branch 5.1.1)
$ siege -c 5 -b -t30s
...
Lifting the server siege...
Transactions: 121892 hits
Availability: 100.00 %
Elapsed time: 299.74 secs
Data transferred: 13689.04 MB
Response time: 0.02 secs
Transaction rate: 406.66 trans/sec
Throughput: 45.67 MB/sec
Concurrency: 7.98
Successful transactions: 121892
Failed transactions: 0
Longest transaction: 0.15
Shortest transaction: 0.00

attached is the nginx.conf

And you need to start the backend plackup with:
plackup -S /tmp/bugzilla.sock -s FCGI -E development /srv/http/bugzilla/app.psgi
(add -D for daemon)

(When I was trying to run bugzilla as user `bugs` and socket on /run/bugzilla.sock I was getting permission denied so I switched to /tmp )
Comment by Florian Pritz (bluewind) - Tuesday, 13 February 2018, 11:07 GMT
Quick status update: I've rewritten the migration script using bugzilla's migration class. It's still WIP and there are some missing parts, but I got distracted by uni and forgot what is missing. I'll try to get it upstreamed eventually. Until then it can be found here: https://git.server-speed.net/users/flo/bugzilla/log/?h=flyspray
Comment by Jelle van der Waa (jelly) - Sunday, 28 April 2019, 09:56 GMT
Some thoughts / features we could really use in bugzilla:

Automatic assignment requires products/component to be set in bugzilla so you can file for a component. Also auto removal of bugs when a package is removed from the repos or notification to a selected group of wranglers. Updating the status of the bug report when a package is updated? Auto triggering the question if it was fixed. Auto closing bugs based on commit messages?
Comment by Florian Pritz (bluewind) - Sunday, 28 April 2019, 11:43 GMT
Status update:
- BLOCKER: waiting for bugzilla upstream to release 6.0 and/or fix MySQL 8.0 compatibility (unquoted table name "groups" is now a reserved keyword)
- TODO: set up a demo environment on a public VM (ansible playbook)
+ package bugzilla and all dependencies (CPAN modules)
+ make available to arch staff (not general public yet)
+ run migration script
+ disable email delivery for all accounts by default so people don't get spamed unless they opt in by enabling delivery
+ verify that all previously private bugs are still private and user groups work as expected
+ gather feedback from arch staff
+ fix bugs/implement feedback
- TODO: set up production environment
+ disable flyspray
+ perform migration
+ set up redirect from old bug URLs to new URLs
+ DONE

Since I don't want to spam everyone who's interested in high-level info about bugzilla, more details can be found in the related kanboard ticket: https://kanboard.archlinux.org/public/task/54/7dd7510424e4229247e8e0b90bf43e1553fce86cdf8475b60edc956ed5a8 I'll try to keep this updated whenever something happens. If anyone wants to help out, feel free to drop by in #archlinux-devops and ping me.

Regarding all those features suggested by Jelle, I think we should implement those in a bug wrangler bot. Bugzilla provides a nice REST API so this shouldn't be too difficult and having it in a bot allows us to keep our bugzilla installation simpler and more stable.
Comment by Florian Pritz (bluewind) - Thursday, 29 August 2019, 15:25 GMT
I'd like to orphan this project due to lack of time/interest. If anyone wants to continue working on migrating to bugzilla, please look at the kanboard ticket linked in my previous comment. I'll happily answer questions if you have any, but I think the kanboard task should contain the most important information.

If you want to reevaluate other bug tracking solutions or work towards migrating to something else, feel free to do so, obviously. That being said, the bugzilla migration is relatively close to being done. Just keep that in mind if you decide to invest time in something else. Also you can likely reuse a lot of my migration script for other software as well. At least the SQL statements that read the flyspray database and the code that converts it to something useful.

Loading...