FS#23194 - [pkgstats] How to obtain the 'pkgstats' data programmatically

Attached to Project: Arch Linux
Opened by Xavion (Xavion) - Tuesday, 08 March 2011, 23:17 GMT
Last edited by freswa (frederik) - Thursday, 10 September 2020, 13:07 GMT
Task Type Support Request
Category Packages: Extra
Status Closed
Assigned To Pierre Schmitz (Pierre)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

What is the URL of the 'pkgstats' database? If it's kept private, is the 'pkgstats' data available in CSV or similar format online yet?

If you want to know why I'm planning to obtain its data programmatically, read my recent forum posting: https://bbs.archlinux.org/viewtopic.php?id=114730
This task depends upon

Closed by  freswa (frederik)
Thursday, 10 September 2020, 13:07 GMT
Reason for closing:  Fixed
Comment by Pierre Schmitz (Pierre) - Friday, 11 March 2011, 07:28 GMT
ATM the database is not available to the public. But I want to provide those data in future in a machine readable format.
Comment by Xavion (Xavion) - Friday, 11 March 2011, 09:32 GMT
It'll be good if you can output the data to a CSV file that's publicly available and updated everyday. For the moment, my "Popular Packages" script grabs the data from the HTML webpage of 'pkgstats'. If you're interested in checking it out, here is its AUR webpage: http://aur.archlinux.org/packages.php?ID=47257
Comment by Greg (dolby) - Monday, 15 October 2012, 08:01 GMT
Is this really relevant to have in the bug tracker?
Comment by Xavion (Xavion) - Tuesday, 16 October 2012, 04:43 GMT
I don't know why it wouldn't be: this task is a "support request".
Comment by Kyle Keen (keenerd) - Saturday, 17 November 2012, 01:52 GMT
Plenty of us use pkgstats in machine-readable format already. It is a very small piece of Beautiful Soup to convert to a CSV. If you want a daily CSV, run a cron job and host the file somewhere. While it is nice to have features integrated into the real site, it is pretty easy to do yourself. So don't get discouraged if this is closed, do it yourself instead.

Edit, example python attached.

Loading...