FS#16630 - [pacman] bash completion

Attached to Project: Pacman
Opened by J Slam (jslam) - Wednesday, 14 October 2009, 03:07 GMT
Last edited by Dan McGee (toofishes) - Wednesday, 26 May 2010, 13:53 GMT
Task Type Bug Report
Category Scripts & Tools
Status Closed
Assigned To Aaron Griffin (phrakture)
Dan McGee (toofishes)
Allan McRae (Allan)
Architecture All
Severity Low
Priority Normal
Reported Version 3.3.3
Due in Version 3.4.0
Due Date Undecided
Percent Complete 100%
Votes 2
Private No


In the function `_installed_pkgs()' in the file `/etc/bash_completion.d/pacman`, `ls' is used to get a list of packages for completion. On my machine (and many others I'm sure), ls is aliased to 'ls --color'. This results in the names of the packages including some color escape sequences, so that they don't complete properly. A possible remedy (and what I've done in the meantime) is just to change `ls' to `ls --color=never'.
This seems like a problem that might come up in other places as well, though I haven't bothered checking.

Additional info:
* package version(s)
This task depends upon

Closed by  Dan McGee (toofishes)
Wednesday, 26 May 2010, 13:53 GMT
Reason for closing:  Implemented
Additional comments about closing:  Commit 80f7c170
Comment by Rick Chen (stuffcorpse) - Wednesday, 14 October 2009, 08:42 GMT
You should use 'ls --color=auto' for your alias instead.
Comment by (gog) - Wednesday, 14 October 2009, 09:40 GMT
Just change every mention of ls to /bin/ls in /etc/bash_completion.d/pacman

Same goes for grep and echo. (I don't know who'd alsias echo, but still).

As a sidenote, I recommend removing this completion from the pacman package. It's very slow and has the tendency of lagging/locking up the terminal when trying to complete pacman -S.
Comment by Dan McGee (toofishes) - Wednesday, 14 October 2009, 15:40 GMT
Patches welcome.
Comment by (gog) - Wednesday, 14 October 2009, 20:05 GMT
I'm guessing that you mean a patch for improving the completion's performance, but just in case here's a patch for hardpathing everything in pacman's completion.
Comment by Aaron Griffin (phrakture) - Wednesday, 14 October 2009, 20:29 GMT
As an FYI, this is the reason it is preferred to use full paths to system utilities in scripts.
Comment by Dan McGee (toofishes) - Thursday, 15 October 2009, 00:13 GMT
Actually I meant the hardcoded paths thing first, so this is perfect. If you want to address the speed too, that would be awesome. Can I get your real name so I can attribute it to you when I push this patch to git?
Comment by Allan McRae (Allan) - Wednesday, 04 November 2009, 11:35 GMT
ping gog! We can also not attribute you for the patch if you would prefer. Just let us know.
Comment by Andres Perera (pwd) - Wednesday, 16 December 2009, 11:39 GMT
Sorry. I'll try to not loose passwords/get banned anymore.

The solution was to use less GNU and more pacman:

enabled_repos=$(/bin/grep '\[' /etc/pacman.conf | /bin/grep -v -e 'options' -e '^#' | /bin/tr -d '[]')
available_pkgs=$(for r in $enabled_repos ; do /bin/echo /var/lib/pacman/sync/$r/*; done)
for i in $available_pkgs; do j=${i##*/}; /bin/echo ${j%-*-*} >/dev/null; done
real 0m10.512s
user 0m3.843s
sys 0m7.363s

/usr/bin/pacman -Sql >/dev/null
real 0m0.152s
user 0m0.057s
sys 0m0.093s

installed_pkgs=$(/bin/ls /var/lib/pacman/local)
for i in $installed_pkgs; do /bin/echo ${i%-*-*} >/dev/null; done
real 0m0.613s
user 0m0.210s
sys 0m0.413s

/usr/bin/pacman -Qq >/dev/null
real 0m0.029s
user 0m0.013s
sys 0m0.013s

installed_groups=$(find /var/lib/pacman/local -name desc -exec /bin/sed -ne '/%GROUPS%/,/^$/{//d; p}' {} \; | /usr/bin/sort -u)
for i in $installed_groups; do /bin/echo ${i%-*-*} >/dev/null; done
real 0m3.208s
user 0m2.690s
sys 0m0.523s

pacman -Qg | sed 's|\s.*||' | uniq >/dev/null
real 0m0.072s
user 0m0.060s
sys 0m0.023s

Another fix: _available_groups was dead code since it wasn't refered to for -Sg completion or in any other place, now fixed. Other minor changes in addition to the hardpaths. In conclusion, it's actually usable now and _much_faster.

No diff since there are so many changes, including whitespace: http://pastebin.com/m6bc2c423
Comment by Dan McGee (toofishes) - Wednesday, 16 December 2009, 11:41 GMT
Um...why whitespace, why no diff? That is the only way we take patches.
Comment by Andres Perera (pwd) - Wednesday, 16 December 2009, 12:07 GMT
> why whitespace
Because it was unreadable.

If that's a non-factor ( :/ ), here's a diff, indent intact.
Comment by Andres Perera (pwd) - Wednesday, 16 December 2009, 12:35 GMT
Non-important changes: forgot a few test [=[['s
Comment by Andres Perera (pwd) - Thursday, 17 December 2009, 01:06 GMT
Last patch broke arguments starting with '--' (e.g. --help). Use pacman-FIXED-completion.diff.
Comment by Andres Perera (salmon) - Thursday, 14 January 2010, 21:18 GMT
Major bugs corrected from the original:

* $arglen (line 195), $i (40), $str (126) are _undeclared_ as local. Meaning that after sourcing bash_completion, using a function that deal with these vars will cause problems. This is specially important since $i and $str are common var names used in short functions. To reproduce: unset arglen i str, . /etc/bash_completion, type `pacman -Q' (or any other operand) and press tab, echo $arglen $str $i.

* Function rem_selected does not have a _ prepended, unlike all functions in bash_completion.d. The function also expands an array 3 times (that at moments can have hundreds of entries) and breaks file names with spaces or newlines.

* It uses ${COMP_WORDS[COMP_CWORD]} instead of _get_cword. This makes compgen fail when there are comp. words with spaces or newlines.

* Both _pacman and _makepkg call rem_selected even when it's the 1st word or when the completion is in the middle of an operand.

* Both _pacman and _makepkg cannot deal with completing special double options (pacman -Qii) and delete them from the list without discrimination. These functions cannot complete combination operands; if you type -Q it will skip to the next word instead of attempting to complete the current.

Departures from the previous patches:

Now commands are using `command cmd_name' instead of hardpaths. This is the canonical way of using commands in Bash completions; it doesn't assume the path of the binary.

The new version [1] is 203 lines, as opposed to 365. Every function is completely different, so I don't know what good would a patch [2] do. All noted bugs and performance issues have been corrected.

The other patch [3] is ontop 011410 and allows removing options when completing short operands. That means that -S will not return -SS as a completion match. This should've been included in the original but was overlooked.

TODO: Add support for detecting conflicting options; e.g., _makepkg shouldn't complete --force when -g is in effect.
Comment by Andres Perera (salmon) - Thursday, 14 January 2010, 22:47 GMT
This patch is ontop 011410. It includes mid_option_fix and fixes some weak globs that didn't consider short operands with spaces between them, along with a small correction to _arch_rem_selected, since it was touching operands instead of just files/packages only.
Comment by Andres Perera (salmon) - Friday, 15 January 2010, 22:39 GMT
Patch [1] on top glob_fixes

Fixes from the original:
* Added $default to the complete options. This fixes completions when there's a redir/pipe character (>|<<>). The original pacman completion did not interpret these correctly.

Changes from 011410:
* makepkg completion should be faster since it's only processing lists depending on $cur, instead of doing every array.
* Added missing double option to pacman -S (yy for sync).
* Switched order of matches so that it prioritizes long options unless $cur is a short combined operand.
Comment by Andres Perera (salmon) - Sunday, 17 January 2010, 08:51 GMT
Fixes from original:

* _filedir is an old bash 2.x hack that is getting deprecated [1-2]. Removed where possible and added -o plusdirs as a replacement. This also happens to fix bugs where _pacman wouldn't complete *.pkg.tar.gz in files without periods before the extension, since it was actually going for *.*.pkg.tar.gz

* Changed complete vars ($filenames, etc.) to their actual options. The people at bash_completion are pulling them for similar reasons.

[1] http://wiki.debian.org/Teams/BashCompletion/Proposals/Roadmap
[2] http://wiki.debian.org/Teams/BashCompletion/Proposals/DropBash2Support

Diff on top the last one.
Comment by Dan McGee (toofishes) - Wednesday, 20 January 2010, 04:01 GMT
So I would be happy to look at this closer if we can just get one diff. If at all possible, this would be a git diff against the lastest version of this file in master. If you're unsure how to get started, check out the Development section located here: http://www.archlinux.org/pacman/
Comment by Andres Perera (salmon) - Wednesday, 20 January 2010, 05:12 GMT
Using absolute paths isn't the fix, toofishes; you need `command' instead.

It's a 96%, but here's the git diff anyways:
Comment by Dan McGee (toofishes) - Wednesday, 05 May 2010, 16:34 GMT
I'm not touching this with your license change, sorry. This codebase is staying GPL2+. If that isn't a sticking point (which I hope it isn't), then let's see if we can get this up to speed with the current file. I think we've had --print changes and package extension changes since the most recent patch.
Comment by Allan McRae (Allan) - Wednesday, 26 May 2010, 11:23 GMT Comment by Dan McGee (toofishes) - Wednesday, 26 May 2010, 13:53 GMT
Yes it should be.