FS#40805 - [pacman] utf8 file names

Attached to Project: Pacman
Opened by Kyle Keen (keenerd) - Wednesday, 11 June 2014, 20:01 GMT
Last edited by Allan McRae (Allan) - Wednesday, 25 June 2014, 13:07 GMT
Task Type Bug Report
Category General
Status Closed
Assigned To Allan McRae (Allan)
Architecture All
Severity Low
Priority Normal
Reported Version 4.1.2
Due in Version 4.2.0
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Summary and Info:

Pacman cannot install packages with utf8 file names if the system locale does not support utf8. The error message indicates this is a problem with libarchive, but pacman might be able to work around it. As far as pacman is concerned paths should be mere arrays of bytes and not do any locale interpretation.

One serious implication of this - our build tools use the C locale. We can't build packages with [make]depends that have utf8 file names.

Steps to Reproduce:

LANG=C pacman -S arch-wiki-docs
error: error while reading package /var/cache/pacman/pkg/....: Pathname can't be converted from UTF-8 to current locale.
error: failed to commit transaction (libarchive error)
Errors occurred, no packages were upgraded.

Or a minimal test case PKGBUILD (from felixonmars)
https://paste.xinu.at/9VZII/

A side note, the "converted to the current locale" part worries me a bit. For example if my system locale is a RTL language, does this mean all the file names will get reversed? If the paths actually are modified during install, changing the locale before an update or removal could cause problems.
This task depends upon

Closed by  Allan McRae (Allan)
Wednesday, 25 June 2014, 13:07 GMT
Reason for closing:  Fixed
Additional comments about closing:  git commit 537a335c
Comment by Allan McRae (Allan) - Thursday, 12 June 2014, 01:12 GMT
Even worse with git version... No files get installed.
Comment by Allan McRae (Allan) - Thursday, 12 June 2014, 01:37 GMT
Build the test package using devtools. I see no issue when I do that.

I'll assume that arch-wiki-docs was not built with devtools... (and it does not work here).
Comment by Kyle Keen (keenerd) - Thursday, 12 June 2014, 01:45 GMT
Test package might work, depending on locales used.

LANG=en_US.utf8 makepkg -f
LANG=en_US.utf8 pacman -U testutf8-0-1-any.pkg.tar.xz
(no problems)

LANG=en_US.utf8 makepkg -f
LANG=C pacman -U testutf8-0-1-any.pkg.tar.xz
error ... can't be converted from UTF-8 to current locale

LANG=C makepkg -f
bsdtar: etc/测试: Can't translate pathname 'etc/测试' to UTF-8
LANG=[either] pacman -U testutf8-0-1-any.pkg.tar.xz
(no problems)
Comment by Allan McRae (Allan) - Thursday, 12 June 2014, 01:47 GMT
Yes - this is why we force LANG=C in devtools (which should be used for all packages for distribution).
Comment by Allan McRae (Allan) - Wednesday, 25 June 2014, 13:06 GMT
arch-wiki-docs issue filed:  FS#40971 

Loading...