FS#55322 - [vte] [vte3] [konsole] missing REP support, required for ncurses >= 6.0+20170827-1

Attached to Project: Arch Linux
Opened by Chih-Hsuan Yen (yan12125) - Monday, 28 August 2017, 17:12 GMT
Last edited by Antonio Rojas (arojas) - Sunday, 04 February 2018, 21:42 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Jan de Groot (JGC)
Jan Alexander Steffens (heftig)
Antonio Rojas (arojas)
Felix Yan (felixonmars)
Architecture x86_64
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 2
Private No

Details

Description:
After upgrading ncurses to 6.0+20170827-1, htop layout is broken. Attached files are screenshots for old (6.0+20170527-1) and new (6.0+20170827-1) ncurses. The problem is reproducible on aur/qterminal-git, konsole and xfce4-terminal, so I guess it's not a terminal emulator issue.

Additional info:
ncurses 6.0+20170827-1
htop 2.0.2-2
qterminal-git 0.7.1.32.gae7ecee-1 (built from AUR)
konsole 17.08.0-1
xfce4-terminal 0.8.6-1

Steps to reproduce:
1. Upgrading ncurses to 6.0+20170827-1
2. Run `htop`
This task depends upon

Closed by  Antonio Rojas (arojas)
Sunday, 04 February 2018, 21:42 GMT
Reason for closing:  Fixed
Comment by Thomas Dickey (dickey) - Monday, 28 August 2017, 20:50 GMT
It's not only a terminal emulator issue, but an FAQ -
http://invisible-island.net/ncurses/ncurses.faq.html#xterm_generic
Comment by Eli Schwartz (eschwartz) - Monday, 28 August 2017, 21:12 GMT
After receiving the testing update to ncurses in my daily -Syu, htop works fine for me. I am using guake FWIW.
Comment by Evangelos Foutras (foutrelis) - Monday, 18 September 2017, 13:08 GMT
This also broke for me starting with ncurses 20170729.

While `htop` works fine in xfce4-terminal with my regular locale (en_US.UTF-8), the text is misaligned with some characters missing when using the POSIX locale (i.e.: `LANG=C htop`). More or less the same behavior as seen in OP's screenshots.
Comment by Thomas Dickey (dickey) - Monday, 18 September 2017, 19:59 GMT
It does not depend upon the locale.
It is a limitation of the terminal emulator, which should not set TERM=xterm.
Comment by Evangelos Foutras (foutrelis) - Monday, 18 September 2017, 22:44 GMT
In my (limited) tests, it does depend on the locale (POSIX has the issue, en_US.UTF-8 does not).

It's reproducible in both xfce4-terminal and gnome-terminal and they both set TERM=xterm-256color.

The FAQ entry linked in your first comment is from 2012; why hadn't this been an issue before 20170729?
Comment by Thomas Dickey (dickey) - Monday, 18 September 2017, 23:12 GMT
The FAQ was updated in 2017, as noted near the end of that item.
The change was made to ensure proper testing of this:
https://invisible-island.net/ncurses/NEWS.html#t20170827
(and taking the limitations of other terminals into account for
the "xterm" entries has never been a factor in updating them).
Comment by Eli Schwartz (eschwartz) - Monday, 18 September 2017, 23:25 GMT
@foutrelis,

I am not even sure why that is a question. xterm is allowed to do whatever it wants and make whatever wild changes it wants, even, and if other terminal emulators break it is legally that other terminal emulator's fault for using xterm when they are not, in fact, xterm. Does your testcase work in xterm? ;) If so, we can assume it is in fact an xfce-terminal (or rather, vte) bug, not an ncurses bug...
Comment by Thomas Dickey (dickey) - Monday, 18 September 2017, 23:34 GMT
I phrased it as a "limitation", since the feature is marked in VTE source as unsupported.
Three programs (ncurses, vttest and xterm) expect a given behavior for this feature,
and it was not desirable to have an ncurses feature depend on a little-publicized xterm feature.
So... seeing that it was available in all xterm configurations, I added it.
That's been my policy for more than 20 years.
Comment by Evangelos Foutras (foutrelis) - Tuesday, 19 September 2017, 00:09 GMT
Breaking existing behavior and placing the blame on popular terminal emulators because 1) you are technically correct and 2) don't care about their limitations, doesn't sit well with me.

I appreciate your input in this bug and the FAQ does indeed explain the issue. The part that's still confusing to me is that it works fine with the en_US.UTF-8 locale.

@Eli Nobody uses xterm so it's irrelevant whether or not it works on that.
Comment by Evangelos Foutras (foutrelis) - Tuesday, 19 September 2017, 00:25 GMT
Looking at VTE's bug tracker, it seems they are going to adapt to the ncurses change. [1] There is also a bug filed for Konsole. [2]

I guess I was wrong to insist that ncurses should keep compatibility with these terminal emulators, and it's probably the other way round.

As long as the issue is fixed, I'm happy. Still wonder why the locale made a difference though.

[1] https://bugzilla.gnome.org/show_bug.cgi?id=787701
[2] https://bugs.kde.org/show_bug.cgi?id=384620
Comment by Thomas Dickey (dickey) - Tuesday, 19 September 2017, 00:48 GMT
I should have mentioned that the 2017/08/27 fix addresses the locale part.
Comment by Thomas Dickey (dickey) - Tuesday, 19 September 2017, 01:05 GMT
To clarify, the initial bug-fix at the end of July fixed one aspect,
but introduced another problem which is fixed at the end of August.

http://lists.gnu.org/archive/html/bug-ncurses/2017-08/threads.html#00039

Presumably Arch's packagers keep an eye on the bug-reporting list,
and will update their package :-)
Comment by Bartłomiej Piotrowski (Barthalion) - Tuesday, 19 September 2017, 08:01 GMT
We do have 20170902 now. I don't quite understand why Unicode locales are not affected even with this version (or is the fix exactly that UTF-8 works fine but C not?)
Comment by Bartłomiej Piotrowski (Barthalion) - Tuesday, 19 September 2017, 08:03 GMT
Okay, heftig explained that repeating multibyte char doesn't work. I see now.
Comment by Jan de Groot (JGC) - Thursday, 05 October 2017, 21:25 GMT
vte3 0.50.1 in testing supports REP now.

KDE bug is still open, so konsole will take more time.
I'm not interested in fixing vte 0.28.x, as it's no longer maintained upstream and the patch to 0.50.1 doesn't apply to it (vte has been rewritten in C++ in the meanwhile).
Comment by Antonio Rojas (arojas) - Sunday, 04 February 2018, 21:42 GMT
konsole fixed in 17.12.1-2

Loading...