FS#63815 - [ttf-joypixels] Replaces normal numbers with emojis.

Attached to Project: Community Packages
Opened by Vista1nik (vista1nik) - Wednesday, 18 September 2019, 07:28 GMT
Last edited by Maxim Baz (maximbaz) - Tuesday, 25 February 2020, 22:04 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Maxim Baz (maximbaz)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 2
Private No

Details

Description:
Package replaces normal numbers with emojis in Chromium.

Additional info:
* package version(s): 5.0.2

Steps to reproduce:
1. Install `ttf-joypixels 5.0.2-2`
2. Link it. `sudo ln -sf ../conf.avail/75-joypixels.conf /etc/fonts/conf.d/75-joypixels.conf`
3. Open Chromium on any site. (e.g. https://github.com/)
This task depends upon

Closed by  Maxim Baz (maximbaz)
Tuesday, 25 February 2020, 22:04 GMT
Reason for closing:  Implemented
Additional comments about closing:  Implemented suggestions for .install notice and the example conf file
Comment by Maxim Baz (maximbaz) - Sunday, 03 November 2019, 11:35 GMT
Hi there! The package does not place the config in `/etc/fonts/conf.d` precisely because it is not intended to be a config that works out of the box for everyone. It does work well for many people (myself included), but seems like in your case a different combination of fonts and configs leads to JoyPixels being the font that is selected first to render numbers.

Don't symlink the file, but create your own `/etc/fonts/conf.d/75-joypixels.conf` and use the file in `conf.avail/` as a reference.

Fontconfig is hard, I know! If you have ideas how I can make this config better (to work for more people out of the box), don't hesitate to shoot me an email.

On my side, I will update the comments and install message to make it clearer that the provided `75-joypixels.conf` is first and foremost an example file to use as a reference.
Comment by Setpill (setpill) - Thursday, 20 February 2020, 14:58 GMT
The .install notice did not fully make me understand what might happen. That it might overrule other emoji fonts, sure, I had no problem with that. I have none installed. But that it might cause regular symbols to become replaced by emojis, that I didn't expect. In a way it's an upstream bug that an emoji font even contains glyphs for numbers... But regardless, some clarification on the .install notice might be helpful.

Would it be a solution to, `mode=delete` the non-emojione emoji fonts instead of `mode=replace` them? That way, they are simply removed from consideration and non-emoji fonts are allowed to fill the number glyphs before the fallback to JoyPixels kicks in for any unfilled emoji glyphs.
Comment by Maxim Baz (maximbaz) - Thursday, 20 February 2020, 16:25 GMT
> Would it be a solution to, `mode=delete` the non-emojione emoji fonts instead of `mode=replace` them?

Those replacements are actually meant not for getting rid of other emoji fonts, but to "fix" apps and websites that hardcode some very specific emoji fonts - this is actually quite common that a website would just declare 'Apple Color Emoji' and nothing else, and in those cases JoyPixels will never be selected as an emoji font.

I would guess that the reason you see numbers rendered as emojis is actually caused by the second block of 75-joypixels.conf, where it is appended to sans, serif, sans-serif and monospace. Try to remove that block and see if it helps you.

> But regardless, some clarification on the .install notice might be helpful.

I'm all for making it more clear! The current message doesn't actually mention that it conflicts with other emoji fonts, it just says "with other fonts", but I can see why you would understand it like that.

Do you have any suggestions? I'm thinking of something along the lines below.

post_install() {
echo
echo " See /etc/fonts/conf.avail/75-joypixels.conf for how to enforce rendering emojis via JoyPixels"
echo " It contains various techniques, some of which can cause issues like rendering numbers as emojis"
echo " Copy the file to /etc/fonts/conf.d/ and modify as needed to make it work for your own setup."
echo
}


If you can confirm that the issue in your case is indeed caused by that second block in 75-joypixels.conf, I can add a comment warning there as well. Let me know.
Comment by Setpill (setpill) - Thursday, 20 February 2020, 19:22 GMT
That explanation is a lot clearer, thanks.

And no, it's not the second block; in the case of the login page on outlook.live.com it is caused by the following block:

```
<match target="pattern">
<test qual="any" name="family"><string>Segoe UI Symbol</string></test>
<edit name="family" mode="assign" binding="same"><string>JoyPixels</string></edit>
</match>
```
Comment by Maxim Baz (maximbaz) - Thursday, 20 February 2020, 19:32 GMT
Interesting... I don't know fontconfig well enough, but I will make another guess :) Could you check in browser devtools what is the font-family defined for that block where you see numbers rendered as emojis? My guess is that the situation is the following:

font-family: font1, font2, Segoe UI Symbol, font3, font4

And on your computer you don't have anything that resolves to font1 and font2, so _everything_ gets rendered as Segoe UI Symbol (i.e. as JoyPixels).

And if I'm right, as soon as you install (or define aliases) for at least one font that is preceding Segoe UI Symbol, it will be used to render numbers, and JoyPixel will only render the remaining glyphs that are not part of those fonts.
Comment by Setpill (setpill) - Thursday, 20 February 2020, 19:32 GMT
If I understand the logic correctly, it would not make sense for the 2nd block to be the cause. The joypixels font is *appended* to those families, so would be very last font that is checked for a glyph. Any fonts that are already in the family would take precedence. And since the glyphs are the regular numbers, these would be present already in any other fonts in those families.
Comment by Setpill (setpill) - Thursday, 20 February 2020, 19:36 GMT
The situation is indeed like you said; the font-family of the element in question is:

"Segoe UI Webfont", -apple-system, "Helvetica Neue", "Lucida Grande", "Roboto", "Ebrima", "Nirmala UI", "Gadugi", "Segoe Xbox Symbol", "Segoe UI Symbol", "Meiryo UI", "Khmer UI", "Tunga", "Lao UI", "Raavi", "Iskoola Pota", "Latha", "Leelawadee", "Microsoft YaHei UI", "Microsoft JhengHei UI", "Malgun Gothic", "Estrangelo Edessa", "Microsoft Himalaya", "Microsoft New Tai Lue", "Microsoft PhagsPa", "Microsoft Tai Le", "Microsoft Yi Baiti", "Mongolian Baiti", "MV Boli", "Myanmar Text", "Cambria Math"

None of these are installed on my system.
Comment by Maxim Baz (maximbaz) - Thursday, 20 February 2020, 19:40 GMT
OK, good to know!

To show you an example from my dotfiles, here's how I create a font alias that would help me (I think) with this particular page:

https://github.com/maximbaz/dotfiles/blob/6880b36107852ef415bd172e76e128f0a7e3c6dd/packages/30-font-aliases.conf#L58-L63

But I don't think it's something I can fix in the joypixels package, they won't remove numbers, and technically the browser _is_ rendering this page for you with the first available font.

Do you agree if I close this bug with changing the .install notice as above?
Comment by Setpill (setpill) - Thursday, 20 February 2020, 19:44 GMT
I am experimenting a bit with the fontconfig, and will report my findings. Because I find that emoji are still rendered correctly on that page even when removing the JoyPixels substition of Segoe UI Symbol, so I want to see how far I can "dress down" the config before it breaks.
Comment by Maxim Baz (maximbaz) - Thursday, 20 February 2020, 19:48 GMT
I think that happens because for emoji glyph the browser goes through that entire font list, doesn't find any font that can render it, and then (somehow) starts looking on your computer for any font that can render it, and stumbles upon JoyPixels. But it's pure luck, emojis like this one for example ( 😄 ) are present in many fonts, so they will most probably not be rendered by JoyPixels unless you do alias Segoe UI Symbol => JoyPixels.

But do have a look and let me know if you find something. And thanks for your help!
Comment by Setpill (setpill) - Thursday, 20 February 2020, 20:46 GMT
All tests performed on the input box found on https://outlook.live.com/owa/?nlp=1

Commented out entirety of 75-joypixels.conf. Outcomes:
* Emojis are shown in joypixels font in firefox
* Emojis are shown in different font (b&w noto emoji font) in chromium

NB: I do not have noto-fonts-emoji installed. I assume it's built into chromium. From now on I will only test in chromium.

Uncommented "emoji" family joypixels assign. Outcome:
* No change (noto emojis)

Uncommented "sans" family joypixels append. Outcome:
* No change (noto emojis)

Uncommented "serif" family joypixels append. Outcome:
* No change (noto emojis)

Uncommented "sans-serif" family joypixels append. Outcome:
* Emojis are shown in joypixels font in chromium

So "sans-serif" is used by chromium as a fallback even though it's not specified by the website. To confirm:

Commented out everything again and added a new "Cambria Math" replacement with JoyPixels with the following:

<match target="pattern">
<test name="family"><string>Cambria Math</string></test>
<edit name="family" mode="assign" binding="same"><string>JoyPixels</string></edit>
</match>

Outcome:
* Numbers are shown as joypixels emojis

This proves that none of the font-families defined in that page are available on my computer (Cambria Math is the very last entry in the font-family of that element). Yet, the browser falls back to sans-serif. So it seems that having joypixels as fallback for the basic font families, combined with the different "emojione" associations, might be a sane default. Then, one might want to add the rejection of Symbola if both are installed, though, considering it's only available in the AUR, might not need to be a usecase supported in an official repository package.

With regards to the emoji you posted above, it also shows as joypixels with my stripped down fontconfig on chromium, but as a text emoji on firefox... Odd.
Comment by Maxim Baz (maximbaz) - Thursday, 20 February 2020, 22:56 GMT
Thanks, interesting, and as usual with fontconfig, odd 🙂

I do think that each app has its own internal rules of how to fallback if all defined fonts fail, that's how you get different rendering of ( 😄 ) in Chromium and Firefox, and the fact that your test on outlook login page worked in Chromium might not indicate that you will have the same results in Firefox or other apps...

> So it seems that having joypixels as fallback for the basic font families, combined with the different "emojione" associations, might be a sane default.

I intentionally decided to refrain from providing _any_ default in the package, because it's still very dependent on user specific setup and a list of available fonts. I will let users create their own /etc/fonts/conf.d/75-joypixels.conf as they see fit for their own setup.

As for the example 75-joypixels.conf that is part of the package, I still think it makes sense to include as many aliases to other emoji fonts as possible because of the example I shared earlier. If after changing the install notice I see that the provided 75-joypixels.conf is still being mistaken for a "default" config, I'd rather move it to /usr/share/ttf-joypixels/examples/ to illustrate that point, how ever non-standard that might be for a font package.

Fair point though about rejection of Symbola specifically, on the one hand it's just an example and I think a quite popular one, on the other hand, maybe you are right and it's too specific. I'll think about it 👍
Comment by Setpill (setpill) - Friday, 21 February 2020, 09:55 GMT
I don't really see why providing a default config that sets joypixels up as a fallback font would be bad. Sure, some people might want to tweak/override it depending on local setup, but if it can be made to work well enough "out of the box" without adverse effects, I don't see a compelling reason not to do so.

With that in mind, it would probably be a bad idea to have the overwriting of the different Apple/Segoe/Noto/whatever emoji/symbol fonts and especially the rejection of the Symbola font in the default config, as it seems very tied to the assumption that you want to have joypixels over everything else rather than just having it available on your system.

I have an idea for a sane default, is it possible for me to open a pull request or whatever on an official arch package?
Comment by Maxim Baz (maximbaz) - Friday, 21 February 2020, 13:25 GMT
Feel free to post a patch here or send me via an email (while we still use svn it's not as easy to send PRs otherwise...), I'll have a look :)

I can already tell you, I don't think it's good idea to add this emoji font as a final fallback to generic families by default, because I would not want to ship a config that has potential to make weird behavior even for a small subset of users. First and foremost goal is to ship the font itself, so if you specify "font-family: JoyPixels" you will get emojis. The rest of the things really depend on what you want, some want the fallback, others might not. Maybe someone will have two emoji fonts installed, and they would want the generic font families to fallback on another emoji font, I wouldn't want for example to add a conflict with noto-color-emoji package.

The only thing that I can see as safe to have by default is those EmojiOne aliases, but on the other hand they rebranded almost a year ago, so I don't think it makes sense to ship a default config purely with legacy names.


Comment by Setpill (setpill) - Friday, 21 February 2020, 18:07 GMT
Understood. I wish number glyphs hadn't been added to the emoji standard - that's utterly stupid. If that hadn't been the case, it would have been possible to do some sane slicing of fonts into emoji and non-emoji components, shipping them seperately. But unfortunately, expecting sanity from a standards committee is not realistic. Feel free to close this.
Comment by Maxim Baz (maximbaz) - Friday, 21 February 2020, 18:57 GMT
I wish so too! By the way, it's rare, but CSS at least has _some_ solution, it is possible to specify which Unicode codepoints should be rendered by a given font, so for example you can set JoyPixels as the very first font in font-family but tell it to never render numbers (I did this for Wire app, but I don't know of any other website that does so - if you are interested to see my implementation, here's a link: https://github.com/wireapp/wire-webapp/blob/4435338510512351d2970745fa831571331cc401/src/style/common/variables.less#L28-L47 )

I wish something like this was possible in fontconfig, then we would be able to just exclude the numbers even if they are present in the font itself. Last time I checked (a few years ago) I wasn't able to find a way to do this, but if you are interested you could give it another try, maybe you will be lucky to find a solution? :)
Comment by Setpill (setpill) - Friday, 21 February 2020, 21:25 GMT Comment by Maxim Baz (maximbaz) - Friday, 21 February 2020, 21:39 GMT
Interesting! Doesn't seem to work for me though, does it do anything for you (or do you see a problem in my snippet)? If I set "font-family: JoyPixels", I still get colorful numbers...

<match target="scan">
<test name="family" compare="eq">
<string>JoyPixels</string>
</test>
<edit name="charset" mode="assign">
<minus>
<name>charset</name>
<charset>
<range>
<int>0x0030</int>
<int>0x0039</int>
</range>
</charset>
</minus>
</edit>
</match>
Comment by Setpill (setpill) - Sunday, 23 February 2020, 23:55 GMT
This is intriguing... According to `fc-list "JoyPixels":charset=0x0030` that DOES seems to work just fine (after making sure to `sudo fc-cache -sf`). However, chromium still shows the outlook page with emoji number glyphs...
Comment by Setpill (setpill) - Sunday, 23 February 2020, 23:59 GMT
But it DOES fix things in firefox 🤯
Comment by Maxim Baz (maximbaz) - Tuesday, 25 February 2020, 16:48 GMT
I spent some more time on this, removing number glyphs and prepending to the generic font families _generally_ works, but has some weird consequences: chromium ignores this (as you mentioned), kitty suddenly won't launch at all, seafile (or all QT apps?) also show numbers rendered by JoyPixels, etc.

However I think this removal trick can still be very useful. On my personal laptop, the black-and-white emojis (like this one -> 😄 ) come from DejaVu Sans font, so if I keep JoyPixels appended and not prepended, but simply remove emoji glyphs from DejaVu Sans font, then rendering in Chromium and Firefox works as expected for me.

<match target="scan">
<test name="family" compare="eq">
<string>DejaVu Sans</string>
</test>
<edit name="charset" mode="assign">
<minus>
<name>charset</name>
<charset>
<range>
<int>0x1f600</int>
<int>0x1f640</int>
</range>
</charset>
</minus>
</edit>
</match>
Comment by Maxim Baz (maximbaz) - Tuesday, 25 February 2020, 18:43 GMT
I'm closing the ticket with the following changes:

- install notice is updated to be more clear that 75-joypixels.conf is just an example
- removed Symbola rejection from 75-joypixels.conf (symbola is not in official repos, plus instead of rejecting it one can simply remove the font)
- add example on how to remove emoji glyphs from DejaVu fonts

Thank you so much for your help along the way! If needed, feel free to ping me or drop me an email.

Loading...