FS#73032 - [firefox] firefox from flathub outperforms firefox from archlinux on speedometer tests

Attached to Project: Arch Linux
Opened by Alexandre Bique (babali) - Wednesday, 15 December 2021, 14:20 GMT
Last edited by Jan Alexander Steffens (heftig) - Tuesday, 15 March 2022, 11:14 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Jan Alexander Steffens (heftig)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 2
Private No

Details

I've ran the speedometer benchmark on firefox 95.0 from both archlinux and flatpak.

https://browserbench.org/Speedometer2.0/

The result is:
- 120 ~3.4 on archlinux
- 134 ~1.7 on flatpak

Does it reveal a problem in the way archlinux is compiling firefox?

Regards,
Alex
This task depends upon

Closed by  Jan Alexander Steffens (heftig)
Tuesday, 15 March 2022, 11:14 GMT
Reason for closing:  Works for me
Comment by Grzegorz (itsnotme) - Wednesday, 15 December 2021, 23:45 GMT
It's probably because Firefox binaries released by Mozilla are built with profile-guided optimization, but many distros build it without PGO.
Comment by Alexandre Bique (babali) - Thursday, 16 December 2021, 07:49 GMT
Then should arch ship the binary version of firefox? Or should arch enable PGO as well?
What's the point of rebuilding firefox is the result is inferior?
Comment by Jonathon (jonathon) - Saturday, 18 December 2021, 01:51 GMT
Arch ships with PGO, https://github.com/archlinux/svntogit-packages/blob/eb0b75befc0e9da68a85a94ec47d1fc47850c60b/trunk/PKGBUILD#L125
Comment by mattia (nTia89) - Saturday, 12 March 2022, 14:01 GMT
PGO enabled. Can we close the issue now?
Comment by Alexandre Bique (babali) - Monday, 14 March 2022, 07:13 GMT
arch: 125 +/- 2.2
flatpak: 133 +/- 3.3

Not there yet, you have more work I suppose.
Comment by mattia (nTia89) - Monday, 14 March 2022, 16:39 GMT
It's very hard to investigate in it.
There is a plethora of variables at stake: compiler options, compilation options, runtime options, different libraries, ...

And we don't know how these variables are handled from the flatpak side.

Just for fun, have you tried running the benchmark with a compiled-by-you version of Firefox?
Comment by Alexandre Bique (babali) - Monday, 14 March 2022, 16:56 GMT
Flathub is using the binary distribution of firefox, they don't build it themself.
Comment by mattia (nTia89) - Monday, 14 March 2022, 17:14 GMT
It's the same... If we do not know how those variables are handled by Mozilla, it's basically impossible for us to get to the root of the problem.

And, since we are talking about an application which bundles in it libraries, this could NOT be an issue of Firefox itself BUT an issue of a dependency of it.
Comment by Alexandre Bique (babali) - Monday, 14 March 2022, 17:34 GMT
If I knew where the problem is I'd tell you.
If Firefox is slower because of a lib, then maybe many applications are affected which would raise the importance of this issue?
One other option would be to not rebuild firefox and use their binary distribution.
Comment by mattia (nTia89) - Monday, 14 March 2022, 18:33 GMT
Ok, we are going OT.

I see the issue (numbers don't lie) and I am in primis concerned about it, but at the same time I can't imagine how to address it.

There are many things to say but I do not think this is the right place.

For this reason I leave the discussion.
Comment by Jan Alexander Steffens (heftig) - Monday, 14 March 2022, 20:11 GMT
Besides using a different build of Clang, our *FLAGS contain a considerable amount of hardening that Mozilla's build does not have. I'd tentatively blame that.
Comment by Alexandre Bique (babali) - Tuesday, 15 March 2022, 08:30 GMT
Why wouldn't firefox build with hardening flags? Just for performance?
Do those flags make firefox more secure?

6% of performance loss for hardening flags seems quite high though.

I think it is especially important to raise the security for a web browser. But what about sandboxing? I actually feel safer with firefox being ran within a sandbox like in flatpak.
Comment by Alexandre Bique (babali) - Tuesday, 15 March 2022, 08:32 GMT
I also think that they know what they are doing at Mozilla when they build Firefox.
Comment by Jan Alexander Steffens (heftig) - Tuesday, 15 March 2022, 09:29 GMT
Here's a build without our hardening:

https://pkgbuild.com/~heftig/firefox-98.0.1-1.1-x86_64.pkg.tar.zst
Comment by Alexandre Bique (babali) - Tuesday, 15 March 2022, 10:19 GMT
I've tested your build. I got variable results, it went down to 120, then I ran it again in private browsing and it went up to 137.

I believe it is maybe related to ublock or the the amount of cookies / history / ...

Sorry seems like a user error!

Loading...