Arch Linux

@Taijian: Thanks for the very verbose ticket.

> The wrapper script, by virtue of it's placement in /usr/bin takes precedent over the default occ provided by nextcloud, potentially breaking existing uses of the script.

/usr/share/webapps/nextcloud is not in PATH. If it is, that's a user modification.

> changing directory ownerships to a new uid on an existing and running system is pretty much guaranteed to break stuff

So is giving read-write ownership to the http user in /usr/share/webapps/nextcloud/ (apart from that being dangerous for other reasons).
We have packaging guidelines for web applications. Nextcloud now meets them.

> and no, changing the user of the web server is not really an option for people running more than one service off of one web server

Unfortunately, we never claimed, that this is a supported use-case.
However, if you would like to do something like that, I suggest introducing further user-separation via sysusers.d and tmpfiles.d and potentially adding more symlinks (in case users are also supposed to be able to e.g. install apps to their respective own locations).
The only interesting change is probably the php-fpm configuration, if that's in use.

> revert the unnecessary changes to this package,

These changes are necessary for separations of concern on a target system. Otherwise the http user has read/write access to all files of all web applications.

Closing: I understand your frustration and personally I would have liked to have done this **way more** orchestrated than it was done. I am currently on the task of revising the wiki article to point out the various configuration options (and can definitely imagine myself doing something nicer this weekend).

@David: Reading up on the packaging guidelines I think I now understand *why* you felt it necessary to introduce the changes you made wrt uid and dir structure.

That said, I would certainly hope that there is also an implicit understanding in the TU bylaws about trying to minimize breakeage of user systems - I mean, there is a testing repo and a process for pushing out news items for a reason. I get now that (and why) you will not roll back the changes you made. That is OK, your reasoning is sound and the resulting package is probably 'better' now, than it was before.

However, could you please try to get a warning / announcement out about the breaking changes in this package before even more people step into that trap? Because people will, and it will ruin their days. And you can still try to minimize the suffering by getting an organized warning out to them. Also, when introducing breaking changes, it would be really nice to point people to a concise guide towards converting existing systems from old to new.

Just to add my two cents, I've also spent half my day now fixing nextcloud; between internal server errors with no clear error message anywhere I could find; "Cannot write into "config" directory" errors; figuring out that I probably need another php-fpm worker (or whatever they're called) now with the 'nextcloud' user instead of the 'http' user and maybe figuring out how to configure apache to use this php-fpm instance for only nextcloud and the default one for the rest, etc.
For sure 90% isn't caused directly by the Arch packaging but it's still a huge change in the setup where surely a lot of people will spend a lot of time on.
But once I've done that please don't change it back :D

I just change my /etc/pacman.d/hooks/nextcloud.hook:

Exec = /bin/chown -R http /usr/share/webapps/nextcloud/config /etc/webapps/nextcloud/config.php ; /usr/bin/runuser -u http -- /usr/bin/php7 /usr/share/webapps/nextcloud/occ upgrade

and switch apache to php7. Did not catch any other issues yet but I am on 21.0.0-2 now.

I've also spent 4 hours trying to fix nextcloud, and I still haven't found a fix yet.

> > and no, changing the user of the web server is not really an option for people running more than one service off of one web server
>
> Unfortunately, we never claimed, that this is a supported use-case.
> However, if you would like to do something like that, I suggest introducing further user-separation via sysusers.d and tmpfiles.d and potentially adding more symlinks (in case users are also supposed to be able to e.g. install apps to their respective own locations).

I am also running more than one service off a single web server and I suspect this is a common use-case. I haven't figured out how to properly configure user-separation between different webapps that are run off an apache webserver (running as a single user 'http'). If anyone knows how to do this, some pointers would be appreciated.

I also experienced troubles with my upgrade today - there's a discussion thread on the forums about it https://bbs.archlinux.org/viewtopic.php?id=263820.
@z3ntu A workaround is to simply add 'config_is_read_only' => true to config.php, which doesn't seem to stop Nextcloud from editing config.php itself.

@dvzrv
Note that the following upgrade message was shown when I upgraded to 21.0.0-2:

WARNING: Major changes introduced to package!
The application is now run as its own user - nextcloud.
Default directories for runtime, state and log files are provided.
The configuration file is now in /etc/webapps/nextcloud/config.php.
A convenience wrapper around occ is provided in /usr/bin/occ.

The upgrade message nor the wiki article at the time of the upgrade indicated:
- that legacy PHP packages are in use, and so PHP packages needed re-installation and reconfiguring
- how to run the webapp with php-fpm using the new "nextcloud" user by adding additional pools

Additionally my instance's config.php was wiped out leaving only the instance ID, requiring setup to be run again.
The newly installed config.php was a copy of config.sample.php which caused errors as it explicitly states it is not meant to be used as the basis for your own config.php.
I imagine this was caused when Nextcloud tried running while the installation was borked, as pacman should've backed it up with a .pacsave file.
Did you test an in-place upgrade and see similar behavior?

I understand and appreciate that you're improving the state of the package with these changes, but _please_ update the wiki beforehand and direct people to it in the upgrade message.

> @z3ntu A workaround is to simply add 'config_is_read_only' => true to config.php, which doesn't seem to stop Nextcloud from editing config.php itself.
I've managed to fix it up completely now; that was just the error I had at the time of writing this.

> The newly installed config.php was a copy of config.sample.php which caused errors as it explicitly states it is not meant to be used as the basis for your own config.php.
I actually had that as well, but fortunately I had the config checked into git so I could get it back. But yes the config being basically wiped is definitely a huge problem for most users.

> The newly installed config.php was a copy of config.sample.php which caused errors as it explicitly states it is not meant to be used as the basis for your own config.php.
> I imagine this was caused when Nextcloud tried running while the installation was borked, as pacman should've backed it up with a .pacsave file.

I can confirm this because I ran into the same issue. I still had a config at /etc/webapps/nextcloud/config/config.php so I overwrote the sample config (at the new config location /etc/webapps/nextcloud/config.php) with it, which fixed these errors.

I have just pushed 21.0.0-5 to [community-testing] which does the following:

* Install the config directory to /etc/webapps/nextcloud again and symlink it into /usr/share/webapps/. This change is required for nextcloud to not error out on not being able to write into it.
* The default config only sets the default paths.

Please do test this!

This has the upside, that with a simple php-fpm config (see attachment) the application can now be used out-of-the-box.

I will now try to update remaining info in the wiki article.

@williamvds:

> Additionally my instance's config.php was wiped out leaving only the instance ID, requiring setup to be run again.
> The newly installed config.php was a copy of config.sample.php which caused errors as it explicitly states it is not meant to be used as the basis for your own config.php.
> I imagine this was caused when Nextcloud tried running while the installation was borked, as pacman should've backed it up with a .pacsave file.

I'm sorry to hear that and I have not had this experience locally. My setup might also be different though.

> Did you test an in-place upgrade and see similar behavior?
>
> I understand and appreciate that you're improving the state of the package with these changes, but _please_ update the wiki beforehand and direct people to it in the upgrade message.

My intention was to first move it to [community-testing], do testing (given the nature of the changes), update the wiki and then move to [community].
In fact, my first changes where only committed to trunk for sergej to have a look.

I am not very happy about the current situation and have spent half the day trying to fix the migration path from 20.0.7 to 21.0.0 and update the wiki (and I am still not done).

Please note, that some of the problems users are now experiencing are due to the fact, that a) the package previously generously changed file ownership and b) due to many inconsistencies *a lot* of hacks have been added to the wiki (which people applied to their setups) to make up for that fact.
I see, that e.g. the directories in an upgrade situation (e.g. 20.0.x -> 21.0.0) are still owned by the http group, but this is for another day...

So, do I understand it right, it's not possible anymore to have nextcloud installed next to other web-services without doing magic things like that: "introducing further user-separation via sysusers.d and tmpfiles.d and potentially adding more symlinks"

and where is the new config?
/etc/webapps/nextcloud
/etc/webapps/nextcloud/config
/usr/share/webapps/

My history (nextcloud with apache):
Since January I ignored package updates for php*, tt-rss, imagemagick because nexcloud wasn't able to use PHP8
Then yesterday with new nextcloud version 21.0.0-4 i gave it a try, did pacman everything and rebooted (maybe early reboot was a mistake)
Result:
- httpd was broken, but easy to fix in /etc/httpd/conf/httpd.conf by replacing php7 with php
- tt-rss was broken but fixable with hints in https://bugs.archlinux.org/task/69462?string=tt-rss and installing/enabling php-7fpm
- but nexcloud didn't run "internal server error":
- After copy of /etc/webapps/nextcloud/config/config.php to /etc/webapps/nextcloud/config.php, I tried to do an update with occ, but "sudo -u http php occ maintenance:mode --on" already failed.

Because of this and this warning: "The application is now run as its own user - nextcloud." I reverted everything back.

So, can I use nextcloud next to tt-rss, wbce, phpadmin, php-forums?

I also had issues after the upgrade, I had already an extra php-fpm pool for Nextcloud, but also running syncthing and nextcloud-inotifyscan as http. The latter runs an occ scan when syncthing changes a file. I changed their users to nextcloud and the most stuff runs now again, but Nextcloud misses "some" files:
https://p.rrbone.net/?8c587c511b370dd4#48JY17uJoaYP21wQ3eXQ7D1P6v6sdycXsH2hFvDdDzwR

@midixinga:
It's possible to run both, I'm running php7-fpm for FreshRSS and wallabag in parallel with php-fpm for Nextcloud.

@bjo : thx für your answer.
Actually I don't use php-fpm, I only installed it (an uninstalled it after going back) to get tt-rss running.

One simple more question: Would it help to do a fresh new install of nextcloud?

@midixinga:
So you used the mod_php before? I have no idea if it's possible to enable both Apache-modules (php and php7) at the same time, but if it's possible I think you could set the module per VirtualHost.
I don't think a fresh install would help, it does not solve the problem that Nextcloud wants PHP 8 and some other webapps still rely on PHP 7.

@midixinga: It is possible to e.g. use php-fpm for nextcloud (see my attachment in https://bugs.archlinux.org/task/69718#comment196955).
Also uwsgi should work (but you will have to deal with it only supporting php7 at this point in time, maybe ever).

@bjo: The issue with the missing files should be fixed with 21.0.0-6 (currently still in [community-testing] - see https://bugs.archlinux.org/task/69721).

I have now added a troubleshooting section to the nextcloud article [1], which explains required steps for a working setup.

As people have reported 21.0.0-6 as working [2][3], I will now move that version from [community-testing] to [community] to have a more streamlined experience for anyone who wants to upgrade now.

Sorry for the introduced inconvenience.

[1] https://wiki.archlinux.org/index.php/Nextcloud#Issues_with_permissions_and_setup_after_upgrade_to_%3E=_21.0.0
[2] https://bugs.archlinux.org/task/69721#comment196969
[3] https://bugs.archlinux.org/task/69725#comment196968

Hello David,

thanks

ok I will give it a try from PHP7 and Nextcloud 20.0.7 after Backup of my SD-Card

In the Wiki:
"neither the data directory nor the writable apps directory is located below /usr/share/webapps/nextcloud/"
Links to "data directory" and "writable apps directory" don't work

php-fmp (or uwsgi) is now manadatory? I don't use it for now

@midixinga Thanks for the heads up! I fixed the links (they refer to a subsection in the Configuration section).

Using php-fpm with a server pool as described by @bjo [1] might be the easiest solution for you. Tbh, I have never used apache for hosting php directly (mainly nginx + application servers).
There is also an example for a php-fpm pool config attached in one of my earlier comments [2]. Please take that with a grain of salt, as it was only for testing. The wiki could still use a proper setup scenario.

Using uwsgi at this point unfortunately requires doing everything with the php7* equivalent packages, required for nextcloud (e.g. php7, php7-acpu, php7-intl, etc.).

[1] https://bugs.archlinux.org/task/69718#comment196967
[2] https://bugs.archlinux.org/task/69718#comment196955