FS#61288 - [elasticsearch] seems to be broken. Some files seem to have incompatible permissions and locations.

Attached to Project: Community Packages
Opened by Aran Mohyeddin (rnmhdn) - Sunday, 06 January 2019, 06:24 GMT
Last edited by Massimiliano Torromeo (mtorromeo) - Monday, 07 January 2019, 07:24 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Massimiliano Torromeo (mtorromeo)
Architecture All
Severity High
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:
The first time I installed it it worked using systemctl then I changed the yml config file under /etc and it went nuts. I reinstalled it a few times but I can't get it to work again.

Additional info:
* package version(s)
$ pacman -Qi elasticsearch
Name : elasticsearch
Version : 6.5.4-1
Description : Distributed RESTful search engine built on top of Lucene
Architecture : any
URL : https://www.elastic.co/products/elasticsearch
Licenses : Apache
Groups : None
Provides : None
Depends On : java-runtime-headless systemd
Optional Deps : None
Required By : None
Optional For : None
Conflicts With : None
Replaces : None
Installed Size : 36.34 MiB
Packager : Massimiliano Torromeo <massimiliano.torromeo@gmail.com>
Build Date : Fri 21 Dec 2018 07:22:23 PM +0330
Install Date : Sun 06 Jan 2019 08:44:11 AM +0330
Install Reason : Explicitly installed
Install Script : No
Validated By : Signature
* config and/or log files etc.


Steps to reproduce:
When I do `$ sudo pacman -S elasticsearch` I get:
> warning: directory permissions differ on /etc/elasticsearch/
> filesystem: 750 package: 755

Also:
$ elasticsearch-keystore
/usr/bin/elasticsearch-keystore: line 3: /usr/bin/elasticsearch-cli: No such file or directory
$ elasticsearch-plugin
/usr/bin/elasticsearch-plugin: line 3: /usr/bin/elasticsearch-cli: No such file or directory
$ elasticsearch-sql-cli
zsh: command not found: elasticsearch-sql-cli
I think this is a weird path.
$ /usr/share/elasticsearch/bin/elasticsearch
Exception in thread "main" java.nio.file.AccessDeniedException: /etc/elasticsearch/jvm.options
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:215)
at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
at java.base/java.nio.file.Files.newByteChannel(Files.java:421)
at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
at java.base/java.nio.file.Files.newInputStream(Files.java:155)
at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:60)

$ sudo /usr/share/elasticsearch/bin/elasticsearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2019-01-06T09:02:27,823][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [unknown] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:140) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.5.4.jar:6.5.4]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.5.4.jar:6.5.4]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86) ~[elasticsearch-6.5.4.jar:6.5.4]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:103) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-6.5.4.jar:6.5.4]
I assume I shouldn't run it as root so I change permissions.

$ sudo chmod 755 /etc/elasticsearch

$ /usr/share/elasticsearch/bin/elasticsearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Could not rename log file 'logs/gc.log' to 'logs/gc.log.00' (Permission denied).
[0.000s][error][logging] Error opening log file 'logs/gc.log': Permission denied
[0.000s][error][logging] Initialization of output 'file=logs/gc.log' using options 'filecount=32,filesize=64m' failed.
Invalid -Xlog option '-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m', see error log for details.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

$ sudo chmod -R 777 /var/log/elasticsearch

$ /usr/share/elasticsearch/bin/elasticsearch
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/elasticsearch/scripts
Likely root cause: java.nio.file.AccessDeniedException: /etc/elasticsearch/scripts
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:428)
at java.base/java.nio.file.Files.newDirectoryStream(Files.java:471)
at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:300)
at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:373)
at java.base/java.nio.file.Files.walkFileTree(Files.java:2760)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:218)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:127)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:302)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86)
Refer to the log for complete error details.

I give up going down the rabbit hole. and uninstall it and try to use it with systemctl.
$ sudo pacman -Rn elasticsearch
$ sudo rm -rf /var/log/elasticsearch
$ sudo rm -rf /var/lib/elasticsearch
$ sudo rm -rf /etc/elasticsearch
$ sudo pacman -S elasticsearch
I removed all the files listed in pacman -Qkk elasticsearch and uninstall and install again and also do systemctl daemon-reload. and I when I run systemctl start elasticsearch.service I get:
Job for elasticsearch.service failed because a timeout was exceeded.
See "systemctl status elasticsearch.service" and "journalctl -xe" for details.

and journalctl -xe gives:
Jan 06 09:45:54 Christopher elasticsearch[9920]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Jan 06 09:45:54 Christopher systemd[1]: elasticsearch.service: Can't open PID file /run/elasticsearch/elasticsearch.pid (yet?) after start: No such file or directory
Jan 06 09:45:55 Christopher elasticsearch[9920]: Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: org.elasticsearch.cli.UserException: unable to create temporary ke>
Jan 06 09:45:55 Christopher elasticsearch[9920]: Likely root cause: java.nio.file.AccessDeniedException: /etc/elasticsearch/elasticsearch.keystore.tmp
Jan 06 09:45:55 Christopher elasticsearch[9920]: Refer to the log for complete error details.
This task depends upon

Closed by  Massimiliano Torromeo (mtorromeo)
Monday, 07 January 2019, 07:24 GMT
Reason for closing:  Fixed
Additional comments about closing:  elasticsearch-6.5.4-2
Comment by Jeb Rosen (jebrosen) - Sunday, 06 January 2019, 06:36 GMT
The pacman -Qkk errors are false positives: the permissions in the package mismatch with the ones set via tmpfiles.d.

The elasticsearch-* files in /usr/bin do look broken: /usr/bin/elasticsearch-keystore and -plugin try to run /usr/bin/elasticsearch-cli which is not installed. /usr/bin/elasticsearch-sql-cli is a broken symlink to another file that's also not installed.

I cannot, however, reproduce the failure to start. elasticsearch starts fine on my machine and is accessible via curl.
Comment by Aran Mohyeddin (rnmhdn) - Sunday, 06 January 2019, 06:37 GMT
I can't find out how I should edit the task but a friend did a fresh install and then did start with systemctl and then curl localhost:9200 and got a response so the package is not completely broken. but I can't get that response eventho the first time I installed It I could get that response before editing the yml config file under /etc. but I can't get it to give me that response anymore.
Comment by Aran Mohyeddin (rnmhdn) - Sunday, 06 January 2019, 07:00 GMT
I restarted my laptop and now I could start it with systemctl. is there any way to decrease the severity of this bug report?
Comment by Eli Schwartz (eschwartz) - Sunday, 06 January 2019, 08:05 GMT
  • Field changed: Summary (Elasticsearch seems to be broken. Some files seem to have incompatible permissions and locations. → [elasticsearch] seems to be broken. Some files seem to have incompatible permissions and locations.)
  • Field changed: Status (Unconfirmed → Assigned)
  • Field changed: Architecture (x86_64 → All)
  • Task assigned to Massimiliano Torromeo (mtorromeo)
the elasticsearch-{keystore,plugin} commands hardcode "`dirname "$0"`" as the directory path for elasticsearch-cli, which is not symlinked as well.

And elasticsearch-sql-cli simply no longer exists, so it is a broken link.

Might be a better idea to just loop through the provided scripts and symlink them all. Though upstream should really use readlink in the scripts.
Comment by Massimiliano Torromeo (mtorromeo) - Sunday, 06 January 2019, 09:37 GMT
I'll fix the scripts as soon as I can, sorry about that.

One note though: elasticsearch is supposed to be run using the "elasticsearch" user and ideally just from systemd. Do not "sudo elasticsearch" or randomly chmod directories because, as you've seen, it gets messy.

Loading...