FS#50283 - [elasticsearch] elasticsearch.pid not readable

Attached to Project: Community Packages
Opened by teraflops (teraflops) - Saturday, 06 August 2016, 16:16 GMT
Last edited by Massimiliano Torromeo (mtorromeo) - Sunday, 07 August 2016, 09:53 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Massimiliano Torromeo (mtorromeo)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:

elasticsearch.pid not readable (yet?)

Additional info:

package version: 2.3.5-1

error:

elasticsearch.service: PID file /run/elasticsearch/elasticsearch.pid not readable (yet?) after start: No such file or directory

Steps to reproduce:

start the service

workaround:

adding PID_DIR="/var/run" and restarting the services fixes it.

source: https://github.com/elastic/elasticsearch/issues/11594

This task depends upon

Closed by  Massimiliano Torromeo (mtorromeo)
Sunday, 07 August 2016, 09:53 GMT
Reason for closing:  Not a bug
Comment by Doug Newgard (Scimmia) - Sunday, 07 August 2016, 04:33 GMT
Yes, I brought this up in  FS#50049 , but the maintainer seems to be content with leaving this package as a mess.
Comment by Massimiliano Torromeo (mtorromeo) - Sunday, 07 August 2016, 09:21 GMT
That is really helpful Doug, THANK YOU!

So in the previous bug you brought up adding PID_DIR? Did you even try installing the package?
I actually made some changes to track some directories that were never meant to be created outside the package.
The tmpfile has a different purpose as explained already.

More importantly as explained in the previous bug report, the message "elasticsearch.pid not readable (yet?)" is not indicative of a failure.
The service file should (and actually does for me) create /run/elasticsearch by respecting the RuntimeDirectory=elasticsearch directive.
The PIDFile=/run/elasticsearch/elasticsearch.pid directive instructs it to expect that file to be created by elasticsearch and elasticsearch should do that since it is run with the command line argument "-p /run/elasticsearch/elasticsearch.pid".

Systemd looks for the file before elasticsearch has the time to create it as the warning tells you (notice the "yet?") but it is eventually created after this time and everything should be working.
The problem is that elasticsearch forks BEFORE creating the pid file while ideally it would be the other way around but that is for upstream to fix.
I think systemd will wait for TimeoutStartSec (default 90s) for the pid file to show up.

If elasticsearch is not starting correctly then the message "PID file /run/elasticsearch/elasticsearch.pid not readable (yet?) after start" is NOT the culprit and I need more informations.
The previous reporter failed to follow up with that and I closed the issue.

Here's a quick test on my system with the default elasticsearch configuration where you can notice that I also have the message in the service status but everything is running correctly nonetheless and how the pidfile is actually there while the directory did not exist before:

$> ls -ld /run/elasticsearch
ls: cannot access '/run/elasticsearch': No such file or directory

$> systemctl start elasticsearch
$> ls -l /run/elasticsearch
total 4
-rw-r--r-- 1 elasticsearch elasticsearch 4 Aug 7 11:07 elasticsearch.pid

$> systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
Active: active (running) since dom 2016-08-07 11:07:35 CEST; 1min 4s ago
Docs: http://www.elastic.co
Process: 9529 ExecStart=/usr/bin/elasticsearch -d -p /run/elasticsearch/elasticsearch.pid -Des.default.path.home=${ES_HOME} -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR} -Des.default.path.conf=${CONF_DIR} (code=exited, status=0/SUCCESS)
Main PID: 9538 (java)
Tasks: 84 (limit: 4915)
CGroup: /system.slice/elasticsearch.service
└─9538 /usr/lib/jvm/default-runtime/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-2.3.5.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d -p /run/elasticsearch/elasticsearch.pid -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.conf=/etc/elasticsearch

ago 07 11:07:34 diavel systemd[1]: Starting Elasticsearch...
ago 07 11:07:34 diavel systemd[1]: elasticsearch.service: PID file /run/elasticsearch/elasticsearch.pid not readable (yet?) after start: No such file or directory
ago 07 11:07:35 diavel systemd[1]: Started Elasticsearch.
Comment by teraflops (teraflops) - Sunday, 07 August 2016, 09:46 GMT
well I didn't notice I had to search for the closed issues too. And since the status message is "at least" suspicious I filled an issue.
My elastic searches are fine btw.

peace bros
Comment by Massimiliano Torromeo (mtorromeo) - Sunday, 07 August 2016, 09:52 GMT
Thanks for letting us know.
Cheers!

Loading...