FS#68726 - [ceph] MGR: Module 'diskprediction_local' takes forever to load

Attached to Project: Community Packages
Opened by likeit (likeit) - Monday, 23 November 2020, 20:12 GMT
Last edited by Thore Bödecker (foxxx0) - Tuesday, 06 April 2021, 15:04 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Thore Bödecker (foxxx0)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

The 'diskprediction_local' takes forever to load.

-> https://tracker.ceph.com/issues/45147
This task depends upon

Closed by  Thore Bödecker (foxxx0)
Tuesday, 06 April 2021, 15:04 GMT
Reason for closing:  Fixed
Additional comments about closing:  upstream workaround has been backported and included into the 15.2.10-1 packages
Comment by Thore Bödecker (foxxx0) - Thursday, 17 December 2020, 15:37 GMT
There are updated ceph 15.2.8-1 packages in [community-testing] now, which should fix this issue.
Please test them and report back.
Comment by likeit (likeit) - Thursday, 17 December 2020, 19:40 GMT
The issue still exist:

2020-12-17T20:26:59.134+0100 7f5733951040 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process ceph-mgr, pid 663
2020-12-17T20:26:59.134+0100 7f5733951040 0 pidfile_write: ignore empty --pid-file
2020-12-17T20:26:59.204+0100 7f5733951040 1 mgr[py] Loading python module 'alerts'
2020-12-17T20:26:59.354+0100 7f5733951040 1 mgr[py] Loading python module 'balancer'
2020-12-17T20:26:59.464+0100 7f5733951040 1 mgr[py] Loading python module 'cephadm'
2020-12-17T20:26:59.854+0100 7f5733951040 1 mgr[py] Loading python module 'crash'
2020-12-17T20:26:59.974+0100 7f5733951040 1 mgr[py] Loading python module 'dashboard'
2020-12-17T20:27:01.184+0100 7f5733951040 1 mgr[py] Loading python module 'devicehealth'
2020-12-17T20:27:01.264+0100 7f5733951040 1 mgr[py] Loading python module 'diskprediction_cloud'
2020-12-17T20:27:01.394+0100 7f5733951040 1 mgr[py] Loading python module 'diskprediction_local' <-- takes forever to load...

So far I haven't had the time to take a closer look at the problem. Since I don't need this module for my cluster, I simply removed it.

rm -r /usr/share/ceph/mgr/diskprediction_local
systemctl restart ceph-mgr.target
Comment by likeit (likeit) - Thursday, 17 December 2020, 20:09 GMT
I was now able to narrow down the problem further:

file: /usr/share/ceph/mgr/diskprediction_local/module.py
# Importing scipy early appears to avoid a future deadlock when
# we try to do
#
# from .predictor import get_diskfailurepredictor_path
#
# in a command thread. See https://tracker.ceph.com/issues/42764
import scipy <-- This import is the root cause of the problem.

Loading...