FS#80129 - [gitlab] references to non-existent sidekiqctl

Attached to Project: Arch Linux
Opened by Oliver Mangold (omangold) - Wednesday, 01 November 2023, 10:57 GMT
Last edited by Caleb Maclennan (alerque) - Tuesday, 07 November 2023, 06:55 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Anatol Pomozov (anatolik)
Caleb Maclennan (alerque)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

Description:

After updating Gitlab from 16.1.4-1 and doing the usual steps of running db:migrate and restarting the stack, sidekiq refuses to start. After some initialization time it stops with this in the log:

--- snip ---
Nov 01 11:51:13 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:13.478Z","message":"Cron Jobs - added job with name: service_desk_custom_email_verification_cleanup"}
Nov 01 11:51:13 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:13.481Z","message":"Cron Jobs - added job with name: ensure_merge_requests_prepared_worker"}
Nov 01 11:51:13 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:13.483Z","message":"Cron Jobs - added job with name: deactivated_pages_deployments_delete_cron_worker"}
Nov 01 11:51:39 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:39.052Z","message":"Booted Rails 7.0.8 application in production environment"}
Nov 01 11:51:39 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:39.052Z","message":"Running in ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]"}
Nov 01 11:51:39 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:39.052Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}
Nov 01 11:51:39 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:39.052Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}
Nov 01 11:51:47 gogs systemd[1]: Stopping GitLab Sidekiq Worker...
Nov 01 11:51:47 gogs gitlab-sidekiq[3231185]: bundler: command not found: sidekiqctl
Nov 01 11:51:47 gogs gitlab-sidekiq[3231185]: Install missing gem executables with `bundle install`
Nov 01 11:51:47 gogs systemd[1]: gitlab-sidekiq.service: Control process exited, code=exited, status=127/n/a
Nov 01 11:51:47 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:47.813Z","message":"Shutting down"}
Nov 01 11:51:47 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:47.815Z","message":"Scheduler exiting..."}
Nov 01 11:51:47 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:47.816Z","message":"Terminating quiet threads"}
Nov 01 11:51:47 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:47.816Z","message":"Scheduler exiting..."}
Nov 01 11:51:48 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:48.317Z","message":"Pausing to allow jobs to finish..."}
Nov 01 11:51:49 gogs gitlab-sidekiq[3231095]: {"severity":"INFO","time":"2023-11-01T10:51:49.320Z","message":"Bye!"}
Nov 01 11:51:49 gogs systemd[1]: gitlab-sidekiq.service: Failed with result 'exit-code'.
Nov 01 11:51:49 gogs systemd[1]: Stopped GitLab Sidekiq Worker.
Nov 01 11:51:49 gogs systemd[1]: gitlab-sidekiq.service: Consumed 51.784s CPU time.
---

Additional info:
* version: gitlab-16.5.0-2
This task depends upon

Closed by  Caleb Maclennan (alerque)
Tuesday, 07 November 2023, 06:55 GMT
Reason for closing:  Fixed
Comment by loqs (loqs) - Wednesday, 01 November 2023, 12:43 GMT Comment by Oliver Mangold (omangold) - Wednesday, 01 November 2023, 13:05 GMT
Found the problem myself.

Sidekiq was restarted by failing puma. Puma was failing due to missing directories /var/lib/gitlab/log and /var/lib/gitlab/tmp/pids. Creating these directories fixed it.

Missing sidekiqctl seems to be an independent issue.
Comment by Toolybird (Toolybird) - Wednesday, 01 November 2023, 20:08 GMT
Related  FS#80077 
Comment by Caleb Maclennan (alerque) - Monday, 06 November 2023, 12:13 GMT
@omangold can you try pkgrel=3 from the extra testing repository and confirm that the changes there fix it for you?
Comment by Caleb Maclennan (alerque) - Tuesday, 07 November 2023, 06:55 GMT
Going to move to [extra] here shortly having heard no feedback. If there are issues after release feel free to open another bug and link back to this one.

Loading...