FS#37674 - [virtuoso] kdepim-kmail freeze and high cpu usage

Attached to Project: Arch Linux
Opened by Nolwenn (Nonolapero) - Thursday, 07 November 2013, 11:05 GMT
Last edited by Andrea Scarpino (BaSh) - Saturday, 23 November 2013, 22:02 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Andrea Scarpino (BaSh)
Sven-Hendrik Haase (Svenstaro)
Architecture x86_64
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description: After kde 4.11.3 update, kontact or kmail have a high cpu usage. I check htop, it's mysql and akonadiserver who eat my cpu. Akonadi correctly check new mail but when I click on a new one it takes minutes to open.


Additional info:
* package version(s)
mariadb 5.5.33.a-2
akonadi 1.10.3-1
kontact and kmail 4.11.3-1

* config and/or log files etc.

Akonadi Server Self-Test Report
===============================

Test 1: SUCCESS
--------

Database driver found.
Details: The QtSQL driver 'QMYSQL' is required by your current Akonadi server configuration and was found on your system.

File content of '/home/nolwenn/.config/akonadi/akonadiserverrc':
[%General]
Driver=QMYSQL

[QMYSQL]
Name=akonadi
Host=
Options="UNIX_SOCKET=/tmp/akonadi-nolwenn.7KmavE/mysql.socket"
ServerPath=/usr/bin/mysqld
StartServer=true

[Debug]
Tracer=null


Test 2: SUCCESS
--------

Akonadi is not running as root
Details: Akonadi is not running as a root/administrator user, which is the recommended setup for a secure system.

Test 3: SUCCESS
--------

MySQL server found.
Details: You have currently configured Akonadi to use the MySQL server '/usr/bin/mysqld'.
Make sure you have the MySQL server installed, set the correct path and ensure you have the necessary read and execution rights on the server executable. The server executable is typically called 'mysqld'; its location varies depending on the distribution.

Test 4: SUCCESS
--------

MySQL server is executable.
Details: MySQL server found: /usr/bin/mysqld Ver 5.5.33a-MariaDB-log for Linux on x86_64 (Source distribution)


Test 5: ERROR
--------

MySQL server log contains errors.
Details: The MySQL server error log file &apos;<a href='/home/nolwenn/.local/share/akonadi/db_data/mysql.err'>/home/nolwenn/.local/share/akonadi/db_data/mysql.err</a>&apos; contains errors.

File content of '/home/nolwenn/.local/share/akonadi/db_data/mysql.err':
131107 8:17:11 InnoDB: The InnoDB memory heap is disabled
131107 8:17:11 InnoDB: Mutexes and rw_locks use GCC atomic builtins
131107 8:17:11 InnoDB: Compressed tables use zlib 1.2.8
131107 8:17:11 InnoDB: Initializing buffer pool, size = 80.0M
131107 8:17:11 InnoDB: Completed initialization of buffer pool
131107 8:17:11 InnoDB: highest supported file format is Barracuda.
131107 8:17:11 InnoDB: Waiting for the background threads to start
131107 8:17:12 Percona XtraDB (http://www.percona.com) 5.5.33a-MariaDB-31.1 started; log sequence number 90091796509
131107 8:17:12 [Warning] Can't open and lock time zone table: Table 'mysql.time_zone_leap_second' doesn't exist trying to live without them
131107 8:17:12 [ERROR] Can't open and lock privilege tables: Table 'mysql.servers' doesn't exist
131107 8:17:12 [ERROR] Native table 'performance_schema'.'events_waits_current' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'events_waits_history' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'events_waits_history_long' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'setup_consumers' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'setup_instruments' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'setup_timers' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'performance_timers' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'threads' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'events_waits_summary_by_thread_by_event_name' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'events_waits_summary_by_instance' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'events_waits_summary_global_by_event_name' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'file_summary_by_event_name' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'file_summary_by_instance' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'mutex_instances' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'rwlock_instances' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'cond_instances' has the wrong structure
131107 8:17:12 [ERROR] Native table 'performance_schema'.'file_instances' has the wrong structure
131107 8:17:12 [Note] /usr/bin/mysqld: ready for connections.
Version: '5.5.33a-MariaDB' socket: '/tmp/akonadi-nolwenn.7KmavE/mysql.socket' port: 0 Source distribution
131107 11:57:26 [Note] /usr/bin/mysqld: Normal shutdown

131107 11:57:26 InnoDB: Starting shutdown...
131107 11:57:26 InnoDB: Shutdown completed; log sequence number 90287899339
131107 11:57:26 [Note] /usr/bin/mysqld: Shutdown complete



Test 6: SUCCESS
--------

MySQL server default configuration found.
Details: The default configuration for the MySQL server was found and is readable at <a href='/usr/share/config/akonadi/mysql-global.conf'>/usr/share/config/akonadi/mysql-global.conf</a>.

File content of '/usr/share/config/akonadi/mysql-global.conf':
#
# Global Akonadi MySQL server settings,
# These settings can be adjusted using $HOME/.config/akonadi/mysql-local.conf
#
# Based on advice by Kris Köhntopp <kris@mysql.com>
#
[mysqld]

# strict query parsing/interpretation
# TODO: make Akonadi work with those settings enabled
# sql_mode=strict_trans_tables,strict_all_tables,strict_error_for_division_by_zero,no_auto_create_user,no_auto_value_on_zero,no_engine_substitution,no_zero_date,no_zero_in_date,only_full_group_by,pipes_as_concat
# sql_mode=strict_trans_tables

# DEBUGGING:
# log all queries, useful for debugging but generates an enormous amount of data
# log=mysql.full
# log queries slower than n seconds, log file name relative to datadir (for debugging only)
# log_slow_queries=mysql.slow
# long_query_time=1
# log queries not using indices, debug only, disable for production use
# log_queries_not_using_indexes=1
#
# mesure database size and adjust innodb_buffer_pool_size
# SELECT sum(data_length) as bla, sum(index_length) as blub FROM information_schema.tables WHERE table_schema not in ("mysql", "information_schema");

# NOTES:
# Keep Innob_log_waits and keep Innodb_buffer_pool_wait_free small (see show global status like "inno%", show global variables)

#expire_logs_days=3

#sync_bin_log=0

# Use UTF-8 encoding for tables
character_set_server=utf8
collation_server=utf8_general_ci

# use InnoDB for transactions and better crash recovery
default_storage_engine=innodb

# memory pool InnoDB uses to store data dictionary information and other internal data structures (default:1M)
# Deprecated in MySQL >= 5.6.3
innodb_additional_mem_pool_size=1M

# memory buffer InnoDB uses to cache data and indexes of its tables (default:128M)
# Larger values means less I/O
innodb_buffer_pool_size=80M

# Create a .ibd file for each table (default:0)
innodb_file_per_table=1

# Write out the log buffer to the log file at each commit (default:1)
innodb_flush_log_at_trx_commit=2

# Buffer size used to write to the log files on disk (default:1M for builtin, 8M for plugin)
# larger values means less I/O
innodb_log_buffer_size=1M

# Size of each log file in a log group (default:5M) larger means less I/O but more time for recovery.
innodb_log_file_size=64M

# # error log file name, relative to datadir (default:hostname.err)
log_error=mysql.err

# print warnings and connection errors (default:1)
log_warnings=2

# Convert table named to lowercase
lower_case_table_names=1

# Maximum size of one packet or any generated/intermediate string. (default:1M)
max_allowed_packet=32M

# Maximum simultaneous connections allowed (default:100)
max_connections=256

# The two options below make no sense with prepared statements and/or transactions
# (make sense when having the same query multiple times)

# Memory allocated for caching query results (default:0 (disabled))
query_cache_size=0

# Do not cache results (default:1)
query_cache_type=0

# Do not use the privileges mechanisms
skip_grant_tables

# Do not listen for TCP/IP connections at all
skip_networking

# The number of open tables for all threads. (default:64)
table_open_cache=200

# How many threads the server should cache for reuse (default:0)
thread_cache_size=3

# wait 365d before dropping the DB connection (default:8h)
wait_timeout=31536000

[client]
default-character-set=utf8


Test 7: SKIP
--------

MySQL server custom configuration not available.
Details: The custom configuration for the MySQL server was not found but is optional.

Test 8: SUCCESS
--------

MySQL server configuration is usable.
Details: The MySQL server configuration was found at <a href='/home/nolwenn/.local/share/akonadi/mysql.conf'>/home/nolwenn/.local/share/akonadi/mysql.conf</a> and is readable.

File content of '/home/nolwenn/.local/share/akonadi/mysql.conf':
#
# Global Akonadi MySQL server settings,
# These settings can be adjusted using $HOME/.config/akonadi/mysql-local.conf
#
# Based on advice by Kris Köhntopp <kris@mysql.com>
#
[mysqld]

# strict query parsing/interpretation
# TODO: make Akonadi work with those settings enabled
# sql_mode=strict_trans_tables,strict_all_tables,strict_error_for_division_by_zero,no_auto_create_user,no_auto_value_on_zero,no_engine_substitution,no_zero_date,no_zero_in_date,only_full_group_by,pipes_as_concat
# sql_mode=strict_trans_tables

# DEBUGGING:
# log all queries, useful for debugging but generates an enormous amount of data
# log=mysql.full
# log queries slower than n seconds, log file name relative to datadir (for debugging only)
# log_slow_queries=mysql.slow
# long_query_time=1
# log queries not using indices, debug only, disable for production use
# log_queries_not_using_indexes=1
#
# mesure database size and adjust innodb_buffer_pool_size
# SELECT sum(data_length) as bla, sum(index_length) as blub FROM information_schema.tables WHERE table_schema not in ("mysql", "information_schema");

# NOTES:
# Keep Innob_log_waits and keep Innodb_buffer_pool_wait_free small (see show global status like "inno%", show global variables)

#expire_logs_days=3

#sync_bin_log=0

# Use UTF-8 encoding for tables
character_set_server=utf8
collation_server=utf8_general_ci

# use InnoDB for transactions and better crash recovery
default_storage_engine=innodb

# memory pool InnoDB uses to store data dictionary information and other internal data structures (default:1M)
# Deprecated in MySQL >= 5.6.3
innodb_additional_mem_pool_size=1M

# memory buffer InnoDB uses to cache data and indexes of its tables (default:128M)
# Larger values means less I/O
innodb_buffer_pool_size=80M

# Create a .ibd file for each table (default:0)
innodb_file_per_table=1

# Write out the log buffer to the log file at each commit (default:1)
innodb_flush_log_at_trx_commit=2

# Buffer size used to write to the log files on disk (default:1M for builtin, 8M for plugin)
# larger values means less I/O
innodb_log_buffer_size=1M

# Size of each log file in a log group (default:5M) larger means less I/O but more time for recovery.
innodb_log_file_size=64M

# # error log file name, relative to datadir (default:hostname.err)
log_error=mysql.err

# print warnings and connection errors (default:1)
log_warnings=2

# Convert table named to lowercase
lower_case_table_names=1

# Maximum size of one packet or any generated/intermediate string. (default:1M)
max_allowed_packet=32M

# Maximum simultaneous connections allowed (default:100)
max_connections=256

# The two options below make no sense with prepared statements and/or transactions
# (make sense when having the same query multiple times)

# Memory allocated for caching query results (default:0 (disabled))
query_cache_size=0

# Do not cache results (default:1)
query_cache_type=0

# Do not use the privileges mechanisms
skip_grant_tables

# Do not listen for TCP/IP connections at all
skip_networking

# The number of open tables for all threads. (default:64)
table_open_cache=200

# How many threads the server should cache for reuse (default:0)
thread_cache_size=3

# wait 365d before dropping the DB connection (default:8h)
wait_timeout=31536000

[client]
default-character-set=utf8


Test 9: SUCCESS
--------

akonadictl found and usable
Details: The program '/usr/bin/akonadictl' to control the Akonadi server was found and could be executed successfully.
Result:
Akonadi 1.10.3


Test 10: SUCCESS
--------

Akonadi control process registered at D-Bus.
Details: The Akonadi control process is registered at D-Bus which typically indicates it is operational.

Test 11: SUCCESS
--------

Akonadi server process registered at D-Bus.
Details: The Akonadi server process is registered at D-Bus which typically indicates it is operational.

Test 12: SUCCESS
--------

Nepomuk search service registered at D-Bus.
Details: The Nepomuk search service is registered at D-Bus which typically indicates it is operational.

Test 13: SUCCESS
--------

Nepomuk search service uses an appropriate backend.
Details: The Nepomuk search service uses one of the recommended backends.

Test 14: SKIP
--------

Protocol version check not possible.
Details: Without a connection to the server it is not possible to check if the protocol version meets the requirements.

Test 15: SUCCESS
--------

Resource agents found.
Details: At least one resource agent has been found.

Directory listing of '/usr/share/akonadi/agents':
akonadinepomukfeederagent.desktop
akonotesresource.desktop
archivemailagent.desktop
birthdaysresource.desktop
contactsresource.desktop
davgroupwareresource.desktop
facebookresource.desktop
googlecalendarresource.desktop
googlecontactsresource.desktop
icaldirresource.desktop
icalresource.desktop
imapresource.desktop
invitationsagent.desktop
kabcresource.desktop
kalarmdirresource.desktop
kalarmresource.desktop
kcalresource.desktop
kdeaccountsresource.desktop
knutresource.desktop
kolabproxyresource.desktop
localbookmarksresource.desktop
maildirresource.desktop
maildispatcheragent.desktop
mailfilteragent.desktop
mboxresource.desktop
microblog.desktop
mixedmaildirresource.desktop
mtdummyresource.desktop
nepomuktagresource.desktop
newmailnotifieragent.desktop
nntpresource.desktop
notesresource.desktop
openxchangeresource.desktop
pop3resource.desktop
sendlateragent.desktop
skroogeakonadiresource.desktop
vcarddirresource.desktop
vcardresource.desktop

Environment variable XDG_DATA_DIRS is set to '/usr/share:/usr/share:/usr/local/share'

Test 16: ERROR
--------

Current Akonadi server error log found.
Details: The Akonadi server reported errors during its current startup. The log can be found in <a href='/home/nolwenn/.local/share/akonadi/akonadiserver.error'>/home/nolwenn/.local/share/akonadi/akonadiserver.error</a>.

File content of '/home/nolwenn/.local/share/akonadi/akonadiserver.error':
Nepomuk Query Server not available


Test 17: ERROR
--------

Previous Akonadi server error log found.
Details: The Akonadi server reported errors during its previous startup. The log can be found in <a href='/home/nolwenn/.local/share/akonadi/akonadiserver.error.old'>/home/nolwenn/.local/share/akonadi/akonadiserver.error.old</a>.

File content of '/home/nolwenn/.local/share/akonadi/akonadiserver.error.old':
Nepomuk Query Server not available
Control process died, committing suicide!


Test 18: ERROR
--------

Current Akonadi control error log found.
Details: The Akonadi control process reported errors during its current startup. The log can be found in <a href='/home/nolwenn/.local/share/akonadi/akonadi_control.error'>/home/nolwenn/.local/share/akonadi/akonadi_control.error</a>.

File content of '/home/nolwenn/.local/share/akonadi/akonadi_control.error':
"D-Bus communication error 'org.freedesktop.DBus.Error.NoReply': 'Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.'"
"D-Bus communication error 'org.freedesktop.DBus.Error.NoReply': 'Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.'"
"D-Bus communication error 'org.freedesktop.DBus.Error.NoReply': 'Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.'"
"D-Bus communication error 'org.freedesktop.DBus.Error.NoReply': 'Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.'"
"D-Bus communication error 'org.freedesktop.DBus.Error.NoReply': 'Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.'"
ProcessControl: Application /usr/bin/akonadi_imap_resource stopped unexpectedly ( "Process crashed" )
ProcessControl: Application /usr/bin/akonadi_imap_resource stopped unexpectedly ( "Process crashed" )
ProcessControl: Application /usr/bin/akonadi_imap_resource stopped unexpectedly ( "Process crashed" )


Test 19: ERROR
--------

Previous Akonadi control error log found.
Details: The Akonadi control process reported errors during its previous startup. The log can be found in <a href='/home/nolwenn/.local/share/akonadi/akonadi_control.error.old'>/home/nolwenn/.local/share/akonadi/akonadi_control.error.old</a>.

File content of '/home/nolwenn/.local/share/akonadi/akonadi_control.error.old':
"D-Bus communication error 'org.freedesktop.DBus.Error.NoReply': 'Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.'"
"D-Bus communication error 'org.freedesktop.DBus.Error.NoReply': 'Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.'"
"D-Bus communication error 'org.freedesktop.DBus.Error.NoReply': 'Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.'"
"D-Bus communication error 'org.freedesktop.DBus.Error.NoReply': 'Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.'"
"D-Bus communication error 'org.freedesktop.DBus.Error.NoReply': 'Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.'"


This task depends upon

Closed by  Andrea Scarpino (BaSh)
Saturday, 23 November 2013, 22:02 GMT
Reason for closing:  Upstream
Additional comments about closing:  https://bugs.kde.org/show_bug.cgi?id=317 470
Comment by Andrea Scarpino (BaSh) - Thursday, 07 November 2013, 12:10 GMT
  • Field changed: Summary (Kmail and Kontact freeze and high cpu usage after kde 4.11.3 update → [kdepim-kmail] freeze and high cpu usage after kde 4.11.3 update)
  • Field changed: Status (Unconfirmed → Assigned)
  • Task assigned to Andrea Scarpino (BaSh), Sven-Hendrik Haase (Svenstaro)
I also had high CPU usage this morning. I guess it's something about nepomuk mail feeder, do you have it enabled too?
Comment by Andrea Scarpino (BaSh) - Thursday, 07 November 2013, 12:46 GMT
My dmesg say:
[Nov 7 13:42] systemd-journald[99]: Failed to write entry (26 items, 53150387 bytes) despite vacuuming, ignoring: Argument list too long
[ +19.837892] virtuoso-t[1592]: segfault at ffffffffffffffff ip 00000000007fb0fe sp 00007fc863fce030 error 7 in virtuoso-t[400000+a1e000]
Comment by Nolwenn (Nonolapero) - Thursday, 07 November 2013, 13:58 GMT
Nepomuk is enabled for everything except for source code.

Edit: there are also a memory leak, I just SIGTERM kontact because it doesn't want to shutdown and I can see on monitoring a gap of memory use.
Comment by Andrea Scarpino (BaSh) - Thursday, 07 November 2013, 22:10 GMT
Seems I cannot reproduce it anymore here. I guess it did some kind of re-indexing then.
Comment by Tomasz Cebula (tomaszc) - Friday, 08 November 2013, 10:27 GMT
For me it is the same, but only on a computer that uses IMAP. I have a second computer with POP3 kmail working properly.
For IMAP, this high CPU usage not only kmail but also akonadi and mysql.
Comment by Nolwenn (Nonolapero) - Monday, 11 November 2013, 21:04 GMT
I disable email indexing and the behavior is the same: high cpu usage from akonadiserver and mysql.
Comment by Nolwenn (Nonolapero) - Tuesday, 12 November 2013, 15:27 GMT
Here a similary bug https://bugs.kde.org/show_bug.cgi?id=317470

So I delete a duplicate trash folder into "local folder" and cpu usage calm down. For me, the bug is fixed.
Comment by Andrea Scarpino (BaSh) - Saturday, 23 November 2013, 22:01 GMT
At the end I removed my nepomuk repository and I don't have this issue anymore.

Loading...