Arch Linux

Thank you for the report. It indeed should be fixed. I grepped osquery sources and found that osquery/tables/system/posix/augeas.cpp and osquery.conf comments also reference /opt/osquery. This needs to be fixed as well.

Please check osquery 5.0.1-2 at the testing repo and let me know if you still any issues.

Yes it's working now. Thanks for fixing.

I get another error now where I am unsure whether it's coming from the switch to 5.0, from removing some patches or just corruption of the DB on my side:

Okt 01 09:11:32 xps13 osqueryd[40300]: I1001 09:11:32.587299 40300 rocksdb.cpp:67] RocksDB: [WARN] [db/db_impl/db_impl_compaction_flush.cc:1217] [configurations] [JOB 0] Compaction error: Corruption: Unsupported compression method or corrupted compressed block contents: Snappy
Okt 01 09:11:32 xps13 osqueryd[40300]: I1001 09:11:32.587396 40300 rocksdb.cpp:165] Cannot compact column family queries: Corruption: Unsupported compression method or corrupted compressed block contents: Snappy

After the next restart I get the following:

Okt 01 09:16:03 xps13 osqueryd[40300]: W1001 09:16:03.990754 40300 rocksdb.cpp:249] Backing up RocksDB database: /var/osquery/osquery.db.backup
Okt 01 09:16:03 xps13 osqueryd[40300]: W1001 09:16:03.990839 40300 rocksdb.cpp:256] Destroying RocksDB database due to corruption

With this a new DB is created and yields new host identifiers => duplicated hosts in fleet.

I can also create a separate ticket for this, if you've an idea whether it's an upstream or packaging issue.
My suspicion is that previously with linked libraries we were using rocksdb 6.23 with enabled Snappy support (see https://github.com/facebook/rocksdb/blob/35d8e36ef1b8e3e0759ca81215f855226a0a54bd/CMakeLists.txt#L132) and now we're back at 6.14 that does not contain Snappy.

To get it fixed we'd need to add Snappy to the osquery deps. I'm wondering whether we could just enable WITH_SNAPPY and link to the arch provided library - although I know you want the package to be more based on the default osquery setting.

Another idea would be submitting a patch for the scenario to osquery. The rocksdb table holding the device identifier is not compressed with Snappy, hence the identifier could be recovered before trashing the DB.

Edit: The backup/destroy happens on shutdown of 1st start, not on 2nd start.

Indeed with the recent osquery package the rocksdb dependency version went backwards to 6.14.5. So either upstream needs to update this (and other) dependency, or we should do it at our side or try to enable snappy with some other way.

I do not see a simple way to enable rocksdb/snappy. The osquery has its own version of build cmake file here https://github.com/osquery/osquery/blob/master/libraries/cmake/source/rocksdb/CMakeLists.txt that differs from the upstream rocksdb.

Could you please file this snappy request to osquery upstream to track it there?

I've created a working version with snappy that loads the old DB. Basically I hooked in snappy as another dependency and compiled rocksdb with snappy enabled. I can send a PR for review to your osquery fork on GitHub if you like.

> I can send a PR for review to your osquery fork on GitHub if you like.

Please do, I am interested to look at this change.

Here we go, but no rocket science involved: https://github.com/anatol/osquery/pull/1

Something else I'd like to tackle (but not for fixing 5.0.1) is getting support of the packman_packages table back in. I took the same approach as with Snappy and included libalpm as a source dependency. So far I struggled with getting it compiled correctly, but I did not give up yet ;) I have to admit that I'm more confident with Java/C# than C/C++ and CMake. My goal would be to submit this patch directly upstream.

> So far I struggled with getting it compiled correctly, but I did not give up yet ;)

This sounds great!

I can help you with C/C++ part. I am not proficient in CMake though.

Thank you for your patches. I made a minor modification to generate required snappy-stubs-public.h.

Please look at osquery-5.0.1-3 that includes these changes.

Thanks. I've tested 5.0.1-3 and it works on my machine.

I'll follow-up regarding pacman_packages table on GitHub once I find time and have something working.