FS#66023 - [samba] and MIT KDC
Attached to Project:
Arch Linux
Opened by Pol Bettinger (Arvoreen) - Monday, 30 March 2020, 04:08 GMT
Last edited by Tobias Powalowski (tpowa) - Friday, 14 August 2020, 14:00 GMT
Opened by Pol Bettinger (Arvoreen) - Monday, 30 March 2020, 04:08 GMT
Last edited by Tobias Powalowski (tpowa) - Friday, 14 August 2020, 14:00 GMT
|
Details
Description:
Samba compiled with --with-system-mitkrb5 and --with-experimental-mit-ad-dc fails to launch. Trying to launch samba after the update failed with Mar 30 05:09:30 dc-1 krb5kdc[5248]: Cannot open DB2 database '/var/lib/krb5kdc/principal': No such file or directory - while initializing database for realm LOCAL.ARVOREEN.NET downgrading to samba-4.11.2-1 helped samba-4.11.2-2 did not Why is this experimental feature configured? Even if one configures MIT krb5 there are still alot of upstream bugs on this feature https://wiki.samba.org/index.php/Running_a_Samba_AD_DC_with_MIT_Kerberos_KDC#Known_Limitations_of_MIT_Kerberos_Support_in_Samba Additional info: * samba-4.11.2-2 and abobe Steps to reproduce: Using samba as domain server with version newer then samba-4.11.2-1 |
This task depends upon
Closed by Tobias Powalowski (tpowa)
Friday, 14 August 2020, 14:00 GMT
Reason for closing: Fixed
Additional comments about closing: 4.12.6-1
Friday, 14 August 2020, 14:00 GMT
Reason for closing: Fixed
Additional comments about closing: 4.12.6-1
ldb-1:1.5.6-2-x86_64.pkg.tar.xz samba-4.10.10-2-x86_64.pkg.tar.xz smbclient-4.10.10-2-x86_64.pkg.tar.xz libwbclient-4.10.10-2-x86_64.pkg.tar.xz
Samba is also using the system krb (/etc/krb5.conf) config rather than the samba one (/var/lib/samba/private/krb5.conf), which I'm assuming is due to the switch from the in tree KRB to the MIT KRB. Although after configuring this file it resulted in the same error above.
samba[665]: root process[665]: exit_daemon: daemon failed to start: Samba failed to prime database, error code 22
This is because Samba 4.11 upgraded the database:
https://wiki.samba.org/index.php/Downgrading_an_Active_Directory_DC#Samba_failed_to_prime_database.2C_error_code_22
As per the article, I had to upgrade to Samba 4.11 again, run 'samba_downgrade_db', then revert again to Samba 4.10.10.2. Following this, all Samba services appears to be working normally.
@tpowa @jelle, @anthraxx, please consider setting up a test VM following the wiki and test your future updates, see here: https://wiki.archlinux.org/index.php/Samba/Active_Directory_domain_controller The maintenance burden is tiny. I wrote a significant portion of it and can do a complete new domain setup from bare metal (or rather virtual metal) in about 20 minutes.
I'm going to see about switching, but I would strongly suggest a new 4.11.3-4 release (working on it locally, forward from 4.11.2-1, as soon as I'm done with this message) and staying with the built in KDC for now. Using MIT is simply not yet ready for prime time yet (but it may be too late since it is already in released packages). If this is not possible at this point, please respond here (or email me) so that I can at least put up the three (-internal-kdc or -heimdal, or whatever) packages in the AUR for existing users in the interim. I'd also like to request that a simple blurb `echo "WARNING!!!! This upgrade will break AD DCs, see <above link>"` in the upgrade section of the install file about this in the interim if this is the case, maybe link to the future AUR package until I (or anybody else) can get the upgrade path sorted in the wiki, at which point, link to the sub portion of the wiki).
Thanks for your consideration. I'll post back here after some initial testing.
-- DJ Lucas
https://aur.archlinux.org/packages/ldb-heimdal/
https://aur.archlinux.org/packages/samba-heimdal/
FYI, I did have to use -dd to install because of my really old samba-dhcp-update package. I'll fix that momentarily. If you use these, once the official packages are capable of supporting AD-DC update (whether no longer broken, or an upgrade path is determined), you will need to explicitly issue 'pacman -S ldb libwbclient smbclient samba' to return to the offical repos. Now, off to figure out how to upgrade on a not semi-productions system. :-)
This worked on a clean build machine without issue. I'm not sure what the answer is regarding the lost functionality for RODC and GPO processing. IIUC, the machine group membership will be fixed in 4.12.2 IIUC, but I'm not sure about RODC and the others.
I tried to recompile sssd 2.2.3-6, but 2-3 test always fails and the performance of these test are much slower, than it was with samba 4.11.3:
FAIL: test_ipa_subdom_server
FAIL: sysdb_ssh-tests
FAIL: sysdb-tests
I hope this helps!
"net ads join" is failing with:
"secrets_domain_info_kerberos_keys: generation of a des-cbc-md5 key failed: Bad encryption type"
With 4.12.1 in testing I can join domain once again, but I'm also using SSSD, and it looks like there are some issues.
There are patches around for 4.11 that should fix the issue with des-cbc-md5 retiring (see https://src.stg.fedoraproject.org/rpms/samba/tree/).
Could they be applied for the stable package?
I did finally add the TBA section about adding a DC to an existing domain in our wiki (got the necessary gotchas and proper configuration order in there as it's kind of painful to jump around on the Samba wiki) and provided necessary links in the Samba wiki about transferring FSMO roles and demoting, cleaning out /var/lib/samba/private, etc..
Obviously it's your distro, do what is appropriate for the most users, but my opinion stands - I think Arch should roll back to internal Heimdal until these issues are ironed out. This change effectively created a major update issue and broke six(?) server features (two of which are major IMO) to introduce one new client feature (how many users does each affect?). That said, using MIT krb5 absolutely *will* be the better method once the broken functionality is restored. See https://wiki.samba.org/index.php/Roadmap_MIT_KDC for the list. As I said before, I'm going to proceed for now with the -heimdal packages I threw up in the AUR, but I'd like to strongly encourage the Arch maintainers to consider rolling back this change or bringing something similar back into the Arch repos (depending on how the package is most used), at least until Samba/MIT is ready for prime time. Unfortunately, split packages are not easily doable - dual builds and not enough overlap to justify it - the four separate packages are better. Also, with the ldb dependency looking for a specific version going forward, you might want to consider bringing ldb back into the PKGBUILD for Samba (just separate it in packaging to handle those few dependent packages that need it).
Here the links to arch archive...
wget https://archive.archlinux.org/packages/s/samba/samba-4.11.2-1-x86_64.pkg.tar.xz
wget https://archive.archlinux.org/packages/l/libwbclient/libwbclient-4.11.2-1-x86_64.pkg.tar.xz
wget https://archive.archlinux.org/packages/s/smbclient/smbclient-4.11.2-1-x86_64.pkg.tar.xz
and how to install them
pacman -U libwbclient-4.11.2-1-x86_64.pkg.tar.xz samba-4.11.2-1-x86_64.pkg.tar.xz smbclient-4.11.2-1-x86_64.pkg.tar.xz
Have fun
Running samba 4.11.6 without MIT Kerberos since 2020-02-10 without problems.
It would be nice if a package maintainer would report here.
Meanwhile there are at least 3 bugs reported as well as in the forum regarding enabling MIT Kerberos.
FS#64537FS#62521FS#66023Apr 21 22:18:00 mycl-01 samba[1123]: [2020/04/21 22:18:00.402005, 0] ../../source4/libcli/dgram/browse.c:106(dgram_mailslot_browse_parse)
Apr 21 22:18:00 mycl-01 samba[1123]: Failed to parse browse packet of length 2: NT_STATUS_BUFFER_TOO_SMALL
Apr 21 22:18:00 mycl-01 samba[1123]: [2020/04/21 22:18:00.435036, 0] ../../source4/libcli/dgram/browse.c:106(dgram_mailslot_browse_parse)
Apr 21 22:18:00 mycl-01 samba[1123]: Failed to parse browse packet of length 2: NT_STATUS_BUFFER_TOO_SMALL
...
Apr 21 22:28:44 mycl-01 named[955]: samba_dlz: starting transaction on zone devnet.lan
Apr 21 22:28:46 mycl-01 named[955]: client @0x7fc6a00837d0 192.168.122.104#49638: updating zone 'devnet.lan/NONE': update unsuccessful: my-arcolinux-01.devnet.lan/A: 'RRset exists (value dependent)' prerequisite not satisfied (NXRRSET)
Apr 21 22:28:46 mycl-01 named[955]: samba_dlz: cancelling transaction on zone devnet.lan
Apr 21 22:28:46 mycl-01 named[955]: samba_dlz: starting transaction on zone devnet.lan
Apr 21 22:28:47 mycl-01 named[955]: samba_dlz: spnego update failed
Apr 21 22:28:47 mycl-01 named[955]: client @0x7fc6a00837d0 192.168.122.104#49638: updating zone 'devnet.lan/NONE': update failed: rejected by secure update (REFUSED)
Apr 21 22:28:47 mycl-01 named[955]: samba_dlz: cancelling transaction on zone devnet.lan
On the client side:
DNS Update for my-arcolinux-01.devnet.lan failed: ERROR_DNS_GSS_ERROR
DNS update failed: NT_STATUS_UNSUCCESSFUL
I hope this helps!
Lots of thanks to Arvoreen!
My AD runs (almost) perfekt with 4.11.2.1 - despite of the non working DNS update. It was really desireable if we had a solution for that soon.
This means that the administration of the domain is still hardly possible ... even the migration to MIT kerberos described above is hardly feasible without the samba tool
kinit -V
Standardzwischenspeicher wird verwendet: /tmp/krb5cc_1000
verwendeter Principal: klaus@SOMEDOM.DE
kinit: KDC für Realm »SOMEDOM.DE« kann nicht gefunden werden bei Anfängliche Anmeldedaten werden geholt.
Samba crashes every time a windows client logs into the AD.
Mai 01 17:06:31 arch mitkdc[1094]: [2020/05/01 17:06:31.725738, 0] ../../lib/util/util_runcmd.c:352(samba_runcmd_io_handler)
Mai 01 17:06:31 arch mitkdc[1094]: /usr/sbin/krb5kdc: krb5kdc: startet …
Mai 01 17:10:07 arch mitkdc[1094]: [2020/05/01 17:10:07.273215, 0] ../../source4/kdc/kdc-service-mit.c:344(mitkdc_server_done)
Mai 01 17:10:07 arch mitkdc[1094]: The MIT KDC daemon died with exit status 6
Mai 01 17:10:07 arch mitkdc[1094]: [2020/05/01 17:10:07.273303, 0] ../../source4/smbd/service_task.c:36(task_server_terminate)
Mai 01 17:10:07 arch mitkdc[1094]: task_server_terminate: task_server_terminate: [mitkdc child process exited]
Mai 01 17:10:07 arch samba[1071]: [2020/05/01 17:10:07.277953, 0] ../../source4/smbd/server.c:377(samba_terminate)
Mai 01 17:10:07 arch samba[1071]: samba_terminate: samba_terminate of samba 1071: mitkdc child process exited
Mai 01 17:10:07 arch systemd[1]: samba.service: Main process exited, code=exited, status=1/FAILURE
Mai 01 17:10:07 arch systemd[1]: samba.service: Failed with result 'exit-code'.
I opened
FS#66494for that. The problem looks rather similar to a former samba bug: https://bugzilla.samba.org/show_bug.cgi?id=13057#c3Archlinux is known to be a good bleeding edge distribution, but not a
distribution for stable productive environments. I guess this is why
they also decided to switch to an unsupported Kerberos implementeation?
If they consider to ship usable software, then you should open a bug
report for Archlinux that MIT krb5 is *not* supported and known to be
unusable with Samba AD. I will send you a samba bugzilla account mail
but you should concentrate on getting the Archlinux people on the right
track again.
Perhaps it was better to switch back to Heimdal, as "MIT krb5 is *not* supported" ?
this morning I took the PKGBUILDs from your AUR packages (ldb-heimdal, samba-heimdal), changed the versions to 2.1.2 (ldb) and 4.12.2 (samba) and built/installed both packages.
Up to now everything looks fine.
I really don't understand why people here insist on MIT KDC
Dear maintainers - I am very, very thankful for the job you do, even if it does not always work for everyone. Ideally I would like to send you a pull request to build a separate samba-mit* packages so we can keep using Heimdal in samba* packages 4.12+ ; trouble is I do not know where to send the PR to, I thought Arch is just about to migrate to gitlab so this should be possible soon?
Those packages keep up with the latest changes made in the official repositories and provide and replacement for the samba package. https://aur.archlinux.org/packages/samba-heimdal/
I'm also providing pre-compiled packages for those not wanting to compile each and every update in my own unofficial repository: https://wiki.archlinux.org/index.php/Unofficial_user_repositories#disastrousaur
[1] https://bbs.archlinux.org/viewtopic.php?pid=1847402#p1847402
[2] https://github.com/samba-team/samba/commit/84aad2ea7d53d8d7645bbd9e4c9ce6b09f6016c5
Can everyone else confirm that problem? If you use Windows 10, you have to install the RSAT tools manually: https://support.microsoft.com/en-us/help/2693643/remote-server-administration-tools-rsat-for-windows-operating-systems
I have to say that I also tried that again with the provided AUR package (samba-heimdal) and experienced the same problem. Right now I am wondering if the reason is the missing flag "--enable-gnutls" (might explain why a connection "suddently" fails). It was removed with samba 4.11.2-1 (https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/samba&id=8a8f642c0d843c49a7cbca6dd30879e557d8c762) and this flag was also not used when building the AUR package. I rolled back to 4.10.x and everything works very well.
I actually have to say that I am surprised that the flag "--enable-gnutls" was removed with "4.11.2-1" because the documentation actuallay says "To use TLS, Samba has to be compiled with --enable-gnutls". So I actually think that the current build is not able to establish a TLS based connection. (Source: https://wiki.samba.org/index.php/Configuring_LDAP_over_SSL_(LDAPS)_on_a_Samba_AD_DC).
@jelly
Would it be possible to update the package in [testing] with a version where "--enable-gnutls" is enabled? Thank you!
from: https://wiki.samba.org/index.php/Configuring_LDAP_over_SSL_(LDAPS)_on_a_Samba_AD_DC
Further reference: https://wiki.samba.org/index.php/Samba_Features_added/changed#Removing_in-tree_cryptography:_GnuTLS_3.4.7_required
I am running the AUR build without problems ... what issues do you have with the RSAT?
When I start RSAT I get immediately the follwing message: "Naming information cannot be located" -unfortunately without any information. It sounds like the client does not find the server but it is actually reachable like I can access the shares, the hostname is correctly resolved, and it accepts my login credentials. When I confirm the error message which is raised by RSAT, the tools starts but the domain is not there. I also noticed that the "netlogon"-scripts are not executed/loaded. I also removed one client/computer completely from the domain and re-joined -that worked without any problems but the mentioned problems (RSAT and netlogon-scripts) are still there :(
So RSAT is working for you (@ch888)? Thanks so far :)
When available, I installed the AUR packages (as far as I remember) without making any changes.
> Kerberos: Server not found in database: ... encryption type 3 not supported
So it really seems to be a problem with the encryption. I just don't understand to what "type 3" refers -and- if the client wants that or the server is trying to force it. Does anyone have an idea?
[1] https://github.com/samba-team/samba/blob/v4-11-stable/source4/heimdal/lib/asn1/krb5.asn1#L226
[2] https://github.com/samba-team/samba/blob/v4-11-stable/source4/heimdal/lib/krb5/crypto-algs.c#L77
Thank you! That is very helpful. I just don't understand why it wants so use "DES-CBC-MD5". My client is also "Windows 10 1909" and I didn't modify the default settings (in respect of the encryption). So why does it work for DJ_Luca and ch888?
@DJ_Lucas and @ch888
Would you mind and execute the following command on your samba server:
1. kinit administrator@DOMAIN
2. net ads enctypes list Administrator
This should print the bitmask of the attribute "msDS-SupportedEncryptionTypes" for the Administrator account. Thank you!
Looking at the code shows that "not set" or 0x0 will cause the default behaviour dependent on the type of the requestor.
Starting with line 338 in https://git.samba.org/?p=samba.git;a=blob;f=source4/kdc/db-glue.c;h=023ae7b580d672377ea127866d54e378b9b36508;hb=659c8c3d733b3de88664460f3a2f72ccb0448a2a
What is the output of your 'klist -e' (mine are 'Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96' with the default settings), but maybe you have encryption types configured in your kerberos config files?
FYI, IPv6 is completely disabled here.
I actually setup the ADDC from scratch (in parallel) and everything works fine (with the package in [Testing]). The settings are the same as in my "old" ADDC environment -so far, I can only guess what might cause the described problem. The "old" setup was initally created with SAMBA 4.1.x so it was already running for a while. I have to admit that I realized at some point that I never updated the schema -so it was still "Windows Server 2003" (You can check your schema my running "gpmc.msc" on your Windows Client and inspecting the properties of the Domain server). I updated the schema to "2008 R2" as it was described in the documentation but this didn't solve my problem. I am wondering if somewhere in the policies of my Domain I have a (legacy) "hook" which forces to use the old encryption because the Domain "thinks" only the legacy encryptions are available. I tried to change a couple of entries in the group policy to force the new encryption but it didn't work. How reasonable does my guess actually sound? -thanks again!
I am likely facing the same problem when upgrading to Samba 4.12 on my Gentoo box. My Google foo ha redirected me here and I think I can add some findings...
My Samba AD Domain controller was upgraded all the way up since Samba 3.x with OpenLDAP. So also I was perhaps a bit lazy upgrading the AD structures etc.
As it looks to me my domain controller was also missing the msDS-SupportedEncryptionTypes attribute in AD (cn=server,ou=Domain Controllers,dc=ad,dc=somedom,dc=tld). Actually I found that this attribute is described in MS-AD_Schema_2K8_R2_Attributes.txt which makes me believe it should be suppored in AD DC 2008R2 feature level.
But then I realized that my DC was still on feature level "server 2003". Check out yours by using "samba-tool domain level show". This has lead me to https://wiki.samba.org/index.php/Raising_the_Functional_Levels for upgrade instructions. So I did the following:
- samba-tool domain level raise
- samba-tool domain level raise --domain-level=2008_R2
- samba-tool domain level raise --forest-level=2008_R2
The goal was to rise the feature level of my DC, domain and forest all the way up to 2008R2 (or newer). Actually the page claims Samba DC also supports 2012_R2 feature level but my level never got higher than 2008R2 - which should be fine though.
Although I still got the "encryption type 3 not supported" messages in the logs and my AD does not work.
So I digged further and also found to run:
- samba domain schemaupgrade
Still this did not resolve my problem.
At some point in time I also did run some checks:
- samba-tool dbcheck
- samba-tool dbcheck --cross-ncs
Which revealed a lot of errors which I fixed then using:
- samba-tool dbcheck --fix
- samba-tool dbcheck --cross-ncs --fix
Still this did not resolve the "encryption type 3" issue. Also "net ads enctypes list Administrator" did show the "encryption type 3 not suppored" issue.
So I was thinking why my clients try to use an outdated encryption type on my DC. And sure this is related to the msDS-SupportedEncryptionTypes attribute (which can be set on a user object but the most important attribute is the one on the DC object).
So I browsed my LDAP and found after all the upgrades done above the msDS-SupportedEncryptionTypes attribute exists on my DC object (cn=server,ou=Domain Controllers,dc=ad,dc=somedom,dc=tld) but it's value is set to "31".
I am not yet fully sure about the correct value for a 2008R2 DC but found in some forum that a value of 28 corresponds to RC4,AES128,AES256 so I updated the attribute:
- msDS-SupportedEncryptionTypes=28
So what shall I say... my problem is resolved now. My Clients do not seem to use DES encryption any more and everything seems to work fine up to now.
I might still have to investigate the proper value for msDS-SupportedEncryptionTypes but 28 currently works fine for me using only Windows 10 2004 clients.
All the fuzz likely came from the fact the Samba team removed insecure encryption (DES) from the built-in heimdal implementation.
I think it might be a Samba bug that the msDS-SupportedEncryptionTypes is not automatically updated to reflect the Kerberos implementation capabilities.
Please note that I am not blaming anyone removing insecure encryption types. Actually this is a good thing and it allows all of us to review and fix this security issue. However I do not yet fully understand why msDS-SupportedEncryptionTypes is not automatically overwritten with a valid value on Samba startup and in fact can even be set to an inconsistent value stating DES is supposted while Samba does not support it any more.
Note: I am using Samba with the built-in Heimdal library. I did use the same Samba installations with system-mitkrb5 previously to use MIT-KRB5 but a known issue causing computer GPOs not to be applied forced me to roll back to built-in Heimdal (see https://wiki.samba.org/index.php/Running_a_Samba_AD_DC_with_MIT_Kerberos_KDC).
I hope this might help others to solve their issues too.
Likely the issue does not appear on a freshly created Sabma 4.12 Domain where the feature level and msDS-SupportedEncryptionTypes is set properly from the beginning.
Edit: I found I can list the supported encryption types using "net ads enctypes list" also on computer accounts including DC:
- net ads enctypes list server\$
On my Samba 4.12 it now shows the following output:
'server$' uses "msDS-SupportedEncryptionTypes": 28 (0x0000001c)
[ ] 0x00000001 DES-CBC-CRC
[ ] 0x00000002 DES-CBC-MD5
[X] 0x00000004 RC4-HMAC
[X] 0x00000008 AES128-CTS-HMAC-SHA1-96
[X] 0x00000010 AES256-CTS-HMAC-SHA1-96
So I personally believe 28 is the correct bitmask, disabling/disallowing any insecure DES encryption, allowing only RC4 and AES.
Just wanted to give you a huge THANK YOU! Your information here solved the "encryption type 3 not supported" issue for me as well. I appreciate your detailed instructions here and am happy to say things are back on track with my Samba install again!
Thank you very much for your post! Sometimes it just feels like a waste of time -so it is very nice to see that I am not the only one with this problem and that I also was on the right track (in respect of the reason of the problem and the solution). I did everything what you described and I experienced the same -but it didn't work. I think now I know why. I think one important point which is missing here, is that you (probably) changed/checked the "msDS-SupportedEncryptionTypes" attribute of your computer machine account "server$" (indicated by the $). I only checked the user accounts (without the $ at the end). Can you confirm that? In general, it would be nice to have maybe an entry in the arch wiki about that problem. I am probably just the wrong person to write that but I am happy to help. Again, thank you! *hugs*
In Samba there are 2 types of accounts, machine and user accounts. Machine account always end with a $.
It does not make sense to set allowed encryption types on each user account individually. So yes, I confirm I only checked the machine account of my domain controller (server$) to fix the problem. None of my user accounts do have allowed encryption types set, so none of them do have the msDS-SupportedEncryptionTypes attribute.
You also don't need to set this attribute on any of the client machine accounts. It's sufficient to set it on the Kerberos server(s) I believe.