pambase-20200721.1-2 should fix this, shadow is pending.

Will make the suggested changes to shadow later this week

The direct pam config allows more choices than ENCRYPT_METHOD in /etc/login.defs though, which can only be set to DES, MD5, SHA256, SHA512 or (undocumented) BCRYPT.
pam_unix(8) supports more algorithms, like yescrypt.

Btw, upstream is planning to handle this differently in the future, and rely on libxcrypt's crypt_preferred_method() instead of a (potentially dated) system preference:
https://github.com/linux-pam/linux-pam/pull/84

The pambase-20200721.1-2 change was reverted by pambase 20210605-1: hash with sha512 by default.

Should this be closed as Won't Fix?

The crypt(5) manpage lists sha512crypt as "acceptable" (if properly configured, not by default), but recommends yescrypt for new hashes.

@ghen you want to submit a patch changing all use of sha512 in pam files to yescrypt, adding it to pam_unix password entries that do not specify an encryption method and removing it from /etc/login.defs?
You could also standardize the use of nullok. You could also consider changing physlock and kbd to using upstreams pam configs that use system-auth rather directly using pam_unix.
Edit:
man 5 crypt notes 5000 rounds is too low for modern hardware for sha512/sha256 but does not provide a recommended number of rounds.
Does the libxcrypt project have a recommendation for the number of rounds that currently could be used? That could then be set in upstream shadows login.defs.
Or patch shadow with yescrypt support [1] and add the option there?

[1] https://github.com/shadow-maint/shadow/commit/5cd04d03f94622c12220d4a6352824af081b8531

I've added the options to build with bcrypt and yescrypt support in SVN (https://github.com/archlinux/svntogit-packages/commit/7d5415618a1e99bb57b873e17224cb8f254d7280).

FWIW, it appears that Fedora builds shadow without libpam support (https://src.fedoraproject.org/rpms/shadow-utils/blob/rawhide/f/shadow-utils.spec#_155)
I wonder why that is and how they configure PAM instead.

With pam, pam_unix.so uses ENCRYPT_METHOD from longin.defs [1] overridden by des md5 bigcrypt sha256 sha512 blowfish gost_yescrypt yescrypt [2] and SHA_CRYPT_MAX_ROUNDS [3] overridden by rounds=n. The rest of the pam stack is used.

Without pam ENCRYPT_METHOD supported values des md5 sha256 sha512 bcrypt yescrypt, SHA_CRYPT_MIN_ROUNDS SHA_CRYPT_MAX_ROUNDS BCRYPT_MIN_ROUNDS BCRYPT_MAX_ROUNDS YESCRYPT_COST_FACTOR [4] may be used.

des md5 sha256 sha512 bcrypt yescrypt intersection with des md5 bigcrypt sha256 sha512 blowfish gost_yescrypt yescrypt is des md5 sha256 sha512 yescrypt eliminating des and md5 leaves sha256 sha612 yescrypt.
scrypt bcrypt sha1crypt SunMD5 bsdicrypt NT are supported by libxcrypt [5] but not by pam_unix [2].

[6] Changes the ENCRYPT_METHOD documentation in login.defs to list the methods supported by pam_unix and removes the per method rounds settings.
[7] and [8] Adds a ROUNDS option to login.defs and changes pam_unix's undocumented support for SHA_CRYPT_MAX_ROUNDS to support it mirroring the rounds option pam_unix already supports.

#SHA_CRYPT_MAX_ROUNDS 5000
pam_unix min 1000 max 9999999 default 5000

#YESCRYPT_COST_FACTOR 5
pam_unix min 3 max 11 default 5

How does Fedora avoid a different number of rounds and or encryption method being used between tools that use pam such login when a password change is forced due to it being expired compared to passwd which Fedora builds without pam?

[1] https://github.com/linux-pam/linux-pam/blob/v1.5.2/modules/pam_unix/support.c#L85
[2] https://github.com/linux-pam/linux-pam/blob/v1.5.2/modules/pam_unix/support.h#L109
[3] https://github.com/linux-pam/linux-pam/blob/v1.5.2/modules/pam_unix/support.c#L103
[4] https://man.archlinux.org/man/login.defs.5
[5] https://man.archlinux.org/man/crypt.5
[6] shadow-encrypt-methods-supported-by-pam-unix.patch
[7] shadow-rounds.patch
[8] pam-rounds.patch