FS#68488 - [python-tensorflow][python-h5py] tensorflow cannot load Keras models in h5 with h5py 3.0

Attached to Project: Community Packages
Opened by Chih-Hsuan Yen (yan12125) - Sunday, 01 November 2020, 05:50 GMT
Last edited by Sven-Hendrik Haase (Svenstaro) - Thursday, 18 February 2021, 10:59 GMT
Task Type Bug Report
Category Packages
Status Closed
Assigned To Sven-Hendrik Haase (Svenstaro)
Bruno Pagani (ArchangeGabriel)
Konstantin Gizdov (kgizdov)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 2
Private No

Details

Description:
Apparently h5py has changed its API in a backward-incompatible manner. I got the following error when I tried to load a Keras model in h5:

Traceback (most recent call last):
File "t.py", line 9, in <module>
model.load_weights(os.path.join(squeezenet_cifar10_path, 'squeeze_net.h5'))
File "/usr/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 2211, in load_weights
hdf5_format.load_weights_from_hdf5_group(f, self.layers)
File "/usr/lib/python3.8/site-packages/tensorflow/python/keras/saving/hdf5_format.py", line 660, in load_weights_from_hdf5_group
original_keras_version = f.attrs['keras_version'].decode('utf8')
AttributeError: 'str' object has no attribute 'decode'

Additional info:
python-h5py 3.0.0-1
python-tensorflow-opt 2.3.1-2 # I believe all tensorflow split packages are affected

Related upstream ticket: https://github.com/tensorflow/tensorflow/issues/44467

Steps to reproduce:
1. git clone https://github.com/yan12125/SqueezeNet_vs_CIFAR10
2. Run the following script

import os.path

import tensorflow as tf

squeezenet_cifar10_path = './SqueezeNet_vs_CIFAR10/models'
with open(os.path.join(squeezenet_cifar10_path, 'squeeze_net.json')) as f:
model_json = f.read()
model = tf.keras.models.model_from_json(model_json)
model.load_weights(os.path.join(squeezenet_cifar10_path, 'squeeze_net.h5'))

FWIW, by patching two lines in tensorflow, I can load this model

--- /usr/lib/python3.8/site-packages/tensorflow/python/keras/saving/hdf5_format.py 2020-10-05 13:08:53.000000000 +0800
+++ /usr/lib/python3.8/site-packages/tensorflow/python/keras/saving/hdf5_format.py.new 2020-11-01 13:35:07.502299844 +0800
@@ -657,11 +657,11 @@
and weights file.
"""
if 'keras_version' in f.attrs:
- original_keras_version = f.attrs['keras_version'].decode('utf8')
+ original_keras_version = f.attrs['keras_version']
else:
original_keras_version = '1'
if 'backend' in f.attrs:
- original_backend = f.attrs['backend'].decode('utf8')
+ original_backend = f.attrs['backend']
else:
original_backend = None

Not sure if those are all needed changes in tensorflow, though.
This task depends upon

Closed by  Sven-Hendrik Haase (Svenstaro)
Thursday, 18 February 2021, 10:59 GMT
Reason for closing:  Fixed
Comment by Sven-Hendrik Haase (Svenstaro) - Monday, 02 November 2020, 08:37 GMT
Check 2.3.1-4.
Comment by Chih-Hsuan Yen (yan12125) - Monday, 02 November 2020, 09:43 GMT
Thanks! I can confirm tensorflow 2.3.1-4 works for me.
Comment by piernov (piernov) - Wednesday, 17 February 2021, 23:50 GMT
  • Field changed: Percent Complete (100% → 0%)
Reopening because the patch added as of 2.3.1-4 is incomplete, still causes error such as:
File "/usr/lib/python3.9/site-packages/tensorflow/python/keras/saving/hdf5_format.py", line 182, in load_model_from_hdf5
model_config = json_utils.decode(model_config.decode('utf-8'))
AttributeError: 'str' object has no attribute 'decode'

A PR was merged into tensorflow master branch, but it would be nice to have the patch in 2.4.0 on ArchLinux:
https://github.com/tensorflow/tensorflow/pull/45487
Comment by Konstantin Gizdov (kgizdov) - Thursday, 18 February 2021, 10:08 GMT
I took that patch and adjusted it to pass on 2.4.1. Currently rebuilding.

Loading...