当前位置 博文首页 > weights、load代码详解_wanggao的专栏：keras读取h5文件load
关于保存h5模型、权重网上的示例非常多，也非常简单。主要有以下两个函数：
1、keras.models.load_model() 读取网络、权重
2、keras.models.load_weights() 仅读取权重
load_model代码包含load_weights的代码，区别在于load_weights时需要先有网络、并且load_weights需要将权重数据写入到对应网络层的tensor中。
下面以resnet50加载h5权重为例，示例代码如下
import keras
from keras.preprocessing import image
import numpy as np
from network.resnet50 import ResNet50
#修改过，不加载权重（默认官方加载亦可）
model = ResNet50()
# 参数默认 by_name = Fasle， 否则只读取匹配的权重
# 这里h5的层和权重文件中层名是对应的（除input层）
model.load_weights(r'\models\resnet50_weights_tf_dim_ordering_tf_kernels_v2.h5')
模型通过 model.summary()输出
def load_weights(self, filepath, by_name=False, skip_mismatch=False, reshape=False):
if h5py is None:
raise ImportError('`load_weights` requires h5py.')
with h5py.File(filepath, mode='r') as f:
if 'layer_names' not in f.attrs and 'model_weights' in f:
f = f['model_weights']
if by_name:
saving.load_weights_from_hdf5_group_by_name(
f, self.layers, skip_mismatch=skip_mismatch,reshape=reshape)
else:
saving.load_weights_from_hdf5_group(f, self.layers, reshape=reshape)
这里关心函数saving.load_weights_from_hdf5_group(f, self.layers, reshape=reshape)即可，参数 f 传递了一个h5py文件对象。
读取h5文件使用 h5py 包，简单使用HDFView看一下resnet50的权重文件。
这里就简单介绍，后面在代码中说明h5py如何读取权重数据。
1、找出keras模型层中具有weight的Tensor(tf.Variable)的层
def load_weights_from_hdf5_group(f, layers, reshape=False):
# keras模型resnet50的model.layers的过滤
# 仅保留layer.weights不为空的层，过滤掉无学习参数的层
filtered_layers = []
for layer in layers:
weights = layer.weights
if weights:
filtered_layers.append(layer)
filtered_layers为当前模型resnet50过滤（input、paddind、activation、merge/add、flastten等）层后剩下107层的list
2、从hdf5文件中获取包含权重数据的层的名字
前面通过HDFView看过每一层有一个[“weight_names”]属性，如果不为空，就说明该层存在权重数据。
先看一下控制台对h5py对象f的基本操作（需要的去查看相关数据结构定义）：
>>> f
<HDF5 file "resnet50_weights_tf_dim_ordering_tf_kernels_v2.h5" (mode r)>
>>> f.filename
'E:\\DeepLearning\\keras_test\\models\\resnet50_weights_tf_dim_ordering_tf_kernels_v2.h5'
>>> f.name
'/'
>>> f.attrs.keys() # f属性列表 #
<KeysViewHDF5 ['layer_names']>
>>> f.keys() #无顺序
<KeysViewHDF5 ['activation_1', 'activation_10', 'activation_11', 'activation_12',
...,'activation_8', 'activation_9', 'avg_pool', 'bn2a_branch1', 'bn2a_branch2a',
...,'res5c_branch2a', 'res5c_branch2b', 'res5c_branch2c', 'zeropadding2d_1']>
>>> f.attrs['layer_names'] #*** 有顺序, 和summary()对应 ****
array([b'input_1', b'zeropadding2d_1', b'conv1', b'bn_conv1',
b'activation_1', b'maxpooling2d_1', b'res2a_branch2a',
..., b'res2a_branch1', b'bn2a_branch2c', b'bn2a_branch1',
b'merge_1', b'activation_47', b'res5c_branch2b', b'bn5c_branch2b',
..., b'activation_48', b'res5c_branch2c', b'bn5c_branch2c',
b'merge_16', b'activation_49', b'avg_pool', b'flatten_1', b'fc1000'],
dtype='|S15')
>>> f['input_1']
<HDF5 group "/input_1" (0 members)>
>>> f['input_1'].attrs.keys() # 在keras中，每一个层都有‘weight_names’属性 #
<KeysViewHDF5 ['weight_names']>
>>> f['input_1'].attrs['weight_names'] # input层无权重 #
array([], dtype=float64)
>>> f['conv1']
<HDF5 group "/conv1" (2 members)>
>>> f['conv1'].attrs.keys()
<KeysViewHDF5 ['weight_names']>
>>> f['conv1'].attrs['weight_names'] # conv层有权重w、b #
array([b'conv1_W:0', b'conv1_b:0'], dtype='|S9')
从文件中读取具有权重数据的层的名字列表
# 获取后hdf5文本文件中层的名字，顺序对应
layer_names = load_attributes_from_hdf5_group(f, 'layer_names')
#上一句实现 layer_names = [n.decode('utf8') for n in f.attrs['layer_names']]
filtered_layer_names = []
for name in layer_names:
g = f[name]
weight_names = load_attributes_from_hdf5_group(g, 'weight_names')
#上一句实现 weight_names = [n.decode('utf8') for n in f[name].attrs['weight_names']]
#保留有权重层的名字
if weight_names:
filtered_layer_names.append(name)
layer_names = filtered_layer_names
# 验证模型中有有权重tensor的层 与 从h5中读取有权重层名字的 数量 保持一致。
if len(layer_names) != len(filtered_layers):
raise ValueError('You are trying to load a weight file '
'containing ' + str(len(layer_names)) +
' layers into a model with ' +
str(len(filtered_layers)) + ' layers.')
3、从hdf5文件中读取的权重数据、和keras模型层tf.Variable打包对应
先看一下权重数据、层的权重变量(Tensor tf.Variable)对象，以conv1为例
>>> f['conv1']['conv1_W:0'] # conv1_W:0 权重数据数据集
<HDF5 dataset "conv1_W:0": shape (7, 7, 3, 64), type "<f4">
>>> f['conv1']['conv1_W:0'].value # conv1_W:0 权重数据的值， 是一个标准的4d array
array([[[[ 2.82526277e-02, -1.18737184e-02, 1.51488732e-03, ...,
-1.07003953e-02, -5.27982824e-02, -1.36667420e-03],
[ 5.86827798e-03, 5.04415408e-02, 3.46324709e-03, ...,
1.01423981e-02, 1.39493728e-02, 1.67549420e-02],
[-2.44090753e-03, -4.86173332e-02, 2.69966386e-03, ...,
-3.44439060e-04, 3.48098315e-02, 6.28910400e-03]],
[[ 1.81872323e-02, -7.20698107e-03, 4.80302610e-03, ...,
…. ]]]])
>>> conv1_w = np.asarray(f['conv1']['conv1_W:0']) # 直接转换成numpy格式
>>> conv1_w.shape
(7, 7, 3, 64)
# 卷积层
>>> filtered_layers[0]
<keras.layers.convolutional.Conv2D object at 0x000001F7487C0E10>
>>> filtered_layers[0].name
'conv1'
>>> filtered_layers[0].input
<tf.Tensor 'conv1_pad/Pad:0' shape=(?, 230, 230, 3) dtype=float32>
#卷积层权重数据
>>> filtered_layers[0].weights
[<tf.Variable 'conv1/kernel:0' shape=(7, 7,
