Keras常用接口与自定义网络

机器学习 / Keras
字数统计: 1.5k字   |   阅读时长≈ 8分钟

Keras != tf.keras

Keras建立在其他接口之上的高层接口

TF2中实际不需要Keras，TF2中有对Keras的实现

  • datasets
  • layers
  • losses
  • metrics
  • optimizers

KerasAPI

Keras.Metrics

在DL中经常要计算loss和accuracy，这并不是指某一次迭代或某batch的loss和accuracy，至少是一个epoch的平均。我们要新建list，每1个batch向其中加入一个loss，经过数次后到达观测的节点，计算出这个list的avg，可以比较好的反应前后两次观测的变化情况，这个功能就是一个测量表的功能，也是Metrics的功能。

  • Metrics 新建

两个meter即为两个测量尺

1
2
acc_meter = metrics.Accuracy() #新建准确度meter
loss_meter = metrics.Mean()
  • update_state 添加数据
1
2
loss_meter.update_state(loss) 
acc_meter.update_state(y,pred) #添加的并不是和loss一样的实时数据
  • result().numpy() 需要使用时得到结果
1
2
print(step,'loss:'loss_meter.result().numpy())
print(step,'Evaluate Acc:',total_correct/total,acc_meter.result().numpy())
  • reset_states 清零
1
2
3
4
5
6
if step % 100 == 0:
	print(step,'loss:'loss_meter.result().numpy())
	loss_meter.reset_states()
if step % 500 == 0:
	total,total_correct = 0.,0
	acc_meter.reset_states()

Compile&Fit

快捷训练方法

  • Compile装载的功能，指定loss、优化器、评估指标的选择
1
2
network.compile(optimizer = optimizer.Adam(lr=0.01),
	loss=tf.losses.CategoricalCrossentropy(from_logits=True))
  • fit 完成标准创建流程
1
network.fit(db,epochs=10)
  • evaluate 进行测试
1
2
3
4
5
6
network.compile(optimizer = optimizer.Adam(lr=0.01),
	loss=tf.losses.CategoricalCrossentropy(from_logits=True),
	metrics['accuracy'])
network.fit(db,epochs=10,validation_data=ds_val,validation_freq=2)
#ds_val 测试集 validation_freq 每？个epoch进行一次validation 可以停止eg:if acc>0.99 save break
network.evaluate(ds_val) 在training结束之后进行

交叉验证

数据集分为训练集，验证集，测试集。一般测试集不在我们手中，所以我们要把训练集分为两份，一份为训练集train，另一份为验证集validation。交叉验证，每个epoch选择不同的验证集进行验证。

1
2
3
4
5
6
7
8
9
10
11
for epoch in range(500):
    idx = tf.range(600000)
    idx = tf.random.shuffle(idx)
    x_train,y_train = tf.gather(x,idx[:500000]),tf.gather(y,idx[:500000])
    x_val,y_val = tf.gather(x,idx[-100000:]),tf.gather(y,idx[-100000:])
    
    db_train = tf.data.Datasets.from_tensor_slices((x_train,y_train))
    db_train = db_train.map(preprocess).shuffle(500000).batch(batchsz)
    
    db_val = tf.data.Datasets.from_tensor_slices((x_val,y_val))
    db_val = db_val.map(preprocess).shuffle(500000).batch(batchsz)

也可以是直接使用fit

1
network.fit(db_train_val,epoch=6,validation_split=0.1,validation_freq=2)

自定义网络

keras.Sequential

容器，可以把已有层和自己的层串联在一起，可以方便组织参数

1
2
3
4
5
6
7
network = Sequential([layers.Dense(256,activation='relu'),
					 layers.Dense(128,activation='relu'),
					 layers.Dense(64,activation='relu'),
					 layers.Dense(32,activation='relu'),
					 layers.Dense(10)])
network.build(input_shape=(None,29*28))
network.summary()

  • model.trainable_variables 所有的训练参数

  • model.call() 调用Sequential时内部调用 使用model(x) ->会调用

    1
    model.__call__(x) #这其中有call方法

Layer/Model

  • Inherit from keras.layers.Layer keras.Model

  • __init__

  • call
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
class MyDense(layers.Layer): #继承自layers.Layer
    def __init__(self, inp_dim,outp_dim):
        super(MyDense,self).__init__()

        self.kernel = self.add_variable('w',[inp_dim,outp_dim])
        self.bias = self.add_variable('b',[outp_dim])

    def call(self,inputs,training=None):
        out = inputs @ self.kernel + self.bias
        return out
class MyModel(keras.Model):
    def __init__(self):
        super(MyModel,self).__init__()
        self.fc1 = MyDense(28 * 28,256)
        self.fc2 = MyDense(256, 128)
        self.fc3 = MyDense(128, 64)
        self.fc4 = MyDense(64, 32)
        self.fc5 = MyDense(32, 10)
    def call(self,inputs,training=None):
        x = self.fc1(inputs)
        x = tf.nn.relu(x)
        x = self.fc2(x)
        x = tf.nn.relu(x)
        x = self.fc3(x)      #在任意层之间可以加任意的操作，这是使用自定义层的优势，比如x=x-1
        x = tf.nn.relu(x)
        x = self.fc4(x)
        x = tf.nn.relu(x)
        x = self.fc5(x)
        return x
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
import tensorflow as tf
from tensorflow.keras import datasets, layers, optimizers, Sequential, metrics
from tensorflow import keras

def preprocess(x, y):
    """
    x is a simple image, not a batch
    """
    x = tf.cast(x, dtype=tf.float32) / 255.
    x = tf.reshape(x, [28 * 28])
    y = tf.cast(y, dtype=tf.int32)
    y = tf.one_hot(y, depth=10)
    return x, y

class MyDense(layers.Layer):
    def __init__(self, inp_dim, outp_dim):
        super(MyDense, self).__init__()
        self.kernel = self.add_weight('w', [inp_dim, outp_dim])
        self.bias = self.add_weight('b', [outp_dim])
    def call(self, inputs, training=None):
        out = inputs @ self.kernel + self.bias
        return out

class MyModel(keras.Model):
    def __init__(self):
        super(MyModel, self).__init__()

        self.fc1 = MyDense(28 * 28, 256)
        self.fc2 = MyDense(256, 128)
        self.fc3 = MyDense(128, 64)
        self.fc4 = MyDense(64, 32)
        self.fc5 = MyDense(32, 10)
    def call(self, inputs, training=None):
        x = self.fc1(inputs)
        x = tf.nn.relu(x)
        x = self.fc2(x)
        x = tf.nn.relu(x)
        x = self.fc3(x)
        x = tf.nn.relu(x)
        x = self.fc4(x)
        x = tf.nn.relu(x)
        x = self.fc5(x)
        return x

def run():
    batchsz = 128
    (x, y), (x_val, y_val) = datasets.mnist.load_data()
    db = tf.data.Dataset.from_tensor_slices((x, y))
    db = db.map(preprocess).shuffle(60000).batch(batchsz)
    ds_val = tf.data.Dataset.from_tensor_slices((x_val, y_val))
    ds_val = ds_val.map(preprocess).batch(batchsz)

    network = MyModel()

    network.compile(optimizer=optimizers.Adam(lr=0.01),
                    loss=tf.losses.CategoricalCrossentropy(from_logits=True),
                    metrics=['accuracy'])
    
    network.build(input_shape=(None, 28 * 28))
    network.summary()
    
    network.fit(db, epochs=5, validation_data=ds_val,
                validation_freq=2)

    network.evaluate(ds_val)

    sample = next(iter(ds_val))
    x = sample[0]
    y = sample[1]  # one-hot
    pred = network.predict(x)  # [b, 10]
    # convert back to number
    y = tf.argmax(y, axis=1)
    pred = tf.argmax(pred, axis=1)

    print(pred)
    print(y)

if __name__ == '__main__':
    run()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Model: "my_model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
my_dense (MyDense)           multiple                  200960    
_________________________________________________________________
my_dense_1 (MyDense)         multiple                  32896     
_________________________________________________________________
my_dense_2 (MyDense)         multiple                  8256      
_________________________________________________________________
my_dense_3 (MyDense)         multiple                  2080      
_________________________________________________________________
my_dense_4 (MyDense)         multiple                  330       
=================================================================
Total params: 244,522
Trainable params: 244,522
Non-trainable params: 0
_________________________________________________________________
Train for 469 steps, validate for 79 steps
Epoch 1/5
469/469 [==============================] - 3s 7ms/step - loss: 0.2781 - accuracy: 0.9147
Epoch 2/5
469/469 [==============================] - 4s 8ms/step - loss: 0.1352 - accuracy: 0.9616 - val_loss: 0.1384 - val_accuracy: 0.9618
Epoch 3/5
469/469 [==============================] - 3s 6ms/step - loss: 0.1055 - accuracy: 0.9709
Epoch 4/5
469/469 [==============================] - 4s 8ms/step - loss: 0.0977 - accuracy: 0.9736 - val_loss: 0.1329 - val_accuracy: 0.9660
Epoch 5/5
469/469 [==============================] - 3s 7ms/step - loss: 0.0898 - accuracy: 0.9750
79/79 [==============================] - 1s 7ms/step - loss: 0.1225 - accuracy: 0.9680
tf.Tensor(
[7 2 1 0 4 1 4 9 5 9 0 6 9 0 1 5 9 7 3 4 9 6 6 5 4 0 7 4 0 1 3 1 3 0 7 2 7
 1 2 1 1 7 4 2 3 5 1 2 4 4 6 3 5 5 6 0 4 1 9 5 7 8 9 3 7 4 6 4 3 0 7 0 2 9
 1 7 3 2 9 7 7 6 2 7 8 4 7 3 6 1 3 6 9 3 1 4 1 7 6 9 6 0 5 4 9 9 2 1 9 4 8
 7 3 9 7 4 4 4 9 2 5 4 7 6 7 9 0 5], shape=(128,), dtype=int64)
tf.Tensor(
[7 2 1 0 4 1 4 9 5 9 0 6 9 0 1 5 9 7 3 4 9 6 6 5 4 0 7 4 0 1 3 1 3 4 7 2 7
 1 2 1 1 7 4 2 3 5 1 2 4 4 6 3 5 5 6 0 4 1 9 5 7 8 9 3 7 4 6 4 3 0 7 0 2 9
 1 7 3 2 9 7 7 6 2 7 8 4 7 3 6 1 3 6 9 3 1 4 1 7 6 9 6 0 5 4 9 9 2 1 9 4 8
 7 3 9 7 4 4 4 9 2 5 4 7 6 7 9 0 5], shape=(128,), dtype=int64)
打赏
  • 版权声明： 本博客所有文章除特别声明外，均采用 Apache License 2.0 许可协议。转载请注明出处！

请我喝杯咖啡吧~

支付宝
微信