当前位置 主页 > 网站技术 > 代码类 >

    有关Tensorflow梯度下降常用的优化方法分享

    栏目:代码类 时间:2020-02-04 15:08

    1.tf.train.exponential_decay() 指数衰减学习率:

    #tf.train.exponential_decay(learning_rate, global_steps, decay_steps, decay_rate, staircase=True/False):
    #指数衰减学习率
    #learning_rate-学习率
    #global_steps-训练轮数
    #decay_steps-完整的使用一遍训练数据所需的迭代轮数;=总训练样本数/batch
    #decay_rate-衰减速度
    #staircase-衰减方式;=True,那就表明每decay_steps次计算学习速率变化,更新原始学习速率;=alse,那就是每一步都更新学习速率。learning_rate = tf.train.exponential_decay(
    initial_learning_rate = 0.001
    global_step = tf.Variable(0, trainable=False)
    decay_steps = 100
    decay_rate = 0.95
    total_loss = slim.losses.get_total_loss()
    learning_rate = tf.train.exponential_decay(initial_learning_rate, global_step, decay_steps, decay_rate, True, name='learning_rate')
    optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(total_loss, global_step)

    2.tf.train.ExponentialMovingAverage(decay, steps) 滑动平均更新参数:

    initial_learning_rate = 0.001
    global_step = tf.Variable(0, trainable=False)
    decay_steps = 100
    decay_rate = 0.95
    total_loss = slim.losses.get_total_loss()
    learning_rate = tf.train.exponential_decay(initial_learning_rate, global_step, decay_steps, decay_rate, True, name='learning_rate')
    optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(total_loss, global_step)
    ema = tf.train.ExponentialMovingAverage(decay=0.9999)
    #tf.trainable_variables--返回的是需要训练的变量列表
    averages_op = ema.apply(tf.trainable_variables())
    with tf.control_dependencies([optimizer]):
       train_op = tf.group(averages_op)

    以上这篇有关Tensorflow梯度下降常用的优化方法分享就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持IIS7站长之家。