如何精确地将L1正则化添加到张量流误差函数
deep-learning
neural-network
python
tensorflow
6
0

嘿,我是tensorflow的新手,即使经过很多努力也无法将L1正则化项添加到错误项中

x = tf.placeholder("float", [None, n_input])
# Weights and biases to hidden layer
ae_Wh1 = tf.Variable(tf.random_uniform((n_input, n_hidden1), -1.0 / math.sqrt(n_input), 1.0 / math.sqrt(n_input)))
ae_bh1 = tf.Variable(tf.zeros([n_hidden1]))
ae_h1 = tf.nn.tanh(tf.matmul(x,ae_Wh1) + ae_bh1)

ae_Wh2 = tf.Variable(tf.random_uniform((n_hidden1, n_hidden2), -1.0 / math.sqrt(n_hidden1), 1.0 / math.sqrt(n_hidden1)))
ae_bh2 = tf.Variable(tf.zeros([n_hidden2]))
ae_h2 = tf.nn.tanh(tf.matmul(ae_h1,ae_Wh2) + ae_bh2)

ae_Wh3 = tf.transpose(ae_Wh2)
ae_bh3 = tf.Variable(tf.zeros([n_hidden1]))
ae_h1_O = tf.nn.tanh(tf.matmul(ae_h2,ae_Wh3) + ae_bh3)

ae_Wh4 = tf.transpose(ae_Wh1)
ae_bh4 = tf.Variable(tf.zeros([n_input]))
ae_y_pred = tf.nn.tanh(tf.matmul(ae_h1_O,ae_Wh4) + ae_bh4)



ae_y_actual = tf.placeholder("float", [None,n_input])
meansq = tf.reduce_mean(tf.square(ae_y_actual - ae_y_pred))
train_step = tf.train.GradientDescentOptimizer(0.05).minimize(meansq)

之后,我使用

init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)

n_rounds = 100
batch_size = min(500, n_samp)
for i in range(100):
    sample = np.random.randint(n_samp, size=batch_size)
    batch_xs = input_data[sample][:]
    batch_ys = output_data_ae[sample][:]
    sess.run(train_step, feed_dict={x: batch_xs, ae_y_actual:batch_ys})

上面是4层自动编码器的代码 “ meansq”是我的平方损耗函数。如何在网络中为权重矩阵(张量)添加L1重新说明?

参考资料:
Stack Overflow
收藏
评论
共 1 个回答
高赞 时间 活跃

您可以使用TensorFlow的apply_regularizationl1_regularizer方法。

根据您的问题的示例:

import tensorflow as tf

total_loss = meansq #or other loss calcuation
l1_regularizer = tf.contrib.layers.l1_regularizer(
   scale=0.005, scope=None
)
weights = tf.trainable_variables() # all vars of your graph
regularization_penalty = tf.contrib.layers.apply_regularization(l1_regularizer, weights)

regularized_loss = total_loss + regularization_penalty # this loss needs to be minimized
train_step = tf.train.GradientDescentOptimizer(0.05).minimize(regularized_loss)

注意: weights是一个list ,其中每个条目都是一个tf.Variable

收藏
评论
新手导航
  • 社区规范
  • 提出问题
  • 进行投票
  • 个人资料
  • 优化问题
  • 回答问题

关于我们

常见问题

内容许可

联系我们

@2020 AskGo
京ICP备20001863号