I am trying to assign a new value to a tensorflow variable in python.
import tensorflow as tf
import numpy as np
x = tf.Variable(0)
init = tf.initialize_all_variables()
sess = tf.InteractiveSession()
sess.run(init)
print(x.eval())
x.assign(1)
print(x.eval())
But the output I get is
0
0
So the value has not changed. What am I missing?
In TF1, the statement x.assign(1)
does not actually assign the value 1
to x
, but rather creates a tf.Operation
that you have to explicitly run to update the variable.* A call to Operation.run()
or Session.run()
can be used to run the operation:
assign_op = x.assign(1)
sess.run(assign_op) # or `assign_op.op.run()`
print(x.eval())
# ==> 1
(* In fact, it returns a tf.Tensor
, corresponding to the updated value of the variable, to make it easier to chain assignments.)
However, in TF2 x.assign(1)
will now assign the value eagerly:
x.assign(1)
print(x.numpy())
# ==> 1
You can also assign a new value to a tf.Variable
without adding an operation to the graph: tf.Variable.load(value, session)
. This function can also save you adding placeholders when assigning a value from outside the graph and it is useful in case the graph is finalized.
import tensorflow as tf
x = tf.Variable(0)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
print(sess.run(x)) # Prints 0.
x.load(1, sess)
print(sess.run(x)) # Prints 1.
Update: This is depricated in TF2 as eager execution is default and graphs are no longer exposed in the user-facing API.
First of all you can assign values to variables/constants just by feeding values into them the same way you do it with placeholders. So this is perfectly legal to do:
import tensorflow as tf
x = tf.Variable(0)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print sess.run(x, feed_dict={x: 3})
Regarding your confusion with the tf.assign() operator. In TF nothing is executed before you run it inside of the session. So you always have to do something like this: op_name = tf.some_function_that_create_op(params)
and then inside of the session you run sess.run(op_name)
. Using assign as an example you will do something like this:
import tensorflow as tf
x = tf.Variable(0)
y = tf.assign(x, 1)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print sess.run(x)
print sess.run(y)
print sess.run(x)
feed_dict
does not permanently assign that value to the variable, but only for that particular run call.
Also, it has to be noted that if you're using your_tensor.assign()
, then the tf.global_variables_initializer
need not be called explicitly since the assign operation does it for you in the background.
Example:
In [212]: w = tf.Variable(12)
In [213]: w_new = w.assign(34)
In [214]: with tf.Session() as sess:
...: sess.run(w_new)
...: print(w_new.eval())
# output
34
However, this will not initialize all variables, but it will only initialize the variable on which assign
was executed on.
I answered a similar question here. I looked in a lot of places that always created the same problem. Basically, I did not want to assign a value to the weights, but simply change the weights. The short version of the above answer is:
tf.keras.backend.set_value(tf_var, numpy_weights)
Here is the complete working example:
import numpy as np
import tensorflow as tf
w= tf.Variable(0, dtype=tf.float32) #good practice to set the type of the variable
cost = 10 + 5*w + w*w
train = tf.train.GradientDescentOptimizer(0.01).minimize(cost)
init = tf.global_variables_initializer()
session = tf.Session()
session.run(init)
print(session.run(w))
session.run(train)
print(session.run(w)) # runs one step of gradient descent
for i in range(10000):
session.run(train)
print(session.run(w))
Note the output will be:
0.0
-0.049999997
-2.499994
This means at the very start the Variable was 0, as defined, then after just one step of gradient decent the variable was -0.049999997, and after 10.000 more steps we are reaching -2.499994 (based on our cost function).
Note: You originally used the Interactive session. Interactive session is useful when multiple different sessions needed to be run in the same script. However, I used the non interactive session for simplicity.
Use Tensorflow eager execution mode which is latest.
import tensorflow as tf
tf.enable_eager_execution()
my_int_variable = tf.get_variable("my_int_variable", [1, 2, 3])
print(my_int_variable)
There is an easier approach:
x = tf.Variable(0)
x = x + 1
print x.eval()
tf.assign
, not addition.
Success story sharing
Variable
x
stored in memory before theassign
operation / mutable tensor was run overwritten or is a new tensor created that stores the updated value?assign()
overwrites the existing value.Variable
without creating any additional operations in the graph? It seems that each variable has an Assign operation created for it already, but callingmy_var.assign()
ortf.assign()
creates a new operation instead of using the existing one.assign
a tensor parameter such as math operation. And in this way create a counter which is updated every time the assign operation is evaluated:op = t.assign(tf.add(t, 1))
.