Make your voice heard.

Take the 2019 Developer Survey now

Home

PUBLIC

Stack Overflow

Tags Tensorflow multiple session.run( ) in same iteration Ask Question

Users
Following are 2 code patterns for the training part of Tensorflow NN.
Jobs
I find it logical to use the model 1. But I am seeing model 2 in multiple
2 places frequently. I feel Model 2 is wrong. Wont model run the graph in
Teams session twice for every iteration for the same data? Is there something I
Q&A for work am missing and people do it for any other reasons?

Learn More Model 1

for epoch in range(epochs):

for iteration in range(num_tr_iter):
_, loss, accuracy = sess.run([optimizer, loss, accuracy], feed_dict)

Model 2

for epoch in range(epochs):

for iteration in range(num_tr_iter):
sess.run(optimizer, feed_dict)
loss, accuracy = sess.run([loss, accuracy],feed_dict)

EDIT : Am expanding the question for more clarity

If the below sess.run() execute optimizer node, it will execute all its
dependent node. It will run the underlying convnet and loss function as
well.

sess.run(optimizer, feed_dict)

Next if the below sess.run() execute the loss node, why doesnt it execute
the convnet with its current weights. I am not inferring that it will run the
optimization again. Even to arrive at the current loss, wouldnt tensorflow
execute the convnet and calculate the loss?

loss, accuracy = sess.run([loss, accuracy],feed_dict)

python tensorflow neural-network

edited Oct 30 '18 at 16:40

asked Oct 30 '18 at 6:06

solver149
196 1 8

2 Answers

Session.Run will only run calculate the graph element(s) given to in the first
argument.
1 So in Model 2, sess.run(optimizer, feed_dict) just applies the weight
updates for the model, and loss, accuracy = sess.run([loss,
accuracy],feed_dict) just calculates the loss and accuracy on the model
after weights have been updated. The whole computation isn't being run
twice since these operations are independent of each other.

Note that the Session.Run docs mention that:

By using our site, you acknowledge

Order that
in which... you have read
operations and understand
are evaluated ourthe
inside Cookie Policy
call is , Privacy Policy, and our Terms of Service.
undefined.
 So Model 1 might not even give an accurate loss and accuracy, since it's
unclear if it was measured before or after the weight update. (In practice
this might be fine, though)

answered Oct 30 '18 at 6:21

Jacob Soderlund
111 8

In Model 1, optimizer is the first fetch for sess.run( ). In that case, are we good
with model 1 as well? – solver149 Oct 30 '18 at 6:39

Based on the documentation, I don't think we can assume that optimizer is

evaluated first just because it's first in the list. Like I said above, though, in
practice it might turn out that it is - I don't know enough about the internals of
Session.Run to really say. – Jacob Soderlund Oct 31 '18 at 0:35

In model 2, the first line of code runs the optimizer on the model, and the
second line calculates the loss and accuracy. It is not running the graph
0 twice.

Model 1 and Model 2 are essentially the same, but points go to model 1 for
brevity.

answered Oct 30 '18 at 6:17

Abhishek Verma
1

By using our site, you acknowledge that you have read and understand our Cookie Policy, Privacy Policy, and our Terms of Service.