tensorflow_basics.md

Tensor Flow

Big idea: Express a numeric computation as a graph.

• Graph nodes are operations which have any number of inputs and outputs
• Graph edges are tensors which flow between nodes

Basic flow:

• Build a graph

  • Graph contains parameter specifications, model architecture, optimization process, …

• Initialize a session

• Fetch and feed data with Session.run

  • Compilation, optimization, etc. happens at this step

---

1. Hello World

import tensorflow as tf

hello = tf.constant('Hellow, TensorFlow!')

# Launch the default graph.
sess = tf.Session()
sess.run(hello)
# Close the Session when we're done.
sess.close()

---

2. Basic Operations I

with constants

a = tf.constant(2)
b = tf.constant(3)

with tf.Session() as sess:
    print "a=2, b=3"
    print "Addition with constants: %i" % sess.run(a+b)
    print "Multiplication with constants: %i" % sess.run(a*b)

a=2, b=3
Addition with constants: 5
Multiplication with constants: 6

with variables

a = tf.placeholder(tf.types.int16)
b = tf.placeholder(tf.types.int16)

add = tf.add(a, b)
mul = tf.mul(a, b)

with tf.Session() as sess:
    print "Addition with variables: %i" % sess.run(add, feed_dict={a:2, b:3})
    print "Multiplication with variables: %i" % sess.run(mul, feed_dict={a:2, b:3})    

Addition with variables: 5
Multiplication with variables: 6

---

3. Basic Operations II: NN feedforward

h_i = ReLU(Wx+b)

W, b:

• Variables are 0-ary stateful nodes which output their current value
• State is retained across multiple executions of a graph.
• (parameters, gradient stores, eligibility traces, …)

x:

• Placeholders are 0-ary nodes whose value is fed in at execution time.
• (inputs, variable learning rates, …)

Mathematical operations:

• MatMul: Multiply two matrix values.
• Add: Add elementwise (with broadcasting).
• ReLU: Activate with elementwise rectified linear function.

b = tf.Variable(tf.zeros(100,))  
W = tf.Variable(tf.random_normal((784, 100), -1, 1))  
x = tf.placeholder(tf.float32, (None, 784))  
h_i = tf.nn.relu(tf.matmul(x, W) + b)

So far we have defined a graph.
We can deploy this graph with a session: a binding to a particular execution context (e.g. CPU, GPU)

"sess.run(fetches, feeds)"

• Fetches: List of graph nodes. Return the outputs of these nodes.
• Feeds: Dictionary mapping from graph nodes to concrete values. Specifies the value of each graph node given in the dictionary.

sess = tf.Session()  
sess.run(tf.initialize_all_variables())  
sess.run(h_i, {x: np.random.random(64, 784)})

---

Summary:

Basic flow:

• Build a graph

  • Graph contains parameter specifications, model architecture, optimization process, …

• Initialize a session

• Fetch and feed data with Session.run

  • Compilation, optimization, etc. happens at this step — you probably won’t notice

---