Exercise 9        Name __________________        Score __/13

1. (2) Plot the following training set. Is it linearly separable? Yes
    

   x1	x0	y
   0	0	0
   0	1	1
   1	0	0
   1	1	1

    

2. (2) Using the algorithm of Figure 20.21, weights of W₀=0.3, W₁=-0.4, and alpha learning rate of 0.2 compute the activation output for input of x₁=0, x₀=1.    

   in = 0.3*1 + -0.4*0 = 0.3

   g(0.3) = 1
    

3. (2) Continuing with Question 2, compute the weights after the first epoch.
   
       Err = y - g(in) = 1 - 1 = 0
       W_i = W_i + (alpha * Err * x_i)
   
       W₀ = 0.3 + (0.2 * 0 * 1) = 0.3
       W₁ = -0.4 + (0.2 * 0 * 1) = -0.4
    
4. (3) Modify the algorithm inputs to attempt learning of the even parity operation for 3 inputs, that is:

   x2	x1	x0	y
   0	0	0	0
   0	0	1	1
   0	1	0	1
   0	1	1	0
   1	0	0	1
   1	0	1	0
   1	1	0	0
   1	1	1	1

   You'll need to run more epochs or training iterations.

   parity = [
             ([0,0,0], 0),
             ([0,0,1], 1),
             ([0,1,0], 1),
             ([0,1,1], 0),
             ([1,0,0], 1),
             ([1,0,1], 0),
             ([1,1,0], 0),
             ([1,1,1], 1)]
   
   NN = [-0.2, 0.4, -0.1]

   Was it able to learn, that is produce correct outputs? No
   Correct: [0, 1, 1, 0, 1, 0, 0, 1]
   Learned:[0, 0, 1, 1, 1, 1, 1, 1]
    

5. (4) Modify algorithm of Figure 20.25 for the 3 input even parity operation. You'll need to change the number of inputs and hidden nodes. Try with 2, 3 and 4 hidden nodes with 8000 epochs; what was the result?

2 hidden nodes

Training and test set:
[([0, 0, 0], [0]), ([0, 0, 1], [1]), ([0, 1, 0], [1]), ([0, 1, 1], [0]),
 ([1, 0, 0], [1]), ([1, 0, 1], [0]), ([1, 1, 0], [0]), ([1, 1, 1], [1])]

Correct:  [[0   ], [1    ], [1     ], [0     ], [1    ], [0     ],[0    ], [1     ]]
Learned: [[0.0], [0.85], [0.86], [0.46], [0.90], [0.47],[0.47], [0.43]]
 

3 hidden nodes

Training and test set:
[([0, 0, 0], [0]), ([0, 0, 1], [1]), ([0, 1, 0], [1]), ([0, 1, 1], [0]),
 ([1, 0, 0], [1]), ([1, 0, 1], [0]), ([1, 1, 0], [0]), ([1, 1, 1], [1])]

Correct: [[0   ], [1    ], [1     ], [0     ], [1    ], [0     ],  [0    ], [1     ]]
Learned:[[0.0], [0.73], [0.95], [0.07], [0.89], [-0.21], [0.00], [0.99]]

4 hidden nodes

Training and test set:
[([0, 0, 0], [0]), ([0, 0, 1], [1]), ([0, 1, 0], [1]), ([0, 1, 1], [0]),
 ([1, 0, 0], [1]), ([1, 0, 1], [0]), ([1, 1, 0], [0]), ([1, 1, 1], [1])]

Correct:  [[0   ], [1    ], [1     ], [0     ],  [1    ], [0   ], [0   ], [1   ]]
Learned: [[0.0], [0.97], [0.97], [-0.06], [0.98], [0.0], [0.0],[0.99]]

---

Data for 4 hidden nodes 3 input even parity

parity = [
([0,0,0], [0]), # Training examples
([0,0,1], [1]),
([0,1,0], [1]),
([0,1,1], [0]),
([1,0,0], [1]), # Training examples
([1,0,1], [0]),
([1,1,0], [0]),
([1,1,1], [1])]

NN = [[
[0.1, -0.2, 0.5], # weights from 3 input to 4 hidden
[-0.3, 0.4, -0.5],
[-0.3, 0.4, -0.5],
[0.1, -0.2, 0.5]],
[[0.5, -0.6, 0.2, -0.1]] # weights from 4 hidden to 1 output
]

print 'Learning results: ', BACK_PROP_LEARNING(parity, NN)
print 'Training and test set: ', parity
print 'Test results: ', BACK_PROP_TEST(parity, NN)