[back to assignments]

# Neural Networks

## CS 440 - October 7th, 2009

### Goals

The purpose of this project is to design several neural networks in Matlab. The main part of this assignment is to design and implement a feed-forward neural network that can be used to recognize hand-written digits.

## Part 1

### Output

1:   -0.9101   1.6195   -1.0202   1.0207

2:   0.9789   -0.9365   -1.0039   -1.2644

3:   0.8274   -0.8986

4:   -0.7389   -0.9737

### Interpretation

Test Vector 1:
-0.9101 : [-1; 2] is close to [-1; 0], meaning the input is close to -1
1.6195 : [2; 2] returns 1 because both inputs with 2 in the first element return 1
-1.0202 : inputs trained with -1 return -1
1.0207 : [2; 5] outputs 1, as learned via training

Test Vector 2:
0.9789 : NN was trained to output 1 on [2; 0]
-0.9365 : [0; 1] is most similar to [-1, 0], giving an input close to -1
-1.0039 : training vectors with a negative element in the first position return -1
-1.2644 : training vectors with a negative element in the first position return -1

Test Vector 3:
0.8274 : [1; 5] is similar to [2; 5], and the NN was trained to return 1 on [2; 5]
-0.8986 : the network was trained to return -1 on [0; 5]

Test Vector 4:
-0.7389 : The network was not trained on [1; 0]; not sure about output
-0.9737 : [0;0] is similar to [-1;0], giving a return value of -1

## Part 2

### Recognition: R

 R with noise 0 was predicted as J R with noise 2.000000e-001 was predicted as Q R with noise 4.000000e-001 was predicted as Q R with noise 6.000000e-001 was predicted as Q R with noise 8.000000e-001 was predicted as Q

### Recognition: G

 G with noise 0 was predicted as S G with noise 2.000000e-001 was predicted as S G with noise 4.000000e-001 was predicted as S G with noise 6.000000e-001 was predicted as B G with noise 8.000000e-001 was predicted as V

### Recognition: C

 C with noise 0 was predicted as Q C with noise 2.000000e-001 was predicted as Q C with noise 4.000000e-001 was predicted as Q C with noise 6.000000e-001 was predicted as Q C with noise 8.000000e-001 was predicted as Q

### Recognition: A

 A with noise 0 was predicted as H A with noise 2.000000e-001 was predicted as H A with noise 4.000000e-001 was predicted as H A with noise 6.000000e-001 was predicted as Q A with noise 8.000000e-001 was predicted as Q 8 with noise 0 was predicted as Q

## Part 3

I defined my Neural Network to have 5 layers. The first layer consists of 250 nodes, and the following layers have 128, 64, 32, and 10 nodes, respectively. My Neural Network uses tansig as a threshold on all 5 layers. My neural network was able to obtain 89.1% accuracy with the entire feature vector set.

The confusion matrix is:
```    83     0     0     0     0     1     2     0     0     0
0   121     0     0     0     0     1     0     0     0
0     0   103     2     0     0     0     3     4     1
0     0     2   100     0     7     2     1     3     0
1     2     0     0    86     1     3     1     0    14
2     0     1     2     0    82     0     1     3     1
2     1     2     0     2     2    77     0     1     0
0     0     0     2     1     0     4    87     4     1
0     0     3     4     3     1     0     2    69     4
0     0     0     0     3     1     0     3     2    83
```

I modified the feature vector by taking the average of adjacent features and using this averaged set as my new feature vector when training and testing the neural network. I was able to obtain 90.2% accuracy using this approach. The confusion matrix is:
```    82     0     0     1     0     1     1     0     0     1
0   122     0     0     0     0     0     0     0     0
1     0   100     2     0     1     0     3     2     4
1     0     0    96     0     7     4     3     3     1
0     2     2     0    96     0     2     0     0     6
2     0     0     3     1    83     1     0     2     0
3     0     1     0     0     1    80     1     0     1
0     1     4     0     1     0     0    92     0     1
0     0     2     3     3     5     1     1    66     5
0     1     0     0     4     0     0     0     2    85```

### Code

Neural Net Project