You are on page 1of 9

Yu Hen Hu

5/9/16

Last (family) name: _________________________
First (given) name: _________________________
Student I.D. #: _____________________________
Department of Electrical and Computer Engineering
University of Wisconsin - Madison
ECE/CS/ME 539 Introduction to Artificial Neural Network and Fuzzy Systems

Take Home Final Examination
(From noon, Monday 5/2/2016 till noon, Monday 5/9/2016)





This is a take home final examination. You are to submit your answer
electronically to the Moodle assignment box before the deadline. If you
prefer hand-writing rather than typing, you should keep the writing
neat and high-light the answer and scan the answer sheet for
electronic submission.
You CANNOT discuss either the questions, or the answers with any one
except the instructor.
Many of the problems require programming. You are required to attach
copies of your code as part of the submission so that the grader can
run the code to verify the answer.
You cannot copy the code elsewhere EXCEPT those post on the course
website.
ABSOLUTELY NO EXTENSION REQUEST WILL BE GRANTED. Be on time.
Any academic mis-conduct will be pursued to the full extend
according to University rule. You must sign below and scan this
page as part of submission to receive credits for the final examination.

I, _____________________ (print your name) promise that I will not commit
academic plagiarism. I understand such an offense will result in failure of
this course for both the person who copy other’s answer and those who
let their answer copied by others.

IMPORTANT!

Question Answering Period

During his regular office hours, 1-2PM, Tuesday, 2-3PM, Wednesday,
11-noon, Thursday, or by appointment, Prof. Hu will be available
answering questions related to this final examination. Questions may also
be submitted via email.

1 of 9

pts 100 Points .Yu Hen Hu 5/9/16 Proble m 1 15 2 10 3 15 4 30 5 15 6 15 Total 2 of 9 Max.

Actually. (iii) MLP. idx2=find(y==+1).idx1). We define a 1  N vector y = sgn(wTX  c[1 1 … 1]1N) as the binary label vector where “sgn” is a sign function such that sgn(x) = + 1 if x > 0 and = 1 if x < 0.Yu Hen Hu 5/9/16 1. the answer should provide w.:). …. D1=x(:. x2. Note that this is a binary classification problem using a linear classifier. These include: (i) Perceptron learning. Suppose there is a hyperplane H: {x. Find a hyperplane in the form of H: {x. g(x) = wTx  c = 0} such that yn = sgn(g(xn)) for 1  n  N where y = [y1. y. xN]mN be a matrix consisting of N m-dimensional feature vectors {xn. N1=length(idx1). Now. y=s16p1dat_1(end. x=s16p1dat_1(1:end-1. % 1 x N yb1=yb(y==-1).N]=size(x). idx1=find(y==-1). The matlab code that compute the Cmat matrix for the given X. The projection of each column of X along the direction of w. g(x) = wTx  c = 0} that separates these feature vectors into two subgroups. the training data and testing data are the same set of X and y.:). You may use any one or more methods you have learned in this class. Its value may be adjusted to maximize the probability of classification. a realization of X and corresponding y vector are stored in the datafile s16p1dat_1. (iv) SVM. % label for +1 class if mean(yb1) < mean(yb2). If you obtain an adquate choice of w and c. the Cmat should be a diagnal matrix (100% correct classification). y2. yb=w'*x. [m. In this particular example. with one labeled with 1 and the other labeled with +1. The first m rows belongs to the X matrix while the last row is the y vector. D2=x(:. namely.txt. % if -1 class is projected to negative values sides=0. …. clear s16p1dat_1. (ii) linear discriminant analysis. w. or (v) maximum liklihood classifier using uni-variate Normal distribution. Hence. The data set given is linearly separable using a particular value of w and c as shown in the figure below.idx2). % label for -1 class yb2=yb(y==+1).txt. and c are as follows. The matlab code to load the data and set up parameters are: load s16p1dat_1. (15 points) linear Classification Let X = [x1. % -1 side is negative projection value 3 of 9 . here c is computed to be in the middle of the gap. yN]. c and corresponding confusion matrix Cmat. one having values greater than c and the other smaller than c. wTX then may be divided into two groups. N2=length(idx2). 1  n  N}.

end if gp < 0.mathworks. % N x 2. (0. ye=double([[yest' < 0] [yest' > 0]]).0286 0. Plot the clustering results using diferent colors for different clusters. ye=double([[yest' > 0] [yest' < 0]]). % +1 side is negative projection value gp=min(yb1)-max(yb2).txt using a linear array of indices of the neurons.5*(min(yb2)+max(yb1)).0082 0. % N x 2 elseif sides==1.txt using 8 clusters. Recall that for polynomial Kernel.0243 0. (15 points) clustering. -1) to (1.0124 0. SV 1 -23 12 2 -2 3 3 4 -4 4 16 -8 5 16 6 6 -3 8 7 -1 4 8 -5 9 9 1 -15 28 -5 -1 -6 6 11 11 8 4 d 1 -1 -1 1 -1 -1 1 -1 1  0. 3.Yu Hen Hu 5/9/16 gp=min(yb2)-max(yb1). ground truth if sides==0.txt and s16p3b. 0). Compute the output of this SVM model when the input feature vector is (13. xi) = (1 + xTxi)2 where xi is a support vector. 8.0615 0. Reporting results with total number of neurons N = 10. Classification = ' num2str(round(100*sum(diag(Cmat))/N)) '%']).m (c) (5 points) Apply the Kmeans algorithtm for s16p3b.5*(min(yb1)+max(yb2)). and 20 respectively. kmeans algorithm.0016 0.com/matlabcentral/fileexchange/52905-dbscan-clusteringalgorithm) to cluster s16p3a. SOM Download the data files s16p3a. disp('*** projection not separated linearly! ***'). else % if +1 class is projected to negative values sides=1.0027 0. y) = (1 + xTy)2. disp(['Cmat = ']) disp(Cmat) disp(['Prob. (a) (5 points) Apply DBSCAN clsutering algorithtm (http://www. end yest=sign(w'*x-c*ones(1. Plot the clustering results as illustrated in somdemo.txt. 1) encoding ygrnd=double([[y' > 0] [y' < 0]]). % N x 2 end Cmat=ygrnd'*ye. c=0. 2. (b) (5 points) Apply the SOM algorithm on s16p3a. 6). Discuss which values of Eps and Minpts should be used to yield two clusters.0085 Suppose that the polynomial kernel is used. Plot the clustering results as demonstrated in clusterdemo. (10 points) A hypothetical SVM model has the following values of support vectors (SV) and corresponding target values d. K(x.txt. c=0.m 4 of 9 .N)+eps). % 1 x N. estimated % converting output from scalar (+1.8703 0. Hence K(x.

100) Refer to the left figure above.95) (0. denoted by (x.Yu Hen Hu 5/9/16 4. we ignore the width of the truck in this problem.0) (0. For consistency. (30 points) Fuzzy Logic Controller Design x (50. and the distance between the center of the rear end of this truck and (x. y) is L. The objective is to back-up this truck to the loading duck (marked with a triangle at coordinate (50. 30o) measured from the truck’s heading direction. y) is defined as the center point between its rear wheels. Three dynamic equations describe the truck’s movement: x(n  1)  x (n)  t  v  cos (n)  cos( (n)   / 2) y (n  1)  y (n)  t  v  cos  (n)  sin( (n)   / 2) v  sin  (n)  (n  1)   (n)  t  L x(n+1) and y(n+1) are trucks new position t seconds later than x(n) and y(n) respectively. The loading duck and the parking lot are surrounded by walls enclosing a 100 meter square area marked with thick lines. Here angles are represented in the unit of radians rather than degrees. The truck’s coordinate. the truck’s heading direction is 90o. 0)) such that the center line of the truck is prependicular to the loading duck wall so that the truck faces south at that position. all positive angles are measured in the counter-clock wise direction. For simplicity. (5. The distance between the center of its front wheels to (x. 40). y) is . Note that due to the 5 of 9 . respectively. or the controller you are to design in this problem. all measured in meters. Otherwise. The truck can not hit the wall during the back-up operation. The driver.40) Parking Lot  (5. the truck will be driven backward at a constant speed v. 0) (100. a truck (shown on the right figure) is initially parked in a parking lot (shaded region) whose lower-left and upper-right coordinates are. is allowed to streer the front wheel by an angle  within the range of (30o. For example in the right figure above.0) y (95. it is consider a failure.  (0    360o) measures the angle between the positive y axis and the heading of the truck. 95) and (95. When the back-up operation begins.

(n) must also be fuzzified. (a) (5 points) Let v = 4 m/s. Write a program to compute the trucks positions and headings for n = 1. the two inputs to the fuzzy controller.360o]. 10 o. compute the fuzified membership vectors of each of them. and initial heading (1) = 80o. …. 30o]. and 100  y  0. x(n) and (n) need to be fuzzified. (c) (10 points) Let v = 4 m/s. with a support containing 61 evenly separated points in the range (including both ends). let us generate triangular fuzzy sets using the Matlab program fsgen. L = 2 m and  = 0. Furthermore. Suppose that the initial truck location (x(1).m to develop a fuzzy logic controller to back-up the truck to the loading duck. and (n) = 23o. 11. (ii) 2  yf  . n = 1.5m. as stated above. If the truck driver maneuvers the truck with the following steering angles (in degrees) {(n). yf) and the heading f satisfy the following conditions: (i) |xf  50|  2 meter. Specifically. (iii) 7 fuzzy sets for output variable (n)  [30o. and the center of area defuzzification method as implemented in defuz. a successful example is given below: 6 of 9 . 2. and (iii) |f  180o |  5o. 13 o. (ii) 7 fuzzy sets for input variable (n)  [0. and  = 0. L = 1. with a support containing 91evenly separated points in the range (including both ends). 2 o. Answer: n 1 x 50 y -75  (n ) 80 2 3 4 5 6 7 8 9 10 11 (b) (5 points) In order to develop a fuzzy logic controller to control the steering angle at each time step.m that can be download from the Matlab programs in the course web page. Hence we also require 0  x  100. t = 2 s. generate (i) 5 fuzzy sets for input variable x(n)  [0. A back-up operation is considered successful if the trucks final coordinate (xf. plot the trucks (x. Use the fuzzy sets defined in part (b). the output of the fuzzy logic controller. suppose that x(n) = 65 m.Yu Hen Hu 5/9/16 dependency between x(n) and y(n). Also. a rule-base defined in the file ruletrk. 10 }= {9 o. Give (n) in terms of degrees. Also. y) coordinates for n = 1 to 11. 100]. 3 o. Tabulate the result. …. only two independent state variables x(n) and (n) will be sufficient to determine the truck’s trajectory and its heading.m (downloadable from the web). y(1)) = (50. in no time during the backup operation the truck may run over the wall. with a support containing 25 evenly separated points in that range (including both ends). 13 o}. Let [x(0) y(0) (0)] = [70 70 90o]. (n) = 130o.5m. 7 o. 13 o.5s.5m. 75). 3 o. 13 o. For the time being. t = 0. Plot these fuzzy sets. 2. Moreover.

txt and the feature vectors of the testing data set ctest.txt can be down-loaded from the course home-page.Yu Hen Hu 5/9/16 trajectory of truck backing to dock 0 x final = 49. Each misclassified sample causes 0. try to improve the performance of this fuzzy logic controller. 5.4777 degree. (d) (10 points) Use the result you obtained in part (c.1546 meter y final = -0. (ii) (4 points) Download a file trkpos. Apply the fuzzy logic controller to each of these initial conditions. ii) as a baseline performance. (15 points) Pattern Classification Use multi-layer perceptron neural network to perform pattern classification. The class labels of the testing set feature vectors are withheld. submit your new definitions of fuzzy sets and the modified rule set you used. The remaining points will be awarded roughly according to the percentage improvements. and a figure like above to show your program is working. You will receive 5 points of this part if any improvement is shown. You score for this problem will be judged in part by your approach to solve the problem. You may modify either the definition of the fuzzy set (but the discrete support for each variable can not be changed). The training data set ctrain. Also. and report how much improvement of performance due to these changes. and the rule base. Report how many trials are successful.txt from the course web site. Answer: 7 of 9 .62813 meter phi final = 176. (a) (5 points) At least two different network structures you have experimented. (b) (10 points) Fill out the table below with testing results for each testing vector. -20 -40 -60 -80 -100 0 20 40 60 80 100 (i) (6 points) Submit a printed copy of the source code of your program. Full credit (10 points) will be given if the overall success rate  98% after improvement. and the remaining points will be graded based on the relative classification rate on the testing results with respect to the best results submitted. Discuss briefly the changes you have made. and your basis for selecting the network for testing. The instructor may choose to verify the results as you claimed in this part when deemed necessary.5 point deduction up to total 10 points deduction on this part. Each row of this file specifies the initial position and heading (in degrees) of this truck of a trial.

071 0.609 0.204 0.287 0.470 0.801 1.444 0.672 -0.987 0.538 0.632 1.241 0.666 0.794 0.834 0.126 0.216 0.Yu Hen Hu 5/9/16 Testing vector 0.386 0.982 Label 0.503 0.116 0.586 1. (15 points) Mixture of Experts Assume a 2-class pattern classification problem with class labels {0.814 0.250 0.038 -0.908 0.550 -0.281 0.283 0.904 0.364 0.383 0.461 0.002 0.053 0.739 0.321 0.030 6.319 0.847 1.167 0.158 0.188 0.358 0.380 1.428 -1.243 0.571 0.850 1.418 -0.350 0.159 0.017 0.208 1. 1 ≤ k ≤ 10}.537 0.629 -0.652 -0.020 -0.531 0.682 0.896 -0.599 -0.084 0.379 0.618 -0. 1} and 10 feature vectors {x(k). The classification results of three expert classifiers as well as the correct class labels are shown in the table below: 8 of 9 .183 0.084 -0.044 0.174 0.243 -0.680 -0.803 0.517 -0.811 1.000 -0.

wB. yB(k).Yu Hen Hu 5/9/16 Feature no. as well as h being a constant threshold. 1 2 3 4 5 6 7 8 9 10 Labels 0 0 0 0 0 1 1 1 1 1 classifier A output 0 0 1 1 0 1 1 1 1 0 classifier B output 0 0 1 0 0 1 1 0 1 1 classifier C output 0 1 1 1 1 0 0 0 1 1 (a) (10 points) Denote the kth outputs of classifiers to be yA(k). Applying the ensemble classifier to the given training samples Feature no. Fill-in the table below so as to minimize PMiss-classification. wC} be a set of weights such that wA + wB + wC = 1. and C form a new feature vector f(k) = [yA(k). Let {wA. (b) (5 points) Note that the outputs of classifiers A. and = 0 otherwise. give the corresponding confusion matrix and PMiss-classification of the ensemble classifier. yC(k)] A nonlinearly combined ensemble classifier then can be regarded an assignment of a class label to each of the f(k). Table 1. wB. yB(k) and yC(k). and output is y  k   I  wA y A  k  wB yB  k  wC yC  k  h where I(x) = 1 if x  0. yB(k) and yC(k) respectively. only one class label can be assigned. Find {wA. Give the confusion matrix of the optimal classifier. Define an ensemble classifier whose inputs are yA(k). and explain why the minimum value of PMiss-classification cannot be 0. Also. Discuss the difficulty to fill in these tables. Labels yA(k) yB(k) yC(k) y(k) 1 0 0 0 0 2 0 0 0 1 3 0 1 1 1 9 of 9 4 0 1 0 1 5 0 0 0 1 6 1 1 1 0 7 1 1 1 0 8 1 1 0 0 9 1 1 1 1 10 1 0 1 1 . Output of the ensemble classifier yA(k) yB(k) yC(k) y(k) 0 0 0 0 0 1 0 1 0 0 1 1 1 0 0 1 0 1 1 1 0 1 1 1 Table 2. wC} and h such that the PMiss-classification is minimum for the training samples. Note that for the same f(k). B.