This is a real time facial expression recognition system. Initially, a front view image of the tester’s neutral face is captured. This image is processed to detect the tester’s face region, extract the eyebrows, eyes, nose and mouth features. The features location are then mapped to the real time video according to the video’s resolution. Once the initialization is completed, the tester can express his emotion freely. The feature points can be predicted and tracked frame by frame using Kalman filter and Lucas-Kanade optical flow method. The displacement and velocity of each feature points are recorded at each frame. Once a expression is occur, the detection system will choose the maximum value among all the normal dot products and make a judgement displaced at the video. When one expression is over, the tester can expression his following emotions or re-initial the system if any tracker is lost.