This paper describes an anomaly behavior detection system for elderly people based on an omni-directional vision sensor. The proposed monitoring system automatically learns daily behavior patterns and detects unusual behavior patterns and actions using probabilistic inference approach. Bayesian Network is constructed using image feature values such as the area and center-of gravity values extracted from the captured image sequence. Human trajectories in last one hour are also included in image features in order to detect the prolonged motionlessness. Some experiments based on the investigation of elderly people's typical daily behavior patterns are performed to verify the effectiveness of the proposed system.