Activity recognition using dynamic Bayesian networks
Justin Muncaster, Yunqian Ma



Overview
We present a method to classify activities based on the trajectory of
feature measurements. We propose a dlevel hierarchically organized graphical
model with roots in the hierarchical hidden markov
model. We propose to start with the minimum constraints to enforce hierarchy
and add dependencies to keep the number of model parameters small. We also
propose to use deterministic annealing to partition the feature space and
automatically discover lowlevel states in our model. We apply our algorithm
to real world data to demonstrate its effectiveness. Our tests show qualitatively
that our method can recognize activities based on features from noisy tracker,
and can robustly classify trajectories that are abnormal with respect to the
training data. Details
1.1 Motivation With the falling cost of inexpensive sensors such as cameras the amount of
data has increased substantially. Frequently, one wishes to classify streams
of data based on the activity that is occurring. For example, surveillance
applications may want to segment normal activities from abnormal activities for
purposes of data retrieval after a crime has occurred. Defining activities
usually relies on labeled training data. Labeling training data is a tedious
task, and thus a technique that minimizes the amount of labeling would be
beneficial for practical use. 1.2 Introduction In this research we propose a technique that relies on providing a single label for the activity present in a sequence of video, as opposed to the labeling of relevant features in the video. Given a trajectory of feature measurements and a label for a highlevel event, our system first clusters the features to discover the lowerlevel events. We suppose that a highlevel event is defined by a sequence of lowlevel events. In a given high level event, the tracked object will “walk” from one low level cluster to another. We then the probability of a given trajectory through lowlevel events given a particular high level event to determine the high level activity. Figure 1. Clustering of lowlevel states. 1.3 Our method During the training phase we use the deterministic annealing technique developed by Rose for clustering. This essentially discretizes the space into a finite number of states, where each data point has fuzzy membership to each cluster. Next, we use this result to initialize a hierarchically constrained dynamic Bayesian network akin to Murphy’s representation of a hierarchical hidden Markov model. Finally, we constrain the lowest level of the hierarchy to follow a Coxian phase distribution in order to robustly model lowlevel activities of varying duration. Figure 2. Pictorial representation of lowlevel HMM from clustering. This constitutes one level of the dlevel hierarchical dynamic Bayesian network. Finally, we use each feature trajectory in the training data to learn the relevant parameters, i.e. transition probabilities and observation likelihoods. We learn the unobservable parameters using the EM algorithm. 1.4 Experimental Results We tested our algorithm using the video clips of a shopping center in
The results in figure 3 are qualitative in nature but suggest that our technique is doing a good job in recognizing activity. We have tested abnormal trajectories through this model and received good classifications of the activities in the video. In future work we plan on obtaining quantitative results and examining how to explicitly distinguish abnormal activity from normal activity. Publications
J. Muncaster and Y. Ma, Activity recognition using dynamic Bayesian
networks with automatic state selection, Submitted
to Workshop on Motion and Video Computing, J. Muncaster, Classification of abnormal activities in video, Graduate
Student Research Conference, UCSB, 2006. 