Fig. 1. Key frames of an activity simulating a multi-agent ‘Snatch Theft’.

Fig. 2. The structure of traditional HMM: S_t is the state sequence, O_t is the observation sequence.

Fig. 3. The structure of ODHMM: S_t is the state sequence, O_t is the decomposed observation sequence and h is the role assignment.

Fig. 4. Trajectories of each player in the seven tested three-agent activities.

Fig. 5. Features are extracted based on each individual trajectory and the relative position between any two persons. A_t and B_t refer to the positions of person A and B at time t.

Fig. 6. The likelihoods of a sample from Activity 3 for Activity 3 and 4 are (a) close to each other when using ODHMMs without role parameters, (b) distinguishable when using ODHMMs with role parameters.

Fig. 7. Sensitivity analysis of the proposed model. The recognition rates drop only when more than 80% of information of a single agent were lost.

The typical agent assignment table for a three-agent activities

The overall recognitin rate of traditional HMM, ODHMM without role model and ODHMM with role model

Using ODHMM without role parameter, class 3 and class 4 are confused by each other

Using ODHMM with role parameter, the confusion between classes is significantly reduced