Fig. 1. An illustration of the principle in computing the correlation coefficient for K=3. (A) The sequence (a section of pdb entry 1msz) is shown to the left. Then, the dssp assignment in three categories is shown by the following three columns, where the number “1” indicates assignment to the category listed by the end of the assignment line. The following three columns (of the same type) are from the psipred prediction (Jones, 1999). The “N” indicates what is to be compared for computing the correlation coefficient contribution for position N in the sequence. The dot product between the two columns has to be carried out (while taking the means over the respective rows into account). Details are described in the following. Note that if the category was -helix vs. non--helix, the two assignments would agree at row “n”. (B) RNA secondary structure comparisons. The top shows two alternative structure assignments for the same sequence, but with a difference in which AU pair (shaded) to assign, which could be because of small confidence in the assignment of the pair. The two structures are shown as upper and lower dot matrices in the matrixplot (Gorodkin et al., 1999) at the bottom, where light grey color indicates non-paired positions, black paired and dark grey positions are the third category of pairs that cannot be assigned confidently to either the pair or the non-pair category.

Fig. 2. RNA secondary structure prediction comparisons. Each curve represents different k-values. In the previous study (Knudsen et al., 2004), k-values of up about 2 indicated useful predictions. Using R₃ to measure the comparisons for three categories, this limit might be pushed towards larger k s. When the cutoffs become larger than 0.4, the R₃ drops drastically off in agreement with the observation in Knudsen et al. (2004), where the base pair content drops off, also at the threshold cutoff of 0.4.

Fig. 3. RNA secondary structure prediction comparisons between original prediction and prediction based on randomized parameters, corresponding to various k-values. Comparisons are made for the initial as well as the cleaned alignments for the cutoffs (A) 0.05 and (B) 0.3. Three categories (K=3) were used and only predicted pairs (non-pairs, respectively) with probability higher than 0.05 (0.3, respectively) were accepted to belong to the category pair (non-pair, respectively), and otherwise were assigned to the “unknown” category. Similar plots are obtained for other cutoff values than 0.05 (0.3, respectively). The higher the k is, the more the randomization, and the worse the prediction becomes (compared to the original prediction). However, in agreement with our expectations, the performance on the cleaned alignment is consistently better, and the generalized correlation coefficient demonstrates this (when k=1, there is no randomization).

The secondary structure prediction methods listed in EVA for different data sets (one to six; August 2003)

The main performance evaluations from EVA are listed along with R₃. The SOV, segment overlap and the three-state prediction accuracy Q₃ measures are indicated along with the overall ranking (rank) by EVA (taking the error signals into account). As few of the downloaded data records were problematic to process due to their inconsistent formats, a revised Q₃ was computed. (For example, some predicted assignments did not have the same length as the dssp assignment line, and the entire entry was therefore discarded.) This is largely in agreement with the original indicated Q₃ values. The “SOV” and “rank” values are the original values. Computing error signals similar to those in EVA (Koh et al., 2003) for R₃ still gives a ranking in complete agreement with the one listed here (data not shown). For each set, the ranking is according to Q₃ values. The protein prediction methods are: phd (Rost, 1996), samt99_sec (Karplus et al., 1998), jpred (Cuff and Barton, 1999), psipred (Jones, 1999), prof_king (Ouali and King, 1999), apssp2 (Raghava, 2000), prospect (Xu and Xu, 2000), phdpsi (Przybylski and Rost, 2001), sspro2 (Pollastri et al., 2002), profsec (Rost, 2003).