CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of MCFold - scored higher in this pairwise comparison

  4. Performance of NanoFolder - scored lower in this pairwise comparison

  5. Compile and download dataset for MCFold & NanoFolder [.zip] - may take several seconds...


Overview

Metric MCFold NanoFolder
MCC 0.456 > 0.352
Average MCC ± 95% Confidence Intervals 0.505 ± 0.068 > 0.404 ± 0.059
Sensitivity 0.509 > 0.404
Positive Predictive Value 0.416 > 0.316
Total TP 1149 > 913
Total TN 308200 > 308068
Total FP 1976 < 2081
Total FP CONTRA 224 < 398
Total FP INCONS 1387 < 1581
Total FP COMP 365 > 102
Total FN 1109 < 1345
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of MCFold and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for MCFold and NanoFolder).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for MCFold and NanoFolder).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for MCFold and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for MCFold and NanoFolder).

^top





Performance of MCFold - scored higher in this pairwise comparison

1. Total counts & total scores for MCFold

Total Base Pair Counts
Total TP 1149
Total TN 308200
Total FP 1976
Total FP CONTRA 224
Total FP INCONS 1387
Total FP COMP 365
Total FN 1109
Total Scores
MCC 0.456
Average MCC ± 95% Confidence Intervals 0.505 ± 0.068
Sensitivity 0.509
Positive Predictive Value 0.416
Nr of predictions 104

^top



2. Individual counts for MCFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01535 0.52 0.57 0.48 21 7096 30 4 19 7 16
CRW_01539 0.70 0.73 0.68 27 7100 28 0 13 15 10
CRW_01563 0.48 0.54 0.43 20 6857 32 3 23 6 17
CRW_01583 0.52 0.57 0.48 21 7096 31 3 20 8 16
CRW_01595 -0.01 0.00 0.00 0 8463 53 11 41 1 37
PDB_00011 - 0.96 0.92 1.00 11 265 0 0 0 0 1
PDB_00017 - 0.66 0.67 0.67 10 688 6 0 5 1 5
PDB_00036 - -0.01 0.00 0.00 0 3447 39 1 38 0 27
PDB_00070 0.60 0.63 0.58 15 2824 14 1 10 3 9
PDB_00113 - 1.00 1.00 1.00 9 222 0 0 0 0 0
PDB_00119 - 1.00 1.00 1.00 10 243 0 0 0 0 0
PDB_00136 - 0.75 0.77 0.74 20 2388 12 0 7 5 6
PDB_00137 - 0.96 0.92 1.00 12 394 0 0 0 0 1
PDB_00160 - 1.00 1.00 1.00 9 267 2 0 0 2 0
PDB_00178 - 1.00 1.00 1.00 11 517 3 0 0 3 0
PDB_00221 - 0.88 0.89 0.89 8 244 2 0 1 1 1
PDB_00312 - 1.00 1.00 1.00 9 426 2 0 0 2 0
PDB_00326 - 0.59 0.64 0.55 18 3708 19 0 15 4 10
PDB_00341 - 1.00 1.00 1.00 16 687 0 0 0 0 0
PDB_00349 - -0.02 0.00 0.00 0 926 20 2 18 0 14
PDB_00417 - 0.41 0.43 0.40 24 14136 40 2 34 4 32
PDB_00430 0.60 0.68 0.53 17 3208 17 6 9 2 8
PDB_00527 - -0.02 0.00 0.00 0 290 10 3 7 0 5
PDB_00570 - -0.01 0.00 0.00 0 2251 27 8 19 0 22
PDB_00571 -0.01 0.00 0.00 0 3287 34 3 31 0 25
PDB_00584 - 0.52 0.65 0.42 11 1459 15 3 12 0 6
PDB_00698 - 1.00 1.00 1.00 19 842 0 0 0 0 0
PDB_00725 - 0.97 0.93 1.00 14 581 1 0 0 1 1
PDB_00734 - 1.00 1.00 1.00 10 221 0 0 0 0 0
PDB_00823 - 0.94 0.89 1.00 8 182 0 0 0 0 1
PDB_00828 0.34 0.37 0.33 10 2455 22 1 19 2 17
PDB_00851 - 0.23 0.24 0.24 10 4711 32 0 32 0 31
PDB_00890 - -0.02 0.00 0.00 0 686 17 2 15 0 14
PDB_00923 - 1.00 1.00 1.00 6 184 1 0 0 1 0
PDB_01009 0.43 0.52 0.35 11 2454 21 5 15 1 10
PDB_01051 1.00 1.00 1.00 13 890 5 0 0 5 0
PDB_01074 - -0.01 0.00 0.00 0 2739 36 3 33 0 25
PDB_01130 - -0.01 0.00 0.00 0 1743 27 7 20 0 20
PDB_01199 - -0.01 0.00 0.00 0 2114 31 2 29 0 24
RFA_00416 1.00 1.00 1.00 15 1470 9 0 0 9 0
RFA_00434 1.00 1.00 1.00 15 1416 8 0 0 8 0
RFA_00436 1.00 1.00 1.00 15 1416 8 0 0 8 0
RFA_00442 1.00 1.00 1.00 15 1416 8 0 0 8 0
RFA_00446 1.00 1.00 1.00 15 1470 9 0 0 9 0
RFA_00604 0.97 0.97 0.97 34 13495 15 0 1 14 1
RFA_00664 -0.02 0.00 0.00 0 974 16 0 16 0 14
RFA_00674 0.96 0.93 1.00 13 1115 7 0 0 7 1
RFA_00675 -0.02 0.00 0.00 0 974 18 0 16 2 14
RFA_00677 0.46 0.50 0.44 7 974 12 0 9 3 7
RFA_00684 0.57 0.57 0.57 8 976 10 0 6 4 6
RFA_00716 0.05 0.07 0.06 1 928 17 0 17 0 13
RFA_00717 0.36 0.43 0.32 6 884 13 0 13 0 8
RFA_00730 0.33 0.42 0.28 5 885 14 5 8 1 7
RFA_00763 0.72 0.75 0.69 9 890 10 0 4 6 3
RFA_00765 0.72 0.75 0.69 9 890 10 0 4 6 3
RFA_00770 -0.01 0.00 0.00 0 1992 29 2 22 5 18
RFA_00773 0.43 0.50 0.38 9 1929 16 4 11 1 9
RFA_00808 0.39 0.44 0.35 7 1996 19 1 12 6 9
SPR_00030 - 0.19 0.24 0.17 5 2455 26 5 20 1 16
SPR_00066 0.26 0.33 0.21 7 2817 26 5 21 0 14
SPR_00228 0.44 0.60 0.33 12 3792 27 13 11 3 8
SPR_00314 0.74 0.81 0.68 17 2901 13 0 8 5 4
SPR_00331 0.47 0.55 0.41 11 2748 19 4 12 3 9
SPR_00333 0.53 0.62 0.46 13 2747 18 4 11 3 8
SPR_00353 0.73 0.81 0.65 17 2824 13 2 7 4 4
SPR_00398 - -0.01 0.00 0.00 0 2187 28 5 19 4 13
SPR_00452 0.19 0.24 0.16 5 2819 27 5 21 1 16
SPR_00456 0.48 0.57 0.41 12 2821 20 2 15 3 9
SPR_00462 0.28 0.37 0.23 7 2384 24 7 17 0 12
SPR_00476 0.34 0.38 0.31 8 2749 22 3 15 4 13
SPR_00496 0.37 0.43 0.33 9 2748 21 0 18 3 12
SPR_00497 0.34 0.38 0.31 8 2749 22 3 15 4 13
SPR_00502 0.45 0.52 0.39 11 2898 21 6 11 4 10
SPR_00509 0.20 0.25 0.18 5 2898 28 0 23 5 15
SPR_00512 0.64 0.70 0.58 14 2902 17 0 10 7 6
SPR_00542 0.45 0.57 0.36 12 3537 23 4 17 2 9
SPR_00557 0.16 0.19 0.15 4 2823 27 3 20 4 17
SPR_00832 - -0.01 0.00 0.00 0 1627 26 5 21 0 15
SPR_00868 0.47 0.57 0.40 12 2896 19 5 13 1 9
SPR_00932 0.21 0.24 0.19 5 2824 26 4 17 5 16
SPR_00957 0.37 0.43 0.33 9 2976 24 1 17 6 12
SPR_01040 0.76 0.81 0.71 17 2751 11 1 6 4 4
SPR_01143 0.48 0.57 0.41 12 2527 19 2 15 2 9
SRP_00005 - 0.09 0.13 0.08 3 3967 36 7 28 1 21
SRP_00038 0.25 0.27 0.23 9 5011 33 1 29 3 24
SRP_00046 0.34 0.38 0.32 12 4715 29 0 26 3 20
SRP_00138 - 0.96 0.96 0.96 26 3978 12 0 1 11 1
SRP_00170 0.40 0.48 0.33 12 4059 26 8 16 2 13
SRP_00193 - -0.01 0.00 0.00 0 6852 51 9 42 0 36
SRP_00194 0.68 0.71 0.65 15 3217 20 0 8 12 6
SRP_00200 0.89 0.92 0.87 33 6865 17 1 4 12 3
SRP_00216 0.76 0.81 0.71 22 3372 15 1 8 6 5
SRP_00233 0.28 0.30 0.26 10 5113 30 0 28 2 23
SRP_00243 0.97 0.97 0.97 30 5120 14 0 1 13 1
SRP_00244 0.47 0.51 0.43 20 6857 28 4 22 2 19
SRP_00266 0.30 0.34 0.27 10 4619 30 3 24 3 19
SRP_00269 -0.01 0.00 0.00 0 5008 43 3 39 1 33
SRP_00273 0.75 0.78 0.74 31 6744 18 1 10 7 9
SRP_00285 0.33 0.37 0.31 11 3705 27 2 23 2 19
SRP_00303 0.65 0.69 0.61 22 6634 26 1 13 12 10
SRP_00333 0.57 0.62 0.54 21 5112 19 5 13 1 13
SRP_00342 0.72 0.76 0.68 19 3977 21 1 8 12 6
SRP_00358 0.65 0.71 0.60 25 6286 24 3 14 7 10
SRP_00383 0.10 0.13 0.09 3 3208 33 3 26 4 21

^top



Performance of NanoFolder - scored lower in this pairwise comparison

1. Total counts & total scores for NanoFolder

Total Base Pair Counts
Total TP 913
Total TN 308068
Total FP 2081
Total FP CONTRA 398
Total FP INCONS 1581
Total FP COMP 102
Total FN 1345
Total Scores
MCC 0.352
Average MCC ± 95% Confidence Intervals 0.404 ± 0.059
Sensitivity 0.404
Positive Predictive Value 0.316
Nr of predictions 104

^top



2. Individual counts for NanoFolder [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01535 0.20 0.24 0.17 9 7088 43 7 36 0 28
CRW_01539 0.10 0.11 0.10 4 7099 40 1 36 3 33
CRW_01563 0.11 0.14 0.11 5 6856 43 5 37 1 32
CRW_01583 0.36 0.41 0.33 15 7095 31 8 22 1 22
CRW_01595 -0.01 0.00 0.00 0 8459 57 10 46 1 37
PDB_00011 - 0.96 0.92 1.00 11 265 0 0 0 0 1
PDB_00017 - 0.66 0.67 0.67 10 688 5 1 4 0 5
PDB_00036 - -0.01 0.00 0.00 0 3457 29 1 28 0 27
PDB_00070 0.25 0.29 0.23 7 2819 24 4 20 0 17
PDB_00113 - 0.94 0.89 1.00 8 223 0 0 0 0 1
PDB_00119 - 0.95 0.90 1.00 9 244 0 0 0 0 1
PDB_00136 - 0.70 0.73 0.68 19 2387 9 1 8 0 7
PDB_00137 - 0.87 0.77 1.00 10 396 0 0 0 0 3
PDB_00160 - 0.89 0.89 0.89 8 267 2 0 1 1 1
PDB_00178 - -0.02 0.00 0.00 0 517 11 1 10 0 11
PDB_00221 - 0.94 0.89 1.00 8 245 0 0 0 0 1
PDB_00312 - 1.00 1.00 1.00 9 426 1 0 0 1 0
PDB_00326 - 0.58 0.64 0.53 18 3707 16 4 12 0 10
PDB_00341 - 0.47 0.44 0.54 7 690 6 0 6 0 9
PDB_00349 - 0.10 0.14 0.10 2 926 18 0 18 0 12
PDB_00417 - 0.00 0.00 0.00 0 14138 58 2 56 0 56
PDB_00430 0.38 0.44 0.34 11 3208 21 6 15 0 14
PDB_00527 - -0.02 0.00 0.00 0 292 8 2 6 0 5
PDB_00570 - -0.01 0.00 0.00 0 2257 21 0 21 0 22
PDB_00571 -0.01 0.00 0.00 0 3286 36 6 29 1 25
PDB_00584 - 0.47 0.53 0.43 9 1464 12 1 11 0 8
PDB_00698 - 0.97 0.95 1.00 18 843 0 0 0 0 1
PDB_00725 - 0.82 0.80 0.86 12 581 2 0 2 0 3
PDB_00734 - 0.83 0.70 1.00 7 224 0 0 0 0 3
PDB_00823 - -0.04 0.00 0.00 0 183 7 0 7 0 9
PDB_00828 0.30 0.33 0.28 9 2453 23 0 23 0 18
PDB_00851 - 0.16 0.17 0.17 7 4711 35 3 32 0 34
PDB_00890 - -0.02 0.00 0.00 0 690 13 2 11 0 14
PDB_00923 - 1.00 1.00 1.00 6 184 2 0 0 2 0
PDB_01009 0.27 0.33 0.23 7 2455 25 3 20 2 14
PDB_01051 -0.02 0.00 0.00 0 885 18 1 17 0 13
PDB_01074 - -0.01 0.00 0.00 0 2747 28 3 25 0 25
PDB_01130 - -0.01 0.00 0.00 0 1742 28 5 23 0 20
PDB_01199 - -0.01 0.00 0.00 0 2116 29 2 27 0 24
RFA_00416 0.94 1.00 0.88 15 1468 6 2 0 4 0
RFA_00434 0.84 1.00 0.71 15 1410 8 6 0 2 0
RFA_00436 0.86 1.00 0.75 15 1411 7 5 0 2 0
RFA_00442 0.94 1.00 0.88 15 1414 4 2 0 2 0
RFA_00446 0.85 0.93 0.78 14 1467 7 3 1 3 1
RFA_00604 0.42 0.54 0.33 19 13472 53 14 25 14 16
RFA_00664 -0.01 0.00 0.00 0 978 13 0 12 1 14
RFA_00674 0.12 0.14 0.13 2 1112 15 0 14 1 12
RFA_00675 0.28 0.29 0.29 4 976 10 1 9 0 10
RFA_00677 0.62 0.64 0.60 9 975 10 0 6 4 5
RFA_00684 0.74 0.71 0.77 10 977 5 2 1 2 4
RFA_00716 -0.02 0.00 0.00 0 929 17 0 17 0 14
RFA_00717 0.64 0.64 0.64 9 889 5 2 3 0 5
RFA_00730 0.38 0.42 0.36 5 889 10 3 6 1 7
RFA_00763 0.38 0.42 0.36 5 889 10 3 6 1 7
RFA_00765 0.39 0.42 0.38 5 890 9 2 6 1 7
RFA_00770 -0.01 0.00 0.00 0 1996 24 0 20 4 18
RFA_00773 0.93 1.00 0.86 18 1932 9 3 0 6 0
RFA_00808 0.83 1.00 0.70 16 1993 7 7 0 0 0
SPR_00030 - 0.47 0.52 0.42 11 2459 16 2 13 1 10
SPR_00066 0.47 0.57 0.39 12 2819 20 4 15 1 9
SPR_00228 0.44 0.60 0.32 12 3791 26 15 10 1 8
SPR_00314 0.47 0.57 0.39 12 2895 21 9 10 2 9
SPR_00331 0.48 0.60 0.39 12 2744 19 6 13 0 8
SPR_00333 0.25 0.33 0.21 7 2741 27 10 17 0 14
SPR_00353 0.64 0.81 0.52 17 2817 16 8 8 0 4
SPR_00398 - -0.01 0.00 0.00 0 2185 28 10 16 2 13
SPR_00452 0.27 0.33 0.23 7 2820 24 7 16 1 14
SPR_00456 0.49 0.57 0.43 12 2822 19 5 11 3 9
SPR_00462 0.65 0.74 0.58 14 2391 11 4 6 1 5
SPR_00476 0.47 0.52 0.42 11 2749 17 4 11 2 10
SPR_00496 0.47 0.57 0.40 12 2745 22 7 11 4 9
SPR_00497 0.27 0.33 0.23 7 2745 23 5 18 0 14
SPR_00502 0.27 0.33 0.23 7 2896 26 2 21 3 14
SPR_00509 0.23 0.30 0.19 6 2895 25 8 17 0 14
SPR_00512 0.61 0.75 0.50 15 2896 15 6 9 0 5
SPR_00542 0.26 0.33 0.22 7 3538 28 4 21 3 14
SPR_00557 0.42 0.52 0.34 11 2818 22 8 13 1 10
SPR_00832 - 0.53 0.67 0.43 10 1630 13 5 8 0 5
SPR_00868 0.49 0.57 0.43 12 2898 19 5 11 3 9
SPR_00932 0.43 0.52 0.35 11 2819 21 7 13 1 10
SPR_00957 0.63 0.76 0.53 16 2973 15 7 7 1 5
SPR_01040 0.15 0.19 0.13 4 2744 27 4 23 0 17
SPR_01143 0.41 0.48 0.36 10 2528 18 8 10 0 11
SRP_00005 - -0.01 0.00 0.00 0 3966 39 6 33 0 24
SRP_00038 0.23 0.27 0.20 9 5005 36 1 35 0 24
SRP_00046 0.33 0.38 0.29 12 4712 30 3 26 1 20
SRP_00138 - 0.44 0.52 0.39 14 3969 24 8 14 2 13
SRP_00170 0.29 0.36 0.25 9 4059 28 4 23 1 16
SRP_00193 - 0.16 0.19 0.15 7 6855 43 4 37 2 29
SRP_00194 0.27 0.33 0.23 7 3209 27 5 19 3 14
SRP_00200 0.41 0.47 0.35 17 6855 31 4 27 0 19
SRP_00216 0.44 0.48 0.41 13 3371 19 4 15 0 14
SRP_00233 0.25 0.30 0.22 10 5106 35 4 31 0 23
SRP_00243 0.47 0.55 0.40 17 5109 25 7 18 0 14
SRP_00244 0.38 0.44 0.34 17 6853 33 2 31 0 22
SRP_00266 0.55 0.66 0.46 19 4615 22 8 14 0 10
SRP_00269 0.46 0.52 0.43 17 5010 24 6 17 1 16
SRP_00273 0.36 0.40 0.33 16 6737 34 6 27 1 24
SRP_00285 0.38 0.43 0.35 13 3704 24 6 18 0 17
SRP_00303 0.36 0.41 0.33 13 6630 27 3 24 0 19
SRP_00333 0.20 0.24 0.19 8 5108 35 3 32 0 26
SRP_00342 0.20 0.24 0.17 6 3970 29 7 22 0 19
SRP_00358 0.32 0.37 0.28 13 6282 33 4 29 0 22
SRP_00383 -0.01 0.00 0.00 0 3205 35 9 26 0 24

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.