Statistical Evaluation in Detail
INTRODUCTION
Each analyst’s data are evaluated using the same statistical parameters. Every assessment is based on comparing a petrographer’s set of results against the group or pooled sample means and group or observed standard deviations. These established values may change slightly from exercise to exercise, as additional values are added to the data base.
STATISTICAL EVALUATION
All data for a particular parameter and sample are pooled, that is, all the data for a particular sample are used in the current exercise to calculate the statistical parameters. The sample mean and observed standard deviation are then calculated as per equations (a) and (b) below.
(a)
Sample mean for i analysts, jth.coal sample:
Observed standard deviation, jth.coal sample:
Two sets of comparative data are then calculated for each analyst, for each sample analysed: the unsigned multiple of the standard deviation (UMSD; equation (c)) and the signed multiple of the standard deviation (SMSD; equation (e)). The individual UMSD and SMSD values are then summed and a mean value is calculated for each analyst (AUMSD, equation (d); ASMSD, equation (f)). Accreditation is based on a petrographer achieving an AUMSD <1.5.
(c)
Unsigned multiple of the standard deviation for the ith. analyst and jth. sample:
- (d)
AUMSD is the mean of the sum of the unsigned multiples, that is, the sum of the unsigned multiples of the standard deviation divided by the j samples analysed. It is an indicator of dispersion about the group means and a measure of the accuracy of the analyst.
- (e)
Signed multiple of the standard deviation for the ith. analyst and jth. sample:
- (f)
ASMSD is the mean of the sum of the signed multiples, that is, the sum of the signed multiples of the standard deviation divided by the j samples analysed. It is a measure of bias about the group means and indicates the consistency of an analyst.
PETROGRAPHER’S DATA BASE
Initial entry into an Accreditation Program requires the analysis of a set of samples (6 for SCAP and DOMVR and two or three blends for the CBAP). Accreditation is granted if data from all samples fall within pre-determined statistical criteria. The data will be rated, based on the UMSD value, in decreasing order of agreement with the pooled values as A1,….,A6, where A5 and A6 are the statistically worst data compared with the established values. In the next Accreditation Exercise, two (2) samples will be distributed for analysis; let us call these B1 and B2. These two data will be added to the analyst’s file, after removing the statistically worst two (2) data points, that is, A5 and A6. This now gives the analyst a new data set comprising A1,…,A4, B1, B2. These six (6) data entries will then be evaluated for concordance using the statistical techniques agreed on by the Accreditation Sub-Committee. Subsequent additions to the data set from further Accreditation Exercises will mean that the next two (2) data points from the original exercise are dropped (that is, A3 and A4) and the two (2) new data points added (that is, C1 and C2) and statistically evaluated. This process of addition of new data and deletion of old data will continue as shown below:
| Initial entry | A1, A2, A3, A4, A5, A6 Exercise 1 (e.g., 2002) Accreditation | |
| Accreditation 1 | A1, A2, A3, A4, B1, B2 Exercise 2 (e.g., 2004) Accreditation | |
| Accreditation 2 | A1, A2, B1, B2, C1, C2 Exercise 3 | |
| Accreditation 3 | B1, B2, C1, C2, D1, D2 Exercise 4 | |
| Accreditation 4 | C1, C2, D1, D2, E1, E2 Exercise 5 |
As the process continues, two (2) new data points are added and two (2) will be dropped from an analyst’s data set.
The procedure is slightly different for the CBAP as the number of components of each blend might be different.
SCREENING METHOD FOR BEGINNERS
For the 2010-2011 exercise a Screening Method for Beginners (SMB) will be established to limit outliers in the accreditation database.
The SMB will be applied as follows:
- After a first evaluation, if the beginner is accredited, he/she will be considered as a normal participant in the future;
- If the beginner is not accredited, the UMSD values are checked. If these values are always below 3SD, then the results are maintained in the database. If one or more values are over 3SD, all the values from the participant are removed from the database and the results for the others are evaluated after removing these results.
The SMB will be also applied to all the participants that analyse six coals.
PARAMETERS EVALUATED IN THE DIFFERENT ACCREDITATION PROGRAMS
The variables used in the different accreditation programs to calculate AUMSD and ASMSD are summarised in the table below.
| ACCREDITATION PROGRAM | VARIABLES TESTED
|
NUMBER OF SAMPLES | ||
| Vitrinite Reflectance (%) | Vitrinite content (% mineral-free basis) |
Initial accreditation | Continuing accreditation | |
| SCAP | X |
X |
6 |
2 |
| CBAP Blend Overall | X |
X |
2/3 |
1 |
| Blend component A | X |
X |
||
| Blend component B | X |
X |
||
| Blend component C (if present) | X |
X |
||
| DOMVR | X |
6 |
2 |
|