The reports for EQA participants


In the EQA Plan you can find brief list of a documents that are usually attached to the final report for each programme. These documents make up the evaluation of the rounds of the programme. The following table gives a brief overview of the most common documents including short description and also sample is attached for each document.

 

Content

Final report to the evaluation of the EQA round


Annual overview of certificates

Annual overview of tests

Certificate of approval

Confirmation of attendance

Complex statistics

Histograms

Result sheet (qualitative results)

Result sheet (quantitative results) with evaluation of comparability

Result sheet (quantitative results) with evaluation of comparability and traceability

Result sheet (scoring system)

Results with uncertainties


Final report to the evaluation of the EQA round
Click the picture to enlarge.
 

The basic document that each EQA participant receives is the Final report to the evaluation of the EQA round. This report is accompanied by other documents, listed in the table at the end of the final report. The final report also includes the supervisor's comment on the results of the round. Other (not listed here) tables, graphs, pictures, etc. may be a part of or annex to the final report. Participants who have not submitted their results also receive the final report, but without attachments.
 
Note: Available in English only if there are the participants from the abroad.

 

Annexes to the final report

Annual overview of certificates
Click the picture to enlarge.
 
The document serves the participants to gain an overall view of the acquired certificates and their validity for the previous year. We send this document to each participant once per year, after finishing the evaluation of all rounds of the year (thus usually in January of the next year).
The report lists all the tests for which the EQA system issued a certificate (regardless of whether the test was carried out at the laboratory mentioned at the top of the document). The tests carried out by the laboratory, the date of issue, and expiry date of the last issued certificate are printed.
 
Note: We do not send this report to the participants who attended only small number of the rounds.

 

Annual overview of tests
Click the picture to enlarge.
 
The document serves the participants to gain an overall view of the success of tests in the previous year. Also provides a comparison of the success of the participant (column individual results) and the set of all participants (summary results). We send this document to each participant once per year, after finishing the evaluation of all rounds of the year (thus usually in January of the next year).
The table shows for EQA programme (and for each test) the number of test results that the laboratory in the appropriate year reported, and success. For comparison purposes, also an overview of the results for each EQA programme and test (including success) is presented.
 
It can ocure (and it is quite common) in the right half of the table, showing the overall results, that the totals at the header (EQA programme) do not match the sum of the numbers for each test of the programme. This is due to the fact that the header row - for the programme - shows totals for the entire programme, but the only lines with individual tests that the laboratory performed are printed. Thus, if the laboratory does not perform all the tests of the programme then there are "missing" some of the lines in the report and totals shown in line for the programme are greater than the sum of individual (printed) tests.
 
Note: We do not send this report to the participants who attended only small number of the rounds.

 

Certificate of approval
Click the picture to enlarge.
 
The certificate of approval is awarded only in some EQA rounds. The certificate of approval is awarded only for successful tests and the laboratories use it mainly in contact with their partners when arranging their research tasks, etc.
Left column lists all analytes where the certificate of comparability was granted (column Certification of comparability).
Additionally, if some analytes dispose of traceable assigned values (traceable to higher metrological standard), the list of these analytes, where the laboratory fulfilled the criteria of traceability, is listed in the second column (column Certification of traceability).
The criteria for comparability and traceability should differ even for one test.

 

Confirmation of attendance
Click the picture to enlarge.
 
The conditions defining the rules for the issuing of confirmations of attendance are set out in the EQA Plan for the relevant year. The confirmation of attendance shall contain a list of all tests the results of which the participant reported in the round, irrespective of the accuracy of those results. This document serves as evidence of the participation in the relevant EQA round. If the participant has provided the identification of the measuring system, this identification is printed for each test (as an additional line).

 

Complex statistics
Click the picture to enlarge.
 

Each participant receives only the evaluation of the test determined in his laboratory. Thus, large amount of paper should be saved. Complete statistics are available on our web page.

First line of the report identifies the EQA round and participant (by means of the code of the laboratory). The name of the test is located on the second line.
Your results (for both samples) you can find in the upper left corner (above the Youden plot) together with the identification of your own group (group ii which your results were evaluated etc.).

The remainder of the page should be divided into 3, relatively independent, parts described in paragraphs followed:

  1. Youden (XY-scatter) graph (upper left)
    The unit, short description of differently coloured points and number of points lying out of the graph’s range are given in the heading of each graph. Each point represents the position of one laboratory’s result; result A is drawn on the x-axis and the result B is drawn on the y-axis. If there is a rectangle inside the graph, it defines the area of correct results. Black points represent results of the participants of your own group and grey points represent other results.
  2. Two year history of your P-score (upper right)
    You can find the P-score of your results in this graph. The symbols used in the graph are explained in the legend of the graph. The interpretation of your own P-score is very simple:
          | P | < 1   excellent result (inside tolerance limit), no comment
          | P | > 1   doubtful result, method check is recommended
    Complete algorithm of P-score calculation can be found on our web page.
  3. Summary statistics (bottom half of the page)
    Calculated statistical parameters are given for each test and also for each evaluated group. The groups may be specified as to the same method or the same method and same manufacturer (of the kit, instrument, etc.). The standard deviation, coefficient of variation and percentiles (16th and 84th) are given only in groups the size of which is 5 at least.

 

Histograms
Click the picture to enlarge.
 
Histograms are intended to be used in the situations where Youden plots cannot be used. Namely it is in these cases:
  • The number of control samples is different from 2.
  • The results of any sample are close to zero.
  • The unit of the results is titre.
Each column represents the number of the results that lie in the interval indicated on the x axes.
White columns describe the set of all results.
Red columns describe the results of the own peer group (peer group that the result of the participant belongs to). Peer group is described in the upper right corner of the graph.
In case of there are some results lying out of the range of the x axes (to the left or right) the number of these results is printed in the left (or right) bottom corner of the graph.
The red spot placed in the top of the graph indicates the position of the individual result of the participant. If the result lies out of the range of the x axes the spot changes to the arrow that shows the direction of the participant's result's value.
 
Note: This report should be printed either in colour (see example to the left) or in grey scale (in this case dark shadow replaces the red colour). Each participant receives histograms only for the results of analytes that he entered in the result sheet.

 

Result sheet (qualitative results)
Click the picture to enlarge.
 
List of qualitative results summarises results of the given laboratory and entire statistics of all results (frequency table) with marked assigned values. The symbols used are explained in the supporting narrative.
Long term success (last 2 years) (both for individual tests and for whole round) is based on the same rules as the certification (if it is performed). By default the test is considered to be successful if the participant found expected or acceptable result for the test in all samples.
In each line starting with the name of the test also the basic information provided by participant are printed (e.g. identification of measuring system, principle of measurement, manufacturer of the kit etc.).
 
Significant note: If both quantitative and qualitative results are required for the analyte, always qualitative results are preferred in the process of the long term success evaluation of the laboratory.

 

Result sheet (quantitative results) with evaluation of comparability
Click the picture to enlarge.
 
List of quantitative results gives summary information about the results and it compares them with assigned values. All symbols are sufficiently explained in the supporting narrative.
Long term success (last 2 years) (both for individual tests and for whole round) is based on the data from the column C (total evaluation of the test determination). Lines where C is "+" are calculated as successful, lines where C is "-" as unsuccessful and lines where C is "±" are not included in the calculation of long term success.
In each line starting with the name of the test also the basic information provided by participant are printed (e.g. identification of measuring system, principle of measurement, manufacturer of the kit etc.).
 
Significant note: If both quantitative and qualitative results are required for the analyte, always qualitative results are preferred in the process of the long term success evaluation of the laboratory.

 

Result sheet (quantitative results) with evaluation of comparability and traceability
Click the picture to enlarge.
 
List of quantitative results supplemented by information about results of the given group (e.g. used the same method and same manufacturer of the diagnostic kit). All symbols are sufficiently explained in the legend. This type of result list is used mainly for immunoanalytical rounds (tumour markers, hormones) where there is no possibility to state one assigned value for all groups. The result of the participant is compared to the assigned value of the group (comparability) and to the RMP, if it is available, (traceability).
In each line starting with the name of the test also the basic information provided by participant are printed (e.g. identification of measuring system, principle of measurement, manufacturer of the kit etc.).
 

 

Result sheet (scoring system)
Click the picture to enlarge.
 


Not only individual answers (classified as true or false) of the participant are evaluated in some programmes, but these answers are also scored. The target of scoring system is to give the general information to the participant concerning any logical group of questions/tests that are evaluated in whole on the basis of score reached. We chose this approach especially in the programmes where results (and thus also result sheet) consist of many partial answers and it is difficult to get the general information about the ratio between true and false results.
Scoring system offers the possibility to serve as the global result in those results sheets where too much individual tests are placed.
Actually we use the scoring system in these programmes:
  • Peripheral Blood Morphology Evaluation (DIF) - individual samples (A and B) are evaluated on the basis of scoring system. Thus 2 tests are evaluated in the end of the result sheet (see pictures to the left).
  • Peripheral Blood Smears - Photos (NF) - individual photos (1, 2, 3 and 4) are evaluated on the basis of scoring system. Thus 4 tests are evaluated in the end of the result sheet.
  • Bone Marrow Aspirate Film (NKDF) - individual photos (1 and 2) and general view of each patient are evaluated on the basis of scoring system. Because there are 2 patients, 6 tests are evaluated in the end of the result sheet.

 

Results with uncertainties
Click the picture to enlarge.
 
Scatter graphs with whiskers (representation of individual uncertainties)
This report shows the individual uncertainties that the participating laboratories reported in the result forms. Only participants, that reported their uncertainties, receive this report - other laboratories can view these reports as the part of the summary evaluation of the EQA round on the web.
One graph is constructed for each test and sample.
The name of the test and sample identification should be found in the header of the graph. Additionally, there is the number of the results placed in the graph area, and number of results, that exceeded the y axes range. Central horizontal line shows the position of assigned value. Horizontal dashed lines above and below this line show the expanded uncertainty of the assigned value. Upper and lower continuous lines enclose the acceptable range of results.
Individual results of participants are shown as the points in ascending order, and for each point the error bar is constructed as the participant's individual uncertainty (up to 2011 error bars showed uc, starting 2012 SEKK the bars show expanded uncertainty Uc (k = 2)). Participant's individual result is printed in the bottom right corner, and it is marked bold in the graph area.
Basic interpretation of the position of individual result and its uncertainty should be found as a model example in the picture below:

  • Laboratories 1, 2, 10 and 11 reported results out of the toleration limit.
  • Laboratories 3 a 9 reported results inside the toleration limit, but uncertainty of their determination (showing the range of probable true value) exceeds the toleration limit (on one side). This show, that their successful determination in this round contributed not only their good work, but also a piece of luck (probability, that their result should lie outside the toleration limit, is not negligible).
  • Extreme values reported laboratory 5: the result of determination is almost ideal, but its uncertainty is so high, that the true value should lie outside the toleration limit (with high probability).
  • Partially we can consider extreme the result of laboratory 6, which reported individual uncertainty much lower than other participants, and than the uncertainty of the assigned value (dashed lines). It shows that the result seeming ideal should be considered specious.

The axes y of the graph is calibrated in %.
The assigned values and results of all participants are recalculated to % and shown in one graph. Recalculation is performed this way: assigned value is declared 100 % and subsequently using this value the uncertainty of assigned value and the results of all participants are recalculated to %. Recalculated values are shown in the graph and the whiskers in the graph show the uncertainties of results in %. This approach prevents the need of construction of many graphs (for each group with different AV).