Appendix A (Diagnostic Codes and Search Strings)
Dx code  String 

dx1  Suspicious for adenocarcinoma 
dx1  Cannot exclude adenocarcinoma 
dx1  Adenocarcinoma can’t excluded 
dx1  Suspicious for invasive adenocarcinoma 
dx2  Invasive adenocarcinoma 
dx2  Adenocarcinoma 
dx2  Invasive carcinoma 
dx3  Intramucosal adenocarcinoma 
dx3  Of highgrade dysplasia 
dx3  Of high grade dysplasia 
dx3  With highgrade dysplasia 
dx3  With high grade dysplasia 
dx3  Focal high grade dysplasia 
dx3  Focal highgrade dysplasia 
dx3  Showing high grade dysplasia 
dx3  Showing highgrade dysplasia 
dx4  Suspicious for lymphoma 
dx4  Atypical lymphoid proliferation 
dx5  MALT lymphoma 
dx5  Mantle cell lymphoma 
dx6  Tubular adenoma 
dx6  Tubular adenomata 
dx7  Hyperplastic polyp 
dx7  Hyperplastic polyp 
dx8  Hyperplastic changes 
dx8  Hyperplastic mucosal changes 
dx8  Hyperplastic features 
dx8  Hyperplasticlike features 
dx9  Tubulovillous adenoma 
dx9  Tubularvillous adenoma 
dx9  Tubulovillous adenoma 
dx9  Villotubular adenoma 
dx10  Villous adenoma 
dx11  Sessile serrated adenoma 
dx11  Sessile serrated polyp 
dx11  Serrated sessile adenoma 
dx11  Serrated polyp, favor adenoma 
dx11  Serrated polyp, favour adenoma 
dx11  Serrated polyps, favor adenomata 
dx12  Traditional serrated adenoma 
dx13  Serrated adenoma 
dx13  Serrated adenomata 
dx13  Serrated polyp 
dx14  No pathologic finding 
dx14  Unremarkable fragment of large bowel mucosa 
dx14  Unremarkable large bowel mucosa 
dx14  Benign colonic mucosa 
dx14  Benign large bowel mucosa 
dx14  Normal colonic mucosa 
dx14  NO DIAGNOSTIC ABNORMALITY 
dx14  Unremarkable colonic mucosa 
dx14  Mucosa without significant pathology 
dx14  Mucosa within normal limits 
dx14  No specific pathology 
dx14  No significant histopathologic abnormality 
dx14  NEGATIVE for evidence of significant pathology 
dx14  No significant pathological changes 
dx14  No pathological changes 
dx14  No evidence of polyp 
dx14  Polypoid mucosa 
dx14  Colonic mucosa with prominent lymphoid follicles 
dx14  Negative for apparent pathology 
dx14  No pathological diagnosis 
dx14  CAUTERY ARTIFACT, NOT FURTHER DIAGNOSTIC 
dx14  Reactive changes, NEGATIVE 
dx14  Colonic mucosa with lymphoid aggregate 
dx14  Large bowel mucosa with no definite polyp 
dx14  Nondiagnostic polypoid colonic mucosa 
dx14  Bowel mucosa with no significant findings 
dx14  Bowel mucosa with no evidence of polyp 
dx14  No polyp 
dx14  No specific polyp 
dx14  No definitive polyp 
dx14  No definite polyp 
dx14  No specific pathologic diagnosis 
dx14  No significant pathology identified 
dx14  No significant pathological abnormalities 
dx14  No findings 
dx14  No histopathological abnormality 
dx14  Prominent mucosal folds 
dx14  Mucosa, likely mucosal fold 
dx14  Without diagnostic abnormality 
dx14  Unremarkable mucosal tissue 
dx14  No significant findings 
dx14  Colonic mucosa with no pathology 
dx14  No significant inflammation or other findings 
dx15  POLYPOID COLONIC MUCOSA 
dx15  Prominent lymphoid aggregate 
dx15  Lymphoid aggregate 
dx15  Large intestinal mucosa slightly polypoid with lymphoid aggregates 
dx15  Mucosa with lymphofollicular hyperplasia 
dx15  Lymphoid follicle 
dx15  Benign lymphoid aggregate 
dx15  Mucosal germinal centre 
dx15  Lymphoid hyperplasia 
dx15  Lymphocytic aggregates 
dx16  Inflammatory polyp 
dx16  Inflammatory pseudopolyp 
dx16  Inflammatory large bowel polyp 
dx16  Inflammatorytype polyp 
dx16  Inflamed polyp 
dx17  Hamartomatous polyp 
dx18  Granulation tissue 
dx19  Active colitis 
dx19  Active proctitis 
dx19  Acute cryptitis 
dx20  Poorly preserved colonic mucosa 
dx21  Solitary rectal ulcer 
dx21  Mucosal prolapse syndrome 
dx21  Mucosal Prolapse 
dx21  Mucosal prolapselike polyp 
dx21  Mucosa with prolapse like changes 
dx22  Melanosis coli 
dx22  Pseudomelanosis coli 
dx22  Slight melanosis 
dx23  Juvenile polyp 
dx23  Juvenile type polyp 
dx23  Retension polyp 
dx24  Lipoma 
dx25  Granular cell tumour 
dx25  Granular cell tumor 
dx26  Ischemic colitis 
dx27  Leiomyoma 
dx28  Xanthoma 
dx28  Xanthoma/xanthelasma 
dx29  Hemorrhoid 
dx30  Prolapse changes 
dx31  Fecal material 
dx31  Fecal matter only 
dx31  Vegetable fibres 
dx31  Vegetable matter 
dx31  Fecal matter 
dx31  Feces 
dx31  Food material 
dx31  Polypoid vegetable resembling seed 
dx31  Degenerated meat fibres 
dx32  NEGATIVE for dysplasia 
dx32  NEGATIVE for evidence of dysplasia 
dx32  No neoplasia present 
dx32  Negative for conventional/adenomatous dysplasia 
dx32  Negative for adenomatous polyp or dysplasia 
dx32  Negative for adenoma 
dx33  Tissue not identified 
dx33  No tissue is identified 
dx33  No tissue present 
dx33  No tissue was found 
dx33  No microscopic assessment possible 
dx33  Tissue did not survive processing 
dx33  Did not survive tissue processing 
dx33  No material present after processing 
dx33  See gross 
dx33  No specimen received 
dx33  No specimen identified 
dx33  Insufficient for evaluation 
dx33  Insufficient for assessment 
dx33  Insufficient tissue for histologic assessment 
dx33  No colon tissue is observed 
dx33  Mucosa, not diagnostic 
dx34  Negative for high grade dysplasia 
dx34  Negative for highgrade dysplasia 
dx34  No evidence of high grade dysplasia 
dx34  No evidence of highgrade dysplasia 
dx34  No evidence of high dysplasia 
dx34  No definite evidence of highgrade dysplasia 
dx34  No convincing evidence of high grade dysplasia 
dx34  Without high grade dysplasia 
dx34  Without highgrade dysplasia 
dx35  NEGATIVE FOR DYSPLASIA OR MALIGNANCY 
dx35  Negative for highgrade dysplasia and malignancy 
dx35  Negative for high grade dysplasia and malignancy 
dx35  Negative for highgrade dysplasia or invasive malignancy 
dx35  Negative for high grade dysplasia or invasive malignancy 
dx35  Negative for highgrade dysplasia or invasive carcinoma 
dx35  Negative for high grade dysplasia or invasive carcinoma 
dx35  Negative for highgrade dysplasia and invasive carcinoma 
dx35  Negative for high grade dysplasia and invasive carcinoma 
dx35  No convincing evidence of high grade dysplasia or adenocarcinoma 
dx36  Cautery/crush artifact 
dx36  Cautery artifact 
dx36  Cautery artefact 
dx36  Cauterized tissue 
dx36  Cauterized colonic mucosa 
dx36  Polypoid cauterized mucosa 
dx36  Crushed fragments of large bowel 
dx37  Focal adenomatous changes 
dx37  Focal adenomatous change 
dx37  Fragments of adenoma 
dx37  Possible adenomatous change 
dx37  Adenoma 
dx37  Suspicious for Adenomatous Changes 
dx37  Adenomatous change 
dx37  Adenomatous mucosal change 
dx37  ADENOMA(S) 
dx38  Chronic inflammation 
dx38  Chronic inflammation only 
dx39  Dysplasia associated lesion or mass 
dx39  Features of DALM 
dx40  Carcinoid 
dx40  Neuroendocrine tumour 
dx40  Neuroendocrine tumor 
Appendix B (Location Codes, Search Strings and Distances)
The location code was based on the location provided in the “source of specimen” section of the report. If the location was given in centimetres from the anal verge, it was converted to named location, based on approximate measures used by NCI.
Appendix B1: Codes and location dictionary
Location code  String 

loc1  Rectal 
loc1  Rectum 
loc2  Rectosigmoid 
loc2  Rectum—sigmoid 
loc2  Rectal—sigmoid 
loc3  Sigmoid 
loc4  Descending 
loc5  Splenic 
loc6  Transverse 
loc7  Hepatic 
loc8  Ascending 
loc9  Cecum 
loc9  Cecal 
loc9  Appendiceal orifice 
loc10  Left colon 
loc11  Right colon 
loc12  Anastomosis 
loc12  Anastomotic 
Appendix B2: Codes and location table
Location code  Distance (cm) 

loc1  < 13 
loc2  < 18, ≥ 13 
loc3  < 58, ≥ 18 
loc4  < 79, ≥ 58 
loc5  < 84, ≥ 79 
loc6  < 130, ≥ 84 
loc7  < 136, ≥ 130 
loc8  < 147, ≥ 136 
loc9  < 151, ≥ 147 
Appendix C (Control charts/normalized funnel plots)
Based on the normal approximation of the binomial distribution:
$${\text{SE}} = \sqrt {\frac{{i \times (1 – i)}}{n}}$$
(1)
where:
SD = standard deviation.
i = ideal (diagnostic) rate †
n = number of specimens interpreted.
† The ideal rate in this study is approximated by the group median rate.
The healthcare provider rate (pathologist diagnostic rate) is normalized as follows:
$$N_{j} = \frac{{M_{j} – i}}{{{\text{SE}}_{j} }}$$
(2)
where:
N_{j} = healthcare provider rate for healthcare provider “j”.
M_{j} = measured rate for the healthcare provider “j”.
i = ideal (diagnostic) rate.
SD_{j} = SD for healthcare provider “j”.
Equation (2) can be substituted into Eq. (1):
$$M_{j} – N_{j} \left[ {\sqrt[i]{{\frac{(1 – i)}{n}}}} \right] + {\text{i}}$$
(3)
To normalize we presume that the “SD” is equivalent and that only “n” changes. This amounts to forming two equations from Eq. 3 and solving for the normed M_{j}. After some rearrangement one can derive a conversion equation:
$$M_{{{\text{jnormed}}}} = \left[ {M_{{{\text{jmeasured}}}} – i} \right]\frac{{\sqrt[i]{{\frac{(1 – i)}{{n_{{{\text{jnormed}}}} }}}}}}{{\sqrt[i]{{\frac{(1 – i)}{{n_{{{\text{jmeasured}}}} }}}}}} + {\text{i}}$$
(4)
where
M_{j normed} = normed (diagnostic) rate for healthcare provider “j”.
M_{j measured} = measured (diagnostic) rate for healthcare provider “j”.
n_{j normed} = normed number of specimens handled (interpreted) by healthcare provider “j”.
n_{j measured} = number of specimens handled (interpreted) by healthcare provider “j”.
i = ideal (diagnostic) rate.
Appendix D (Probability Calculation)
The probability (Y) of n and < n outliers in k samples is dependent on the probability of the individual outlier (p), and the binomial cumulative distribution function:
$${\text{Y }} = {\text{ binocdf}}\left( {{\text{n}},{\text{ k}},{\text{ p}}} \right)$$
(5)
The probability (X) of n and > n outliers in this context is the complement of ‘n1’:
$${\text{X }} = { 1} – {\text{ binocdf }}\left( {{\text{n}} – {1},{\text{ k}},{\text{ p}}} \right)$$
(6)
Appendix E: Understanding statistical process control
Overview
Statistical process control is a cyclical process that involves:

(1)
Repeated measurement

(2)
Assessment of the variation in the measurement (using statistics)

(3)
Possible adjustments in the process to ensure that future measurements fall within a prescribed range, as defined by the socalled “control lines”
Statistical process control applied to diagnostic pathology at a conceptual level
In the diagnostic pathology context—if all the following are true:

(1)
Cases are assigned randomly to pathologists from a given population

(2)
Pathologists see large numbers of a particular type of case (e.g. 200 cases)
Then:
It is likely that the diagnostic rate (of say ‘tubular adenoma’) is similar for different pathologists (e.g. Pathologist ‘A’ diagnoses 102 tubular adenoma in 200 cases, Pathologist ‘B’ diagnosies 105 tubular adenomas in 200 cases) ****
**** Statistically, it can be stated that there is range in which the diagnostic rate (number of diagnoses/total cases interpreted) will fall 95% of the time.
If pathologists have significantly different diagnostic rates:
It is likely that they interpret cases differently and it may be possible to reduce diagnostic variation, via diagnostic calibration.
Diagnostic calibration is not new
Pathologists can change their diagnostic rates over time and may do this when (1) their cases are reviewed and found discrepant, (2) they review cases of (trusted) other (more experienced or subspeciality trained) pathologists, and (3) when new diagnostic entities are discovered or diagnostic criteria revised.
If a pathologist consistently (over time) has a diagnostic rate for tubular adenoma (e.g. 20/200 = 0.2) that is outside the range expected (in relation to the diagnostic rates of all their colleagues (~ 100/200 = 0.5)), one can infer that it would be possible to “retrain” that individual (or all the other pathologists) to arrive at the same diagnostic rate.
SPC in a nutshell is: systematically looking at the data (with statistics) to calibrate a process; in anatomical pathology it would be a process to look at diagnostic rates and feed those diagnostic rates back to the pathologists—such that they can adjust to a target rate/find agreement on what the target rate should be.
Statistical process control is predicated on two conditions
Condition (1) the process that one wants to control is stable over time (such that it is possible to predict the future) or can be made stable.
Condition (2) there is an ability to adjust the control variable * in a meaningful way—in relation to the control lines **.
* a control variable is: a parameter that one wants to control, e.g. the diagnostic rate of ‘sessile serrated adenoma’.
** control lines in SPC are determined by the “expected” statistical variation—when the process is in control/running optimally, e.g. the control parameter falls within a given range 95% of the times. Control limits are confidence intervals and are directly analogous to the funnel lines on funnel plots.
“Ability to adjust the control variable in a meaningful way” implies the following:

(1)
the variation (between providers) one expects to see due to chance is smaller than the (actual) variation that is observed; this implies that improvement is possible ***

(2)
an intervention can change the control variable in a substantive way, such that the variation is reduced
*** If the variation is less than the variation by chance the process is in control (or one may need a larger sample size).
The conditions for statistical process control and the objective of the manuscript
‘Condition 1’ for SPC is met if there is diagnostic stability.
‘Condition 2’ for SPC is met if there is significant diagnostic variation—that is stable (e.g. one pathologist is a consistent outlier in relation to the median diagnostic rate *****), and it is assumed that pathologists want to improve their practice/can be encouraged to make positive changes (see section “Diagnostic Calibration is Not New”).
‘Condition 1’ and ‘Condition 2’ are sufficient to infer that SPC should be feasible and could be used to improve care.
***** It should be noted that: the ‘median diagnostic rate’ may not be the ideal diagnostic rate for a given population. It is possible that an ‘outlier’ pathologist represents the ideal diagnostic rate.
In SPC, one talks of variation due to an “assignable cause” [a modifiable factor] and “common cause” [unmodifiable factors]. In the language of SPC, the question succinctly is: Is the pathologist an assignable cause?
If diagnostic rates are stable [Condition 1], and the pathologist is an “assignable cause” [Condition 2], SPC should be feasible.