The importance of specificity in diagnostic tests for autoantibodies

Detection of autoantibodies in the blood of a patient helps to support the diagnosis of autoimmune disease. A challenging aspect is that autoantibodies often appear in a myriad of diseases, and it is important to consider this before making a diagnosis. For example, antibodies against the protein Sm could indicate that a patient may have lupus (systemic lupus erythematosus, SLE). However, only five percent to 30 percent of lupus patients have Sm antibodies ¹; thus, a negative result does not exclude the disease in question.² On the other hand, the marker is very specific for lupus; therefore, clinicians may erroneously presume that all patients with Sm antibodies have the disease.

Diagnostic ambiguity with RA

Another example could involve diagnostic markers for rheumatoid arthritis (RA). Early identification and a correct diagnosis of RA is critical. If left untreated, or undertreated, RA is associated with progressive and irreversible joint destruction leading to disability, reduction of quality of life, and increased mortality.³ In the twentieth century, rheumatoid factor was the only autoantibody used as a diagnostic marker for RA. It is present in about 70 percent to 90 percent of RA patients. However, rheumatoid factor is also positive in other diseases and even in healthy individuals.⁴

In 2000, a team led by Walther Van Venrooij in the Netherlands introduced a new marker for RA, which created an unrivalled success story in autoimmune diagnostics: antibodies against CCP (cyclic citrullinated peptide).⁵ Antibodies to CCP appear more than 10 years before the first symptoms.⁶ In older patients, serum anti-CCP is detectable well before the development of clinical symptoms of RA, while in younger patients, the detection of anti-CCP occurs closer to the time of disease onset.⁷

The important difference between anti-CCP and rheumatoid factors is the high specificity of anti-CCP for RA.^5,8 This means test selection and timing are important. There is a large variability among different anti-CCP assays, and numerical test results are not interchangeable.³ During the last 20 years, anti-CCP tests have evolved significantly,⁷ with several generations of anti-CCP assays introduced. The anti-CCP assays using the CCP2 peptide mix have repeatedly demonstrated the highest sensitivity for RA when compared at a stratified 98.5 percent specificity.⁹

Alone, RF or anti-CCP tests are not sufficient for the diagnosis of RA, because some patients have only one or the other biomarker, or lack both. However, in combination with other clinical measures, the biomarkers together provide important diagnostic and prognostic information about different RA patient populations depending on which of the biomarkers (if any) are present in patient sera.7 Combination testing with anti-CCP, RF, sIgM, and RFIgA is beneficial when excluding the diagnosis of RA, rather than testing for individual antibodies.¹⁰ In fact, combining the presence of anti-CCP antibodies and the presence of any of the individual RF isotypes in samples predating symptom onset results in a higher specificity, with increases to 99 percent and 100 percent,⁶ providing a greater sense of confidence in the diagnosis.

The use of enzyme immunoassays to detect autoantibodies is common. In enzyme immunoassays, the antigen (the target of the antibodies) is coated to the wall of a small cavity. The antibody in the patient‘s blood binds to this antigen. The quality of the assay is tied directly to the purity of the antigen used. When it is not purified thoroughly enough, contaminants may bind to the cavity, resulting in antibodies against those contaminants causing a false positive result. With that in mind, consider the assay attribute of specificity and its importance particularly in low prevalence disease.

Specificity and accuracy

A laboratory’s decision about test selection plays a critical role in the accurate detection of antibodies. Using anti-CCP as an example, since there are assays available that use different antigens, the specificity for rheumatoid arthritis can differ among these tests.⁸

Small differences in assay specificity make a large difference in the accuracy of a result, mostly due to the very low prevalence of autoimmune diseases in general. The specificity of a test is the proportion of unaffected individuals with a negative test. The sensitivity of a test is the proportion of affected individuals with a positive test.² In a setting of relatively high clinical suspicion (high pre-test probability), a positive anti-CCP result means the patient has a strong likelihood of having or developing RA.¹¹ The pre-test probability in a large commercial lab is particularly low, because the sera are often sent in by non- specialists who want to exclude autoimmunity in patients with unclear clinical scenarios. Since the prevalence of autoimmune disease is low in the general population, for most autoimmune markers tested in immunological labs, most samples are probably not from patients with the diseases in question. In addition, most antibodies are only positive in a subset of the patients with the disease.²

The earlier mentioned Sm example is also useful to elaborate on with regard to this topic. Natural Sm protein is complex and consists of six single proteins (Sm B, B‘, D1/D2/D3, E, F and G). Anti-Sm antibodies react with the double protein Sm-B’/B and SmD but rarely with E, F, and G.1 However, only the presence of anti-D can be regarded as characteristic for an anti-Sm serum, as antibodies against Sm B’/B may occur in other diseases. Hence, a good, specific Sm-test should contain only the SmD protein. If the test contains the whole Sm complex, it will deliver false positive results by being positive in patients with other autoimmune diseases.¹

Keep in mind that no assay is 100 percent sensitive and 100 percent specific. While the anti-Sm antibodies are known for being extremely specific to lupus, a diagnosis should not be made based solely on a single test result taken in isolation. Doing so could lead a clinician to form a presumptive diagnosis of lupus and prescribe a treatment for the patient for a disease which is not present, with drugs that may not help and often have devastating side effects.

Hypothetical illustrations

Let’s look at two hypothetical examples that demonstrate this concept. If a lab evaluates 10,000 sera per year for the presence of Sm antibodies, there actually might be fewer than 400 sera from patients with lupus. A test with an average sensitivity of 22 percent and an average specificity of 98 percent will correctly find 88 lupus sera to be positive. Alternately, false positive results will occur in two percent of the 9,600 non-lupus-patients. Therefore, a total of 88 correct positives occur with 192 false positives, and less than a third of the positive results are really from lupus patients (Table 1). If the test has the same sensitivity, but a 99 percent specificity (one percent more than in the first example), only 96 non-lupus patients will falsely be identified as positive. This means about half of the positive results are correctly identified as positive (Table 2) due to the improved specificity of the assay, instead of a third of the patients. On the other hand, an increase in sensitivity of one percent, from 22 percent to 23 percent, results in only an additional four sera being classified as positive (92 instead of 88), yielding very little change in the number of correct versus false positive results. (Table 3)

Table 2. The same lab uses another anti-Sm test, which has the same sensitivity (22 percent) but a higher specificity of 99 percent. The positive predictive value increases to 48 percent.

Why does this matter? Due to low prevalence, the pretest probabilities of disease for almost all autoantibodies are very low. A difference in specificity of two percent, as demonstrated in a study by Van Hoofels et al, causes a marked impact on whether sera are classified correctly as positive versus false positive results.³ This means that a slightly lower specificity can lead to a wrong diagnosis in many patients.

Specificity is a critical metric in assays associated with autoimmune disease. Unfortunately, during a typical assay comparison performed in many laboratories, the specificity for each assay in review is not usually assessed. Without knowing the clinical characteristics and histories of the patients and samples, a laboratory can only determine whether the two tests in the evaluation deliver comparable results. In addition, in the rare cases where the clinical diagnosis is known for samples, often too few sera from controls are available to form a statistically valid conclusion about the specificity of each assay being evaluated.

Expanding validation testing

A consequence of the difficulty in determining specificity is the underestimation of its importance. This is mirrored and exemplified in the United States in proficiency testing (PT) programs where usually only the sensitivity of a test method is judged. Comparable to the evaluations in a clinical laboratory, the providers of sera for quality assessment programs often do not have any information about the diagnosis of the serum donors. This limitation may be the reason behind the need to rely upon a consensus, where the results reported by the dominant method used by participants (the majority) influence whether a sample is deemed negative or positive. The market leader and respectively the market-dominant method therefore essentially determine whether a test result is considered correct or not. The question to ponder is whether this inadvertently causes innovative tests, which are markedly different from the majority, to be underutilized, even when they have a higher diagnostic value as it relates to disease. Usage of clinically characterized samples in PT programs would be a positive step forward in how these programs are designed and allow for not only sensitivity to be assessed, but, importantly, specificity of assays as well.

In summary, the specificity in diagnostic tests for autoantibodies matters a great deal, and laboratories considering the adoption of an assay would benefit greatly from expanding the validation testing to include clinically characterized samples; the same can be said for PT programs.

References

1. Satoh M, Fitzler MJ, Chan EKL. Autoantibodies. 3rd ed.; Shoenfeld, Meroni, Gershwin (eds) Elsevier; 2014.151-159.
2. Birtane M, Yavuz S, Tastekin N. Laboratory evaluation in rheumatic diseases. World J Methodol. 2017; 7(1): 1-8.
3. Van Hoofels L, Jacobs J, Vander Cruyssen B, et al. Performance characteristics of rheumatoid factor and anti-cyclic citrullinated peptide antibody assays may impact ACR/EULAR classification of rheumatoid arthritis. Ann Rheum Dis. 2018;0:1–11. doi:10.1136/annrheumdis-2017-212365.
4. Ingegnoli F, Castelli R, Gualtierotti R. Rheumatoid factors: clinical applications. Disease Markers. 2013;35(6):727-734.
5. Schellekens GA, Visser H, de Jong BAW, et al. The diagnostic properties of rheumatoid arthritis antibodies recognizing a cyclic citrullinated peptide. Arthritis Rheumatism. 2000;43(1):155-163.
6. Rantapäa-Dahlqvist S, de Jong BAW, Berglin E, et al. Anti-cyclic citrullinated peptide assays (ACPA) and IgA rheumatoid factor predict the development of rheumatoid arthritis. Arthritis Rheum. 2003;48(10):2741–2749.
7. Taylor P, Gartemann J, Hsieh J, et al. A systematic review of serum biomarkers anti-cyclic citrullinated peptide and rheumatoid factor as tests for rheumatoid arthritis. Autoimmune Diseases. 2011;2011:815038. doi:10.4061/2011/815038.
8. Pruijn GJM, Wiik A, Van Venrooij WJ. The use of citrullinated peptides and proteins for the diagnosis of rheumatoid arthritis. Arthritis Res Ther. 2010;12:203. https://arthritis-research.biomedcentral.com/articles/10.1186/ar2903
9. Bizzaro N, Tonutti E, Tozzoli R, et al. Analytical and diagnostic characteristics of 11 2nd- and 3rd-generation immunoenzymatic methods for the detection of antibodies to citrullinated proteins. Clin Chem. 2007.53:8,1527–1533.
10. Castro C, Gourley M. Diagnostic testing and interpretation of tests for autoimmunity. J
    Allergy Clin Immunol. 2010 February; 125(2 Suppl 2): S238–S247. doi:10.1016/j.jaci.2009.09.041.
11. Aggarwal R, Liao K, Nair R, Ringold S, Costenbander KH. Anti-citrullinated peptide antibody (ACPA) assays and their role in dagnosis of rheumatoid arthritis. Arthritis Rheum. 2009;61(11):1472–1483. doi:10.1002/art.24827.