Measuring physicians' judgment - The use of clinical data by Australian rheumatologists

Most therapeutic decisions depend upon the clinical judgment of physicians assessing their patients. However the inherent and wide variation in such judgments is usually ignored. Rheumatoid arthritis typifies those diseases in which much information is available on which to base decisions, but little is known about how physicians combine the data to evaluate their patients' response to treatment. Thirty-four Australian rheumatologists recorded their assessments of the progress of 50 rheumatoid patients treated with 'second line' agents, based on data presented on previously validated written forms. Clinical judgment analysis, a form of multiple regression analysis, was then used to model the way physicians' judgments related to the available data. There were major differences of judgment in the assessments of response to therapy. This was so even when only 'clinically important' changes were identified. The variance in judgments which could be modelled by clinical judgment analysis ranged from 45% to 94%. Both individual inconsistency and differences in the underlying use of data contributed to disagreements between clinicians' assessments of identical cases. Identifying underlying differences in the way clinical data relate to clinicians' judgments is a step towards improving clinical consistency.