To Err or Not to Err? Towards Extracting Error of Law Findings From the UK's Upper Tribunal Immigration and Asylum Chamber Decisions

We present preliminary experiments towards extracting error of law findings and outcome from second-instance judicial decisions. The overall aim of the PhD is to use ML/NLP approaches to quantify error patterns over time as found in decisions of the United Kingdom's Upper Tribunal Immigration and Asylum Chamber in order to 1) gain a better understanding of the corrective mechanism between first and second instance courts, and 2) to identify possible patterns of training needs that could usefully inform first instance judicial training. Running several simple binary classifiers, we find best performance (average ROC AUC score of 0.82) in a five-fold cross-validation k-Nearest Neighbor Model. We discuss challenges in error extraction and plans for future work.