The STOIC2021 COVID-19 AI challenge: applying reusable training methodologies to private data

Published in Medical Image Analysis, 2024

Recommended citation: Boulogne, Luuk et al. "The STOIC2021 COVID-19 AI challenge: applying reusable training methodologies to private data." Medical Image Analysis. Elsevier, 2024 https://www.sciencedirect.com/science/article/pii/S1361841524001555

Challenges drive the state-of-the-art of automated medical image analysis. The quantity of public training data that they provide can limit the performance of their solutions. Public access to the training methodology for these solutions remains absent. This study implements the Type Three (T3) challenge format, which allows for training solutions on private data and guarantees reusable training methodologies. With T3, challenge organizers train a codebase provided by the participants on sequestered training data. T3 was implemented in the STOIC2021 challenge, with the goal of predicting from a computed tomography (CT) scan whether subjects had a severe COVID-19 infection, defined as intubation or death within one month. STOIC2021 consisted of a Qualification phase, where participants developed challenge solutions using 2 000 publicly available CT scans, and a Final phase, where participants submitted …