AbstractObjectiveThe Interagency Integrated Triage Tool (IITT) is a novel, three‐tier triage system recommended by the World Health Organization. The present study sought to assess the validity and reliability of a pilot version of the tool in a resource‐limited ED in regional Papua New Guinea.MethodsThis pragmatic prospective observational study, conducted at Mount Hagen Provincial Hospital, commenced 1 month after IITT implementation. The facility did not have a pre‐existing triage system. All ED patients presenting within a 5‐month period were included. The primary outcome was sensitivity for the detection of time‐critical illness, defined by 10 pre‐specified diagnoses. The association between triage category and ED outcomes was examined using Cramer's V correlation coefficient. Reliability was assessed by inter‐rater agreement between a local and an experienced external triage officer.ResultsThere were 9437 presentations during the study period and 9175 (97.2%) had a triage category recorded. Overall, 138 (1.5%) were classified as category 1 (emergency), 1438 (15.7%) as category 2 (priority) and 7599 (82.8%) as category 3 (non‐urgent). When applied by a mix of community health workers, nurses, health extension officers and doctors, the tool's sensitivity for the detection of time‐critical illness was 77.8% (95% confidence interval 64.4–88.0). The admission rate was 14.5% (20/138) among emergency patients, 12.0% (173/1438) among priority patients and 0.4% (30/7599) among non‐urgent patients (P = 0.00). Death in the ED occurred in 13 (9.4%) of 138 emergency patients, 34 (2.4%) of 1438 priority patients and four (0.1%) of 7599 non‐urgent patients (P = 0.00). The negative predictive value for these outcomes was >99.5%. Among 170 observed triage assessments, weighted κ was 0.81 (excellent agreement). On average, it took clinicians 2 min 43 s (standard deviation 1:10) to complete a triage assessment.ConclusionThere is limited published data regarding the predictive validity and inter‐rater reliability of the IITT. In this pragmatic study, the pilot version of the tool demonstrated adequate performance. Evaluation in other emergency care settings is recommended.