OBJECTIVE:: An independent cohort of patients with acute lung injury was used to evaluate the external validity of a simple prediction model for short-term mortality previously developed using data from ARDS Network (ARDSNet) trials. DESIGN, SETTING, AND PATIENTS:: Data for external validation were obtained from a prospective cohort study of patients with acute lung injury from 13 intensive care units at four teaching hospitals in Baltimore, MD. INTERVENTIONS:: None. MEASUREMENTS AND MAIN RESULTS:: Of the 508 nontrauma, patients with acute lung injury eligible for this analysis, 234 (46%) died inhospital. Discrimination of the ARDSNet prediction model for inhospital mortality, evaluated by the area under the receiver operator characteristic curves, was 0.67 for our external validation data set vs. 0.70 and 0.68 using Acute Physiology and Chronic Health Evaluation II and the ARDSNet validation data set, respectively. In evaluating calibration of the model, predicted vs. observed inhospital mortality for the external validation data set was similar for both low-risk (ARDSNet model score = 0) and high-risk (score = 3 or 4+) patient strata. However, for intermediate-risk (score = 1 or 2) patients, observed inhospital mortality was substantially higher than predicted mortality (25.3% vs. 16.5% and 40.6% vs. 31.0% for score = 1 and 2, respectively). Sensitivity analyses limiting our external validation data set to only those patients meeting the ARDSNet trial eligibility criteria and to those who received mechanical ventilation in compliance with the ARDSNet ventilation protocol did not substantially change the model's discrimination or improve its calibration. CONCLUSIONS:: Evaluation of the ARDSNet prediction model using an external acute lung injury cohort demonstrated similar discrimination of the model as was observed with the ARDSNet validation data set. However, there were substantial differences in observed vs. predicted mortality among intermediate-risk patients with acute lung injury. The ARDSNet model provided reasonable, but imprecise, estimates of predicted mortality when applied to our external validation cohort of patients with acute lung injury.