Some BTC types respond to pembrolizumab, but there are no known prognostic factors to predict its treatment benefits. In this study, we attempted to identify the prognostic factors associated with pembrolizumab as a second-line treatment for gemcitabine-refractory BTC. This retrospective and single tertiary-center study involved all the consecutive patients (n = 80) with refractory advanced BTC, who were diagnosed as programmed cell death ligand 1-positive and treated with pembrolizumab between August 2017 and February 2021. The overall survival (OS) was analyzed using Cox regression analysis. The median OS was 6.0 months [95% confidence interval (CI): 3.87–8.20]; median progression-free survival was 1.9 months (95% CI: 1.82–1.98); and the response rate was 15.9%. In the multivariate Cox regression analysis, the TB [adjusted hazard ratio (HR) = 2.286; 95% CI: 1.177–4.440; p = 0.015), albumin levels (adjusted HR = 0.392; 95% CI: 0.211–0.725; p = 0.003), ALP levels (adjusted HR = 1.938; 95% CI: 1.105–3.400; p = 0.021), and LMR (adjusted HR = 0.325; 95% CI: 0.173–0.609; p < 0.001) were identified as significant variables associated with the OS. High albumin levels and LMR and low ALP levels and TB were significantly associated with better OS in patients treated with pembrolizumab.