Abstract Background Non-steroidal anti-inflammatory drugs (NSAIDs), the most widely prescribed drugs in the world, can cause gastrointestinal damage, including colitis. However, the prevalence of NSAID-induced colitis is unknown because the disease is often asymptomatic. Case presentation We report the case of a 64-year-old female patient with a history of long-term NSAID use, who was hospitalized with septic shock caused by Klebsiella pneumoniae bacteremia. Computed tomography revealed multiple renal and splenic abscesses with diffuse colon wall thickening. A colonoscopy confirmed colitis with diffuse ulcers. NSAIDs were discontinued after this hospitalization. The abscesses improved after antibiotic treatment. A short course of balsalazide treatment was given under the suspicion of ulcerative colitis. Balsalazide was discontinued four months later due to a non-compatible clinical course. A follow-up colonoscopy two years later revealed a normal colon mucosa, and NSAID-induced colitis was diagnosed. Conclusion This is the first reported case of combined bacterial splenic and renal abscesses without intestinal manifestations as the initial presentation of NSAID-induced colitis. In contrast to cases of K. pneumoniae bacteremia with primary liver abscesses in patients with diabetes mellitus in Taiwan, we presented the first case with abscesses caused by community-acquired K. pneumoniae in the kidneys and spleen without liver invasion. In conclusion, our case report alerts clinicians to the possibility that K. pneumoniae bacteremia combined with multiple abscesses can be associated with severe NSAID-induced colitis.