Abstract Background Microglial activation-mediated neuroinflammation is one of the essential pathogenic mechanisms of sepsis-associated encephalopathy (SAE). Mounting evidence suggests that high mobility group box-1 protein (HMGB1) plays a pivotal role in neuroinflammation and SAE, yet the mechanism by which HMGB1 induces cognitive impairment in SAE remains unclear. Therefore, this study aimed to investigate the mechanism of HMGB1 underlying cognitive impairment in SAE. Methods An SAE model was established by cecal ligation and puncture (CLP); animals in the sham group underwent cecum exposure alone without ligation and perforation. Mice in the inflachromene (ICM) group were continuously injected with ICM intraperitoneally at a daily dose of 10 mg/kg for 9 days starting 1 h before the CLP operation. The open field, novel object recognition, and Y maze tests were performed on days 14–18 after surgery to assess locomotor activity and cognitive function. HMGB1 secretion, the state of microglia, and neuronal activity were measured by immunofluorescence. Golgi staining was performed to detect changes in neuronal morphology and dendritic spine density. In vitro electrophysiology was performed to detect changes in long-term potentiation (LTP) in the CA1 of the hippocampus. In vivo electrophysiology was performed to detect the changes in neural oscillation of the hippocampus. Results CLP-induced cognitive impairment was accompanied by increased HMGB1 secretion and microglial activation. The phagocytic capacity of microglia was enhanced, resulting in aberrant pruning of excitatory synapses in the hippocampus. The loss of excitatory synapses reduced neuronal activity, impaired LTP, and decreased theta oscillation in the hippocampus. Inhibiting HMGB1 secretion by ICM treatment reversed these changes. Conclusions HMGB1 induces microglial activation, aberrant synaptic pruning, and neuron dysfunction in an animal model of SAE, leading to cognitive impairment. These results suggest that HMGB1 might be a target for SAE treatment.