Background: Atrial fibrillation (AF) is common in heart failure with mildly reduced or preserved ejection fraction (HFmrEF/HFpEF) and has a negative impact on outcome. Reliable data on prevalence, incidence, and detection of AF from contemporary, prospective HFmrEF/HFpEF studies are scarce. Methods: This was a prespecified sub-analysis from a prospective, multicenter study. Patients with HFmrEF/HFpEF underwent 12-lead electrocardiography (ECG), 24 h Holter monitoring, and received an implantable loop recorder (ILR) at the study start. During the 2 year follow-up, rhythm monitoring was performed via ILR, yearly ECG, and two yearly 24 h Holter monitors. Results: A total of 113 patients were included (mean age 73 ± 8 years, 75% HFpEF). At baseline, 70 patients (62%) had a diagnosis of AF: 21 paroxysmal, 18 persistent, and 31 permanent AF. At study start, 45 patients were in AF. Of the 43 patients without a history of AF, 19 developed incident AF during a median follow-up of 23 [15–25] months (44%; incidence rate 27.1 (95% confidence interval 16.3–42.4) per 100 person-years). Thus, after the 2-year follow-up, 89 patients (79%) had a diagnosis of AF. In 11/19 incident AF cases (i.e., 58%), AF was solely detected on the ILR. Yearly 12-lead ECG detected six incident AF cases and four of these cases were also detected on two yearly 24 h Holter monitors. Two incident AF cases were detected on an unplanned ECG/Holter. Conclusions: Atrial fibrillation is extremely common in heart failure with HFmrEF/HFpEF and may inform on symptom evaluation and treatment options. AF screening with an ILR had a much higher diagnostic yield than conventional modalities.