Background: The optimal strategy for stroke prevention in cancer patients is unknown. We compared the underlying mechanisms of coagulopathy and the effects of anticoagulants in patients with active cancer and atrial fibrillation (AF). Methods: We retrospectively enrolled 46 consecutive patients with embolic stroke of unknown source and active cancer (cancer stroke). We consecutively screened patients with cancer patients without stroke (n = 29), AF stroke (n = 52), and healthy subjects (n = 28), which served as controls. Patients with cancer stroke were treated with either enoxaparin (a low-molecular-weight heparin) or a factor Xa inhibitor, and those with AF stroke were treated with factor Xa inhibitors. D-dimer, factor Xa, and circulating cell-free DNA (cfDNA), a marker of neutrophil extracellular traposis, were measured at both before and after anticoagulation. Results: In AF stroke, factor Xa activity and cfDNA and D-dimer levels were decreased by treatment with factor Xa inhibitors. In contrast, in cancer stroke, factor Xa activity was decreased, D-dimer levels were unchanged, and cfDNA levels were increased by treatment with factor Xa inhibitors. In cancer stroke patients treated with enoxaparin, D-dimer levels were decreased (p = 0.011) and cfDNA levels were unchanged. Conclusion: The anticoagulation effects of factor Xa inhibitors differed between cancer stroke and AF stroke.