Objectives: We investigated the extent to which Brazilian and Portuguese Men Who Have Sex with Men (MSM) had casual sex partners outside their homes during the period of sheltering in place for the COVID-19 pandemic. Methods: An online survey was conducted in Brazil and Portugal in April, during the period of social isolation for COVID-19, with a sample of 2361 MSMs. Recruitment was done through meeting apps and Facebook. Results: Most of the sample (53.0%) had casual sex partners during sheltering. Factors that increased the odds of engaging in casual sex in Brazil were having group sex (aOR 2.1, 95% CI 1.3–3.4), living in an urban area (aOR 1.6, 95% CI 1.1–2.2), feeling that sheltering had a high impact on daily life (aOR 3.0, 95% CI 1.1–8.3), having casual instead of steady partners (aOR 2.5, 95% CI 1.8–3.5), and not decreasing the number of partners (aOR 6.5, 95% CI 4.2–10.0). In Portugal, the odds of engaging in casual sex increased with using Facebook to find partners (aOR 4.6, 95% CI 3.0–7.2), not decreasing the number of partners (aOR 3.8, 95% CI 2.9–5.9), usually finding partners in physical venues (pre-COVID-19) (aOR 5.4, 95% CI 3.2–8.9), feeling that the isolation had a high impact on daily life (aOR 3.0, 95% CI 1.3–6.7), and HIV-positive serostatus (aOR 11.7, 95% CI 4.7–29.2). Taking PrEP/Truvada to prevent COVID-19 was reported by 12.7% of MSM. Conclusions: The pandemic has not stopped most of our MSM sample from finding sexual partners, with high-risk sexual behaviors continuing.