Purpose: Rectal cancer patients who present with peritumoral abscesses and fistulas at the time of diagnosis may be denied chemoradiotherapy (CRT) as the safety is unknown. The aim of this study was to investigate the safety of preoperative CRT in this patient group. Methods: We performed a retrospective nested case–control study to compare outcomes between patients with locally advanced rectal cancer with peritumoral abscesses and fistulas (study group) and patients with T4 locally advanced rectal cancer with no evidence of abscesses and fistulas (control group). These groups were matched by treatment center and radiotherapy delivery method. All patients received 50–54 Gy of conventionally fractionated RT with concurrent chemotherapy. Primary endpoint was grade 3–5 toxicity (by National Cancer Institute Common Toxicity Criteria for Adverse Events (NCI CTCAE). Secondary endpoints included postoperative morbidity, pathologic complete response (pCR), disease-free survival (DFS), and overall survival (OS) at 2 years. Results: A total of 33 patients were included in each group. Grade 3 toxicity was observed in 2 (6.1%) patients in the study group and 4 (12.1%) patients in the control group (p = 0.672). No patients developed grade 4–5 toxicity. Grade 3–4 Clavien–Dindo complications were observed in 5 (15.2%) patients in the study group and in 6 (18.2%) patients in the control group (p = 1.0). Pathologic CR was achieved in 3 (9.1%) and 5 (15.2%) patients, respectively (p = 0.708). Two-year OS was 78.3% vs. 81.8% (p = 0.944), 2‑year DFS was 62.8% vs. 69.7% (p = 0.693), respectively. Conclusion: The presence of peritumoral abscesses and fistulas in patients with locally advanced rectal cancer is not associated with increased toxicity or inferior clinical outcomes after preoperative CRT. © 2021, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany.