Early life adversity (ELA) describes stressful experiences that may increase risk for psychopathology and impact emotion regulation and executive functioning systems. The influence of ELA on the development of empathy—the ability to understand and resonate with others’ thoughts and emotions—remains understudied, despite the fact that empathy development relies on cognitive and emotional abilities often affected by ELA. This scoping review summarized 43 empirical articles on ELA and empathy to clarify the muddled literature and address limitations to inform future research. Across various operationalizations of ELA and empathy, 15 articles suggested that ELA was associated with increased empathy, 19 that ELA was associated with decreased empathy, and 12 pointed to a null association. ELA and empathy showed differing associations across developmental periods, with ELA being more related to higher affective empathy and lower cognitive empathy in youth and higher personal distress in adulthood. Categorization by type of adversity revealed a lack of studies on deprivation and environmental adversity, while examination of empathy operationalization revealed a need for the assessment of empathy components among youth and more task-based measures of empathy. Recommendations for future research include the need to (a) clarify operationalizations of ELA, (b) explore empathy components and naturalistic measures, and (c) focus on outcomes in adolescence. Continued efforts to understand the connection between ELA and empathy will provide valuable insight into the impact of adversity on socioemotional development and guide psychosocial interventions for individuals at risk for maladaptive outcomes following adverse childhood experiences.