Electronic Health Records (EHR) promise improvement for patient care and also offer great value for biomedical research including clinical, public health, and health services research. Unfortunately, the full potential of EHR big data research has remained largely unrealized. The purpose of this study was to identify rate limiting factors, and develop recommendations to better balance unrestricted extramural EHR access with legitimate safeguarding of EHR data in retrospective research. By exploring primary, secondary, and tertiary sources, this review identifies external constraints and provides a comparative analysis of social influencers in retrospective EHR-based research. Results indicate that EHRs have the advantage of reflecting the reality of patient care but also show a frequency of between 4.3-86% of incomplete and inaccurate data in various fields. The rapid spread of alternative analytics for health data challenges traditional interpretations of confidentiality protections. A confusing multiplicity of controls creates barriers to big data EHR research. More research on the use of EHR big data is likely to improve accuracy and validity. Information governance and research approval processes should be simplified. Comprehensive regulatory policies that do not exclusively cover health care entities, are needed. Finally, new computing safeguards are needed to address public concerns, like research access only to aggregate data and not to individually identifiable information.