From Legacy Systems to Scalable Cloud Platforms: Building Modern Data Pipelines with Data Engineering Strategies, Scaling Trust, Compliance, and Performance in Public Health

Public health data motivates impactful decisions. Healthcare centers often process data using legacy systems. These are monolithic with silos and lack flexibility. Latency in responses is a common side effect of these models. Shifting into scalable cloud-supported data platforms allows real-time analytics and increases trust by ascertaining regulatory compliance, thereby increasing performance. The paper studies key data engineering strategies such as modular ETL dataflows. Robust data modeling, metadata-supported design, automatic quality manifestation, and governance architectures. The functionalities are proposed to modernize citizen health data modeling and management frameworks. This study outlines regarding use of cloud native tools and architecture trends to support delivering scalable and resilient outcomes. Meticulous data privacy management, security, and uptime needs are addressed. Finally, strategies to develop stakeholder trust and regulatory compliance are discussed in dynamic data environments.