In many machine learning projects, simulated data is used to test algorithms under different conditions. However, simulated datasets may differ from real-world data due to the simplifications inherent in simulations. Understanding how realistic simulated data is compared to experimental or observational data is essential for ensuring that the results of benchmarks and model evaluations are applicable beyond synthetic settings.
The focus of this project will be on developing and comparing methods to assess the similarity between simulated and real-world data distributions. The simulated data will be generated by different machine learning algorithms or probabilistic graphical models. The aim is to systematically evaluate how well these models reproduce key statistical and structural properties of the original data.
The project should include a review of existing metrics and visualization techniques to compare data distributions (distance measures, feature-based comparisons, dimensionality reduction, and visualizations), a framework for comparing simulated and experimental data based on selected metrics and techniques, and a practical comparison for a few real-world datasets.