MAT
Software to analyze and validate
scientific theories and models using real-world data
Model Analyst's Toolkit (MAT)
One of the most powerful things scientists can do to help analyze and validate scientific theories is to create models that effectively describe the world around us. Models help scientists organize their theories and suggest additional experiments to run. Validated models also help others in more practical applications. For instance, in the hands of military decision makers, human social cultural behavior (HSCB) models can help predict instability and the socio-political effects of missions, whereas models of the human brain and mind can help educators and trainers create curricula that more effectively improve the knowledge, skills, and abilities of their pupils.
Although the scientific community uses various software tools to help develop and analyze models, they are either too general or too focused, making them unsuitable for rapid model exploration or for use by non-computational scientists. These tools also largely ignore the problem of validating the models.
Screenshots from the MAT data visualization and exploration tool
The Charles River Analytics Solution
Under an effort for the Office for Naval Research (ONR), Charles River Analytics developed the Model Analyst’s Toolkit (MAT) to specifically address the shortcomings of existing software tools. MAT is a software application for analyzing and validating scientific theories and models using real-world data. MAT has also been used on a variety of research programs to help analyze and extract meaning from complex data. A few examples are:
- Evaluating sensor data from medical manikins and laparoscopic surgery training systems to support training doctors to use advanced medical devices
- Evaluating physiological data from training systems to identify periods of trainee stress and excessive workload
- Identifying relationships in social/political/economic data gathered by the World Bank
MAT enables social scientists to visualize and explore data, develop causal models, and validate those models against available data, as displayed in the figure below. MAT is particularly useful for analyzing and validating scientific theories and models of phenomena that happen over time, such as changes in unemployment, changes in crime rates, changes in financial markets, or changes in physiological measures like heart rate. It combines cutting-edge automated data analysis capabilities with a graphical user interface that together help scientists understand and explore their data and how their models do and do not match that data. It is natural and intuitive to use, even for users without mathematical or computational training. It can be used independently specific computational modeling frameworks. Specifically, scientists can use MAT to:
- Analyze many types of important scientific models
- Support easy data exploration
- Validate models against data for independent validation of models
The Benefit
Using MAT, scientists create qualitative causal/predictive models, which are constructed graphically, as seen in the figure below. Analysts then associate data with the model’s elements and can run an automatic validation process that informs the analyst to what extent the model matches the data. Hypothetical data or future outcomes are analyzed the same way. The resulting analysis can then be used to improve the model or to better understand when the model can be used reliably. MAT will even recommend fixes to models.
MAT is a powerful tool for scientists who are working to understand the world in a systematic way. It can be used to validate existing scientific theories and help to create new ones. MAT has been used to analyze a variety of scientific models, including social science models, neurophysiological models, and financial models. It provides deep understanding of model results, creating better explanations of why a model did what it did, uncovering invalid model assumptions or results, and providing greater trust of validated models.
Additional benefits:
- MAT is natural and intuitive to use, even for users without mathematical or computational training
- MAT can be used to analyze many types of important scientific models
- MAT uses data mining and machine learning to recommend improvements to user-defined models
- MAT is independent of any specific computational modeling framework
- MAT supports easy data exploration
- MAT can be used to validate models against data for independent validation of models
MAT provides deep understanding of model results, creating better explanations of why a model did what it did, uncovering invalid model assumptions or results, and providing greater trust of validated models.
MAT’s causal/predictive model editor
Beta Version
We are looking for people who are interested in trying out a free beta version of the MAT software. We will provide the software and as much support as we can to help you use MAT for yourself. In return, we ask only that you tell how we can improve mat.
If you are an academic or U.S. government researcher who is interested in getting a copy of MAT, please email us at: mat_project (@) cra.com
Screenshots from the MAT data visualization and exploration tool
Contact us to learn more about MAT and our other capabilities.
This material is based upon work supported by the United States Navy, under Contract No. N00014-12-C-0653. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Navy.
