Researchers at The University of Texas at Dallas have used automated machine learning in a new way to forecast state violence in Africa, and they expect the technology to have even wider predictive applications.
Dr. Vito D’Orazio, associate professor of political science in the School of Economic, Political and Policy Sciences, and his team created the dynamic forecasting model as part of a competition sponsored by the Violence Early-Warning System (ViEWS) project at Uppsala University’s Department of Peace and Conflict Research. The research was subsequently published online Jan. 15 in the journal International Interactions.
The ViEWS contest challenged competitors to forecast — up to six months out — the change in the number of fatalities in a country or region stemming from state-based violence, which is armed conflict in which at least one party is a government.
Forecasts from the UT Dallas team were so accurate on the subnational level — consisting of randomly gridded map areas that don’t take countries’ borders into account — that they won that part of the competition for predictive accuracy and split the win for originality.
The UT Dallas team approached the problem using a variation of machine learning (ML). In standard ML, an algorithm analyzes data to learn patterns and ultimately make decisions on new data with minimal human input. But ML still requires a data scientist to select which algorithm to use, among other decisions.
In contrast, automated machine learning (autoML) solves the problem using many different machine learning algorithms and automates other decisions as well. This reduces the need for human involvement during the model selection process. Yu Lin, a computer science PhD student at UT Dallas, worked with D’Orazio on the project and said he was pleasantly surprised by the accuracy of the results using autoML.
“I thought some of the more sophisticated ML models in the competition would lead to better results, but our autoML model won,” he said.
One pitfall of using autoML for this type of violence-prediction analysis, D’Orazio said, is the danger of “overfitting,” which happens when a computer algorithm learns the training dataset too well.
D’Orazio said his team mitigated that problem by using only a very limited amount of the data provided by ViEWS, deliberately leaving out information they did not think would improve the accuracy of their predictions, such as population numbers and descriptions of geographic terrain in the given regions.
“The data that we used are all data about conflict itself, because the best predictor of future violence is current violence,” he said. “If you have 12 months of civil war, the first month is the only time the violence came out of nowhere. The following 11 months had violent months preceding them.”
D’Orazio said this promising autoML method could have broad applications, including forecasting state violence in other geographic areas outside of Africa.
“Policymakers want to know what to expect from a situation. Is an area going to be violence-prone? Will current violence de-escalate?” D’Orazio said.
Businesses interested in investing in certain regions also could use such tools to assess the risk of political violence. This method could even extend to predictions beyond state-based violence, he said.
“The data that we used are all data about conflict itself, because the best predictor of future violence is current violence.”
Dr. Vito D’Orazio, associate professor of political science in the School of Economic, Political and Policy Sciences
“AutoML has applications anywhere machine learning and artificial intelligence are used,” he said.
For policymaking purposes, the technology could be used in anticipatory action frameworks to predict refugee numbers after natural or human-made disasters, or for forecasting food shortages.
The next step is to refine the accuracy of the predictions.
“I want to look at when these forecasts are reliable and when they’re not,” D’Orazio said. “For example, the forecast accuracy was better in Nigeria than Libya. Understanding why can help us to improve the model for future applications, and hopefully to improve our understanding of armed violence.”
The UT Dallas research was supported by the Defense Advanced Research Projects Agency.