We Need Metrics to Assess How Useful Our AI Systems Are
2 MIN READ
By RANI URBIS
www.nordis.net
Recently, Philstar published an article about Packworks, a Philippine tech company, and its AI-powered app for sari-sari stores.
Packworks reported that sari-sari stores participating in its Store Insighting Project saw improved sales. The company said stores using the tool had a 79 percent increase in median sales of their top 50 products, a 29 percent increase in median total sales, and a 20 percent increase in median transactions.
The figures are massive, which, based on its metrics, makes it a highly efficient tool. This shows how important for us to measure how efficient AI systems are. Numbers don’t lie. They tell us if an AI tool does its job and makes our lives better.
So, as the country is again facing difficult climate and disaster conditions, it would be useful to look at AI tools in weather forecasting and disaster mitigation. In some areas, people are dealing with very high heat index levels. In others, heavy rains and bad weather remain a problem. The recent 7.8-magnitude earthquake that affected General Santos and nearby areas also reminds us how much we need more effective disaster preparedness.
Last February, PAGASA shared information about its newer AI-related disaster and weather projects, including the GaBAI Project and AI-SWaMP.
These tools aim to deliver clearer technical improvements, especially in speed, forecast range, and local detail. Public reports say the goal is to extend useful forecasts from the usual 2 to 5 days to about 14 days, giving agencies and communities more time to prepare. The tools also aim to make forecasts more localized, improving forecast detail from about 3 kilometers to 2 kilometers, so warnings can better reflect conditions in specific towns or areas instead of only broad regional forecasts. Another reported improvement is faster processing: AI models may reduce weather model runtime from around 3 hours to about 15 minutes, allowing updates to come more quickly as weather changes.
I am more than curious about how these AI tools have improved forecasting and disaster mitigation.
PAGASA and other disaster agencies should also make it easier for the public to check progress. We need the metrics and performance evaluation of these AI projects. If these tools are meant to improve speed, range, local detail, and disaster response, then agencies should release clear updates showing which targets have already been reached, which are still being tested, and which areas are already benefiting from them.
It would also help to see practical results, not just technical claims: Were warnings issued earlier? Did local governments respond faster? Were communities able to protect lives, crops, property, or livelihood because the information arrived in time? Without those public measures, it is hard to know whether AI is already improving disaster preparation or is still mainly in the promising stage.
The metrics should clearly tell us whether we have the right AI tools in place or need better, more efficient ones.# nordis.net
