In the
financial world, Model Risk is a type of risk that occurs when a financial
model used for any financial decision, does not perform as designed due to
various reasons, the major one being the reality being a far cry from the underlying
assumptions of the model. Model risk is a subset of operational risk, as it is
usually a failure of either the systems or the people running the systems,
might not completely understand its assumptions and limitations.
Increasing
models are now being used in a wide variety of fields, other than finance.
The following anecdote is a classic case
of Model Risk. In plain layman terms it exemplifies Blind leading Blind!
It was autumn, and the American Indians on the remote
reservation asked their new Chief if the winter was going to be cold or mild.
Since he was a new Indian Chief in a modern society, he had never been taught
the old secrets, and when he looked at the sky, he couldn't tell what the
weather was going to be. Nevertheless, to be on the safe side, he replied to
his tribe that the winter was indeed going to be cold and that the members of
the village should collect wood to be prepared. But also being a practical
leader, after several days he got an idea. He went to the phone booth, called
the National Weather Service and asked,
"Is the coming winter going to be cold?"
"It looks like this winter is going to be quite cold indeed," the meteorologist at the weather service responded.
So the Chief went back to his people and told them to collect even more wood in order to be prepared. A week later he called the National Weather Service again.
"Is it going to be a very cold winter?"
"Yes," the man at National Weather Service again replied, "it's going to be a very cold winter."
The Chief again went back to his people and ordered them to collect every scrap of wood they could find. Two weeks later he called the National Weather Service again.
"Are you absolutely sure that the winter is going to be very cold?"
"Absolutely," the man replied. "It's going to be one of the coldest winters ever."
"How can you be so sure?" the Chief asked.
The weatherman replied, "Well our weather prediction model considers a lot of factors, but over the years, due to back-testing and fine tuning we have given a 75% weight to the actions of the Indians, who know better than us. And this year although the other weather indicators are pointing to a very mild winter, the Indians are collecting wood like crazy!"
"Is the coming winter going to be cold?"
"It looks like this winter is going to be quite cold indeed," the meteorologist at the weather service responded.
So the Chief went back to his people and told them to collect even more wood in order to be prepared. A week later he called the National Weather Service again.
"Is it going to be a very cold winter?"
"Yes," the man at National Weather Service again replied, "it's going to be a very cold winter."
The Chief again went back to his people and ordered them to collect every scrap of wood they could find. Two weeks later he called the National Weather Service again.
"Are you absolutely sure that the winter is going to be very cold?"
"Absolutely," the man replied. "It's going to be one of the coldest winters ever."
"How can you be so sure?" the Chief asked.
The weatherman replied, "Well our weather prediction model considers a lot of factors, but over the years, due to back-testing and fine tuning we have given a 75% weight to the actions of the Indians, who know better than us. And this year although the other weather indicators are pointing to a very mild winter, the Indians are collecting wood like crazy!"
