Pegasystems Reaction to FCA Partnership With Turing Institute, on AI FinTech

The FCA’s recent partnership with the Alan Turing Institute to examine the transparency and explainability of AI in the financial sector is an incredibly important and timely move, particularly for the insurance market.

However, Tony Tarquini, European Insurance Director, Pegasystems, argues there are already ‘black box’ solutions on the market and it’s likely we could see some hefty fines being dished out to those organisations who don’t have the ability to explain to a consumer why a machine has denied them an insurance product or service. Could this initiative be too little, too late?

“For an insurance company, being able to demonstrate how a calculation was made is a fundamental rule. And yet we are seeing both insurtechs and legacy organisations using ‘black box’ AI to determine insurance premiums. Although they aren’t currently doing anything illegal, rules change over time. Therefore, if an insurer cannot prove an audit trail a year after a deal was made, they could be facing a financial penalty.

“The main problem with using AI as part of a decision making process is that technology will always reflect market bias. Some would even say that having a completely unbiased decision is subjective. For example, consider a wealthy town with low crime rates in an otherwise deprived borough. If an AI were to make a risk calculation of a heavily protected 5-bedroom house in said town, it would probably say it was extremely likely to be burgled, which would be incorrect. What this shows is how every decision must be made in context, which can be very difficult to teach AI.

“Also, there are acceptable and unacceptable forms of bias. Basing a decision on someone’s sex or race is against the law, however, taking someone’s age into account is considered acceptable. Therefore it’s imperative that AI is taught what is and isn’t suitable bias. Without this there is no reason why AI won’t overstep the line on social acceptability, because ‘doing something wrong’ is subjective.

“Where AI does have a role is providing a solution for balancing the best financial outcome for the company with what is best for the customer. But until these organisations can prove that their machine isn’t too ruthless or lenient, then I advise they treat AI with caution.”