Generative AI Could Create a Fake New World of Claimants, Witnesses & Risks

Consider this one fact; fake AI girls are now replacing real women on sites like Only Fans. That may not seem relevant to your insurance book right now, but the spread of fake AI video evidence and fake ID undermines the concept of claims itself.

Did a road accident actually happen, or were the pictures, video and witnesses actually faked? Think about wider implications on insurance and reinsurance pricing, like faking aerial surveys of new construction, hull repairs, carbon offset forests, or insured sites with the oil and gas rigs edited out for ESG compliance. AI fakery could save a company millions in premiums. It could also create a cat event claim that could profit an entire country.

One more example closer to home; virtual PI claims hearings and injury claims portals sound great to politicians and insurers alike since they negate the need for in person hearings at Civil Courts. But if you don’t know if the expert witness, the evidence or the claimant themselves, are real, or just very good AI, then how can you process genuine claims from the generative AI ones? Something to think about in the future there and DLB Investigations posted some useful insights on LinkedIn recently. 

Here are some insights from Gartner;

The mass availability of generative AI, such as OpenAI’s ChatGPT and Google Bard, became a top concern for enterprise risk executives in the second quarter of 2023, according to Gartner, Inc.

“Generative AI was the second most-frequently named risk in our second quarter survey, appearing in the top 10 for the first time,” said Ran Xu director, research in the Gartner Risk & Audit Practice. “This reflects both the rapid growth of public awareness and usage of generative AI tools, as well as the breadth of potential use cases, and therefore potential risks, that these tools engender.”

In May 2023, Gartner surveyed 249 senior enterprise risk executives to provide leaders with a benchmarked view of 20 emerging risks. The Quarterly Emerging Risk Reports includes detailed information on the possible impact, time frame, level of attention, and perceived opportunities for these risks.

Third-party viability was the top fast-emerging risk that organizations are monitoring most closely in the 2Q23 survey (see Table 1). Financial planning uncertainty was the third ranked risk, followed by cloud concentration risk. China trade tensions rounded out the top five risks that were split between issues symptomatic of the current broad macroeconomic and geopolitical volatility, and technology-related concerns.

Table 1: Top Five Most Commonly Cited Emerging Risks in Q2 2023

Source: Gartner (August 2023)

Mass Generative AI Availability
Gartner has previously identified six risks of generative AI and four areas of AI regulation that are relevant to assurance functions. In terms of managing enterprise risk, three main aspects must be addressed, according to Gartner experts:

• Intellectual Property

“Information entered into a generative AI tool can become part of its training set, meaning that sensitive or confidential information could end up in outputs for other users,” said Xu. “Moreover, using outputs from these tools could well end up inadvertently infringing the intellectual property rights of others who have used it.”

It’s important to educate corporate leadership on the necessity for caution and transparency around the use of such tools so that intellectual property risks can be properly mitigated both in terms of input and output from generative AI tools.

• Data Privacy

Generative AI tools may possibly share user information with third parties, such as vendors or service providers, without prior notice. This has the potential to violate privacy law in many jurisdictions. For example, regulation has already been implemented in China and the EU, with proposed regulations emerging in USA, Canada, India and UK among others.

• Cybersecurity

“Hackers are always testing new technologies for ways to subvert it for their own ends, and generative AI is no different,” said Xu. “We’ve seen examples of malware and ransomware code that generative AI has been tricked into producing, as well as ‘prompt injections’ attacks that can trick these tools into giving away information they should not. This is leading to the industrialization of advanced phishing attacks.”

Causes and Implications of Third-Party Viability Risk

“Persistent inflation that is less responsive to interest rate rises and contuse longer than anticipated has escalated costs and margin pressures on third parties,” said Xu. “As central banks increase interest rates to fight inflation, this also brings about a process of credit tightening that may force suppliers to suspend operations or become insolvent as borrowing costs rise.”

If economic conditions deteriorate broadly, this may cause an unexpected drop in demand that could affect vendor viability or their ability to provide goods and services in a timely manner. Gartner experts identified three potential third-party viability consequences for risk managers to monitor as the situation develops:

• Loss of Key Inputs and Materials: If third-parties are increasing their prices due to the wider economic situation there is a clear risk of losing access to key inputs and materials as third parties would favour customers willing to pay higher prices.
  
  
• Flawed Financial Planning Assumptions: Cost assumptions will be rendered invalid as suppliers increase prices or fail, necessitating switching costs and increased prices for obtaining goods and services.
  
  
• Challenges Outside the Supply Chain: Partners, such as managed service providers or commercial partners, creditors, or technology vendors may cease or curtail operations.

Gartner clients can read more in 2Q23 Emerging Risk Report.