Innovation, which provides software and services to help insurers process claims, said the talks may or may not lead to a deal.
Shares in Innovation Group, which have risen by 18 percent in the last 12 months, were trading 6 percent higher at 37.5 pence at 1029 GMT.
Under takeover rules, Carlyle has until Sept. 25 to make a firm offer for Innovation or announce it does not intend to make an offer.
Innovation Group is being advised by Investec.
($1 = 0.6503 pounds)
(Reporting by Paul Sandle, Reuters)