According to The Sun, Simon ordered producers to “break the bank” to secure the singer as a judge.
Lily’s notorious acid tongue and spikey wit would make her a surefire hit on the show if she accepts the offer.
But the pop star, who has been avoiding the spotlight since tragically suffering a miscarriage last year, is yet to be won over.
A source told the paper: “Lily was flattered to get Simon’s call. It’s a big job and an honour to be asked. But she’s not convinced.
“She has tried to take a step back from the limelight over the last six months and doesn’t want to be thrust right back in to the madness of the showbiz pantomime that comes with the show.
“She doesn’t need the money or the hassle at this stage in her life. Maybe in a few years’ time when circumstances change she will be more receptive.”
Simon is rumoured to be leaving the show so he can concentrate on launching the US version of X Factor.
During a recent US TV interview Simon dismissed reports P Diddy was set to join the show, describing the rapper as “too expensive.”
He also ruled out Christina Aguilera and Mariah Carey, saying “absolutely not.”
However he admitted to having spoken to singers Jessica Simpson and Shania Twain.