Tyson Fury and Anthony Joshua have been urged to arrange the mega all-British fight by early 2025 or it will be 'too late'.
Former boxer Anthony Ogogo is concerned that interest in the bout will decrease the longer fans are made to wait.
Promoter Eddie Hearn has confirmed that Joshua is in line to fight the winner of Fury's undisputed clash against Oleksandr Usyk in May.
But Fury has repeatedly stated that he will first give Usyk a rematch, suggesting any possible fight with Joshua won't happen until next year.
Joshua has already lost twice to Usyk and Ogogo does not think the 34-year-old will ever get the better of the Ukrainian.
The former Olympic bronze medalist says a fight between Joshua and Fury is exactly what the sport needs.
Ogogo told Sky Sports News: "We all want the Fury fight don't we, we all want the Fury fight.
"It will probably be the biggest fight in British boxing history.
"I don't think Joshua beats Usyk any day of the week. Usyk's too good, he's too tricky. I don't think anybody can beat Usyk right now.
"But the Fury-Joshua fight is what we all want to see. Two of Britain's best-ever heavyweight boxers coming together.
"That's a fight that will get non-boxing fans interested. And that's what boxing needs.
"Boxing needs the crossover to be even more mainstream than it currently is. So I think a big Joshua-Fury dust-up is the fight we all want to see surely."
Ogogo is confident that Fury and Joshua will eventually battle it out in the ring.
But the 35-year-old says some of the gloss will be taken off the bout if it doesn't happen by early 2025.
Ogogo compared it to the much-anticipated clash between Manny Pacquiao and Floyd Mayweather, which eventually happened in 2015 after years of waiting.
He added: "I think it will happen. My concern is it's going to be the Manny Pacquiao-Mayweather thing.
LATEST SPORTS NEWS:
"That should have happened five years before it actually happened.
"When it happened, people were less bothered, they were less interested. Pacquiao got knocked out by Rafael Marquez and it wasn't as big as it could have been.
"My worry is that it's going to happen two or three years too late. It's got to happen in 2024 or early 2025 in my humble opinion."
Ogogo admits the fight will define either Fury or Joshua's career, no matter when it occurs.
He added: "Is it career-defining? Of course it's career-defining, of course it is. It will be the biggest fight in British boxing history.
"Of course it's career-defining! Win or lose, they're both British boxing legends.
"They're both going to earn so much money from it, it has to happen."
