New analysis of the government's shared parental leave scheme shows uptake is skewed against lower earners in favour of wealthier families in south-east England.
Shared parental leave was introduced 10 years ago this week with the aim of allowing dads to play a more prominent role in fatherhood.
But campaign group The Dad Shift says it is "failing working families".
Labour says it is carrying out a review of parental leave.
The figures, from a Freedom of Information request submitted to HMRC by The Dad Shift, show the top 20% of earners make up 60% of those to use shared parental leave.
Just 5% of those who took up shared parental leave came from the bottom 50% of earners.
And since its introduction in 2014, the government has paid £40m in shared parental leave payments to families in London - almost 10 times more than in the north-east of England.
George Gabriel, co-founder of The Dad Shift says: "There's a big awareness gap, there's a complexity problem and there's a challenge about most men feeling uncomfortable taking leave off the mother of their child."
A government review from 2023 suggests almost half (45%) of all dads weren't even aware shared parental leave was an option.
And it was used in fewer than 2% of all births last year.
Those who have used the scheme are positive about its benefits, though.
Pete Target, from London, took nine weeks off work to look after his son, Jay, when he was nine months old.
"It was everything I’d hoped for," says Pete, 36.
"We’re closer than we could have possibly otherwise been."
But he admits it was a "difficult financial decision".
Because his section of shared parental leave fell outside the 37-week window for government support, he wasn't paid for the time he took off with his son.
He says "something has to improve" to ensure more dads can benefit.
"I feel so lucky that we could afford the financial hit, [but] it needs to be practically possible for most men to do it.
"We need men to feel like they can take it."
As the sole earner in his young family, 21-year-old Josh Wiborg couldn't take shared parental leave when his twin girls were born.
Like so many dads, he was entitled to only two weeks off work, at reduced pay.
He says it left him "feeling like a stranger" to his twin girls, Autumn and Winter, in the first few weeks of fatherhood.
“It was awful," he says. “My partner was struggling and I was struggling as well. It just wasn’t enough time.”
As his wife wasn't working, Josh wasn’t entitled to shared parental leave and instead took statutory paternity leave, the government-funded scheme that allows dads or secondary care providers to take two weeks off after their baby is born.
Those eligible receive £184.03 a week or 90% of their average earnings, whichever is lower.
It meant Josh's earnings dipped immediately after his twin girls were born.
"I had to go back to work [after two weeks] and leave my partner on her own," he says.
"I don’t know how we got through it."
The government has committed to reviewing parental leave during its first year in office.
It says it wants statutory paternity leave to be a day-one right for all employees. Currently, it applies to dads who have been employed for 26 weeks or more.
But even with these proposed changes, campaigners say the UK's parental leave system is one of the worst in Europe.
Blair McDougall, a father of two, is one of a number of Labour MPs backing The Dad Shift's campaign for more generous parental leave.
While he says the government is right to "keep one eye on the economic reality", he is in discussions with a group of fellow backbench Labour MPs to discuss whether future changes can go even further than those already outlined.
"Society has moved on but the rules around paternity haven't," he says.
"We need to bring those up to date to help fathers be the type of dads that most of us want to be."
Some businesses pay out of their own pocket to offer enhanced paternity leave and shared parental-leave packages.
This can include more time off and pay that is boosted to full earnings or a higher rate than the statutory offer.
But The Dad Shift says taxpayers should be funding "more substantial" parental-leave packages, to take the burden off businesses and ensure every parent can benefit.
Paul Bowen, who employs roughly 30 people at his pie factory and shop in Chorley, Lancashire, says the rise in National Insurance contributions is the latest in a long line of budgetary squeezes for small businesses like his.
"I'd love to give our employees extra time off, but we simply can't afford it," he says.
"If we can have help from the government, that's where it needs to come from."
Josh is now heavily involved in bringing up his twin girls but he still remembers how "tricky" those first few weeks were.
"The situation needs to be changed completely," he says.
When he was struggling in the early days of fatherhood, he approached North East Young Dads and Lads (NEYDL) for help.
He now works for the charity as a "peer enabler", offering the same advice and support from which he benefited.
"I understand the struggles of other dads," he says.
"I know it's stressful, so if I can just help one person it is so worth it."
Additional reporting from George Walker.