Foreign Affairs Committee chairwoman Dame Emily Thornberry became the first high-profile figure to announce she was thinking about a bid on Sunday.
Dame Emily was a shock omission from Sir Keir's government, having been his shadow international trade secretary in opposition.
On Monday, former transport secretary Louise Haigh called for sweeping "economic reset" and reforms to Labour's rigid fiscal rules in an article in the New Statesman, but fell short of declaring a bid to become deputy leader.
In government, Haigh was seen as a leading left-wing voice at the cabinet table, before quitting last year after it emerged she pleaded guilty to a fraud offence a decade ago.
Speaking at the weekend, Labour's mayor of Greater Manchester, Andy Burnham said the contest was an opportunity to have a "discussion about the internal management".
Burnham warned the cabinet lacked "balance" after Rayner's exit, and called for another voice from the north of England as deputy.
He suggested Haigh or ex-Commons leader Lucy Powell would help to counter what he called the "London-centricity" under Sir Keir's leadership.
Shabana Mahmood, who on Friday was appointed home secretary in a major shake up of ministers, has ruled herself out of the race.
"My top priority is securing our borders - I will not be running for deputy leader of the Labour party," she told reporters.
Other ministers have been floated as potential replacements, including Alison McGovern, while among MPs, NHS doctor Rosena Allin-Khan has been touted.