While Vicario's mistake dominated headlines, this was an all-round bad day for Spurs to continue their home woes.
This was their 10th home defeat of 2025 in the league, a joint club record along with 1994 and 2003.
They still have home games against Frank's old side Brentford and champions Liverpool to come before the end of the year.
Only one of those wins have come since Frank replaced Ange Postecoglou in the summer.
"When you're down 2-0 after six minutes, there is a mountain to climb," said Frank.
"When you're in a bad spell, everything seems to go against you as well - the first was a deflected shot, the second is a mistake from Vic.
"This result leaves us in a place where we have lost another game. Every game has a single story, this game we lost in the first six minutes.
"We just need to keep working. The second half was much better and hopefully something we can use to learn."
Spurs have lost four consecutive home London derbies for the first time in the Premier League.
And they are averaging 9.5 shots and 3.2 shots on target per game in the Premier League - their lowest averages on record in a single campaign (since at least 2003-04).
Former Fulham midfielder Danny Murphy told Match of the Day that Frank has to ride the storm.
"He's got to take the stick," Murphy said. "He's taken a high profile job at a huge football club with massive expectation. There is pressure and responsibility that comes with that.
"The performances at home have been poor and they have to improve quickly. I don't think at the moment he knows his best team and he's not sure on his best formation and that's a problem.
"Confidence is really low. He has to ride the criticism, keep working on the training pitch and get that team winning some games at home."