This academic year, parents are still able to see the four grades across sub-categories.
However, 72% of school leaders said they did not agree with the retention of Ofsted grades across four sub-categories as an interim arrangement this year.
Three in four also did not believe sub-grades should be used in a future report card system, the poll found.
Mr Whiteman said: "While some of the interim measures taken last year to make inspection more humane have been a step in the right direction, they clearly do not go anywhere near far enough."
He said the fundamental problems with the inspections remained and the current system was "clearly broken".
"Now is the moment for a genuinely new approach that gives parents the detailed information they need and addresses the hugely harmful impact inspection currently has on school staff," he said.
"What we cannot accept is a revised version of the existing system."
Formal consultations on the report card system, new framework, and inclusion and safeguarding grading are due to be launched later this month.
Ofsted's new chief inspector, Sir Martyn Oliver, previously said the body had to ensure high standards, but admitted it could also be "far more empathetic".