The announcement came as part of a wider speech on reforming the state, which Sir Keir described as "overstretched" and "unfocused".
"Over a number of years politicians have chosen to hide behind vast arrays of quangos, arm's length bodies, regulators, reviews," the prime minister said.
"A sort of cottage industry of checkers and blockers using taxpayer money to stop the government delivering on taxpayer priorities."
The number of quangos has fallen by more than half since 2010 but there are still more than 300 across the UK.
They include regulators, cultural institutions and advisory bodies, ranging from large organisations like NHS England, to smaller ones like the Gambling Commission and the British Film Institute.
The water regulator Ofwat could be among other government quangos in line to be scrapped.
Ofwat's role - along with the Environment Agency, the Drinking Water Inspectorate and Natural Resources Wales - is among the organisations already being looked at in a review of the whole water system in England and Wales, by an Independent Water Commission established last year.
Government sources would not pre-empt the outcome of the review but all options for regulating water are understood to be on the table, except nationalising water companies.
The new Labour government has set up more than 20 new quangos since winning power.
These include Great British Energy, which will invest in renewable energy to help meet the government's clean power goals, and Skills England, which aims to support people to find jobs.