"You'll never need ID to go into a hospital or anything like that," he said.
"For people who simply don't want it, well, they don't need it - apart from the right to work."
It's not clear the idea will be welcomed by the public however. A petition calling for it to be scrapped has attracted nearly three million signatures.
The prime minister's intervention appears to have done little to win over those with privacy concerns.
"Keir Starmer has already lost the public's trust on digital IDs," said Silkie Carlo, head of civil liberties campaign group Big Brother Watch.
"The only way to safeguard the public's privacy and right to choose is to reject plans for a mandatory digital ID, as millions of people have petitioned the Prime Minister to do."
But the PM denied it would ever be used as a surveillance tool - and said the personal data which would have to be involved in creating it would "absolutely have very strong encryption".
It would also exist on your device rather than being stored in data centre computers, for additional security.
While making it non-mandatory may address some of the criticism around digital inclusion and those without access to smartphones, it may also make it a less powerful tool as a result.
Digital ID is already in use in many countries around the world including India, Denmark and Singapore.
China has introduced a voluntary system this year but there are fears about its likely use by the police to track online activity. One of the first countries to adopt e-citizenship was Estonia in 2002.
Sir Keir did not expand on who might operate the UK digital ID scheme and whether it was likely to be a US tech giant.
It has emerged today that it will now be overseen by the Cabinet Office, rather than the Department for Science, Innovation and Technology, which is responsible for other government digital services.
Sign up for our Tech Decoded newsletter to follow the world's top tech stories and trends. Outside the UK? Sign up here.