Local residents have also attempted to convince the miners to come out of the mineshaft.
"Those people must come out because we have brothers there, we have sons there, the fathers of our kids are there, our children are struggling," local resident Emily Photsoa told AFP.
The South African Human Rights Commission says it will investigate the police for depriving the miners of food and water.
It said there is concern that the government’s operation could have an impact on the right to life.
Minister Ntshavheni's remarks have provoked mixed reaction from South Africans, with some praising the government's unyielding approach.
"I love this. Finally, our government is not tiptoeing on these serious matters. Decisiveness will help this country," one person wrote on X.
While others felt the stance was inhumane.
"In my view, this kind of talk from the Minister in the Presidency is disgraceful and dangerous hate speech," one user said.
Another wrote: "They are criminals but they have rights too."
Illegal mining is a lucrative business across many of South Africa's mining towns.
Since December last year, nearly 400 high-calibre firearms, thousands of bullets, uncut diamonds and money have been confiscated from illegal miners.
This is part of an intensive police and military operation to stop the practice that has severe environmental implications.