VAT on private school fees was introduced on 1 January across the UK.
Lord Pannick, one of the country's leading barristers in cases relating to government decisions, said the families he is representing "are not wealthy".
Two of the claimants have no alternative schools in their area to meet the needs of their children who have SEND, the court heard.
Another family sent their child to a single sex independent primary school after they experienced "harassment" in their previous co-educational school.
Lord Pannick argued that adding VAT to their fees impedes their right to access education and the policy is "unprecedented" in all other Council of Europe states.
Around a quarter of families affected by the VAT changes fall below the average wealth level, and the policy has been brought in "irrespective of the needs of the child" or of the ability of the family to pay, he told the court.
Lord Pannick is leading the challenge for the Independent Schools Council (ISC) which represents more than 1,400 private schools. There are two further claims also being heard by a group of faith schools and the parent-led group Education not Discrimination.
They are seeking a declaration of incompatibility with Article 2 of the First Protocol of the European Convention on Human Rights, which is the right to education, and that the policy is also discriminatory.
It is "not good enough" for the government to say "it has been debated in Parliament" and will "raise revenue", Lord Pannick argued, stating that the policy needed stricter scrutiny.
Barristers representing Chancellor Rachel Reeves, along with HMRC and the Department for Education (DfE), are due to give their side of the case on Wednesday.
In his written submission, Sir James Eadie KC, representing the government, said parents wishing to opt out of "universally accessible state-funded education" are free to choose "any private education for their child that they can afford", or they can "educate their child at home".
Sir James will argue that taxes, minimum wage laws and national insurance, which all affect the cost of private school fees, do not interfere with "with freedom to offer or receive private education."
SEND pupils who have an education, health and care plan which names a private school have their fees paid for by their local council.