More than 60 institutions, including several top private schools and church bodies, are being investigated.
The inquiry, which is being chaired by Lady Smith, is looking in detail at historical abuse of children in residential care.
It is expected to report in late 2019 – four years after it was set up.
The opening session in Edinburgh heard apologies from groups who said they “deplored that physical sexual abuses could occur”.
They included Quarrier’s, Daughters of Charity of St Vincent de Paul, Sisters of Nazareth, Good Shepherd Sisters, De La Salle Brothers and Christian Brothers.
Lady Smith began the public hearings by acknowledging that many children in Scotland have been abused in residential care over the years.
The High Court judge revealed the number of survivors who had already spoken to the inquiry team was “very far in excess” of 200.
She said: “They suffered some terrible treatment inflicted by those to whom their care was entrusted. That is a matter of grave concern.
“It is critically important that our community engages in facing up to the fact that children in care were wronged and failed in the past and to commit to seeing to it that children of today and of the future are safe.
“It’s not easy to do that, for many it will be a painful process. But if we are to achieve real, substantial and lasting change for the better it has to be done.”
John Scott QC, for the In Care Abuse Survivors group (Incas), said children had been “cruelly betrayed by those supposed to care for them”.
He said the inquiry had come too late for those who have already died, but added: “It is not too late for at least some acknowledgement and accountability, not too late for some compensation, not too late for other survivors to come forward.”
The hearings have now been adjourned until Thursday.
The inquiry states its purpose as being “to investigate the nature and extent of abuse of children whilst in care in Scotland”, while considering “the extent to which institutions and bodies with legal responsibility for the care of children failed in their duty”, in particular seeking any “systemic failures”.
It classes “in care” as being:
- Children’s homes (including residential care provided by faith based groups)
- Secure care units including List D schools
- Borstals and Young Offenders’ Institutions
- Places provided for Boarded Out children in the Highlands and Islands
- State, private and independent Boarding Schools, including state funded school hostels
- Healthcare establishments providing long term care, and any similar establishments intended to provide children with long term residential care
- Children in foster care
But it does not cover children who were abused while living with their natural or adoptive families, while using sports and leisure clubs or attending faith based organisations on a day to day basis.
The inquiry will also not examine allegations of children being abused in non-boarding schools, nursery or day-care centres.
Its terms of reference say it covers a time period “within living memory of any person who suffered such abuse”, up until the point the inquiry was announced in December 2014, and will consider if “changes in practice, policy or legislation are necessary” to protect children in care from abuse in future.
Analysis by Reevel Alderson, BBC Scotland home affairs correspondent
The inquiry’s remit has been criticised as being too narrow, even though the government says it is the widest public inquiry they have ever established.
Some campaigners say that while abuse in residential schools, for example by a priest, would be covered, abuse by the same person in a parish or working with a youth organisation like the Scouts would not.
And it has also been criticised for not dealing with the question of redress or compensation, which by contrast has been the case with other inquiries in the Republic of Ireland, Northern Ireland, Jersey and Australia.
The inquiry at Rosebery House, in Edinburgh’s Haymarket, will hear from survivor groups and large care providers as well as faith-based organisations.
Among those appearing will be representatives of the Church of Scotland/CrossReach, the Benedictines, Barnardo’s and Aberlour Child Care Trust.
The Scottish government will give evidence on “the nature, extent and development of the state’s areas of responsibility for children in residential and foster care in Scotland”.
Some survivors and campaigners staged a vigil outside the building before the inquiry got under way.
They also organised a minute’s silence at 13:00 “to remember those children whose lives have been taken by abusers, those who have lost their lives from the affects of abuse and those who have died without gaining justice for the crimes committed against them”.
The inquiry has been plagued by problems since it was set up in October 2015. Around £6m has been spent on it during this period.
Its original chairwoman, Susan O’Brien QC, resigned from her post in July 2016, citing government interference.
A second panel member, Prof Michael Lamb, also resigned, claiming the inquiry was “doomed”.
Deputy First Minister John Swinney said at the time that he rejected any charges of interference in the independence of the inquiry, and that the Scottish government wanted a “robust independent inquiry that can operate without fear or favour”.