During one of the late Diana, Princess of Wales' fits of pique, she threatened to take her sons and move to Australia. She was quickly reminded that she did not have legal custody of her sons, and could not take Prince William and Prince Harry out of the country without Queen Elizabeth's permission.
It was also noted that when the Prince and Princess of Wales were divorced in 1996 (as well as the Duke and Duchess of York's divorce in the same year) the divorce agreement did not include custody arrangements for the children. The arrangements for custody were made privately between the two parents. Although the two princes' primary residence was with their mother, the divorce agreement did not given Diana "the care and control" of her two sons.
This is because neither the Prince and Princess of Wales nor the Duke and Duchess of York had legal custody of their children. Sounds far fetched, but true.
In 1717, after King George I had a major disagreement with his son, the Prince of Wales, over the latter's children, the king put the question of custody to his judges. Ten out of 12 judges ruled that the "king's right of supervision extended to his grandchildren and this right of right belongs to His Majesty, King of the Realm, even during their father's lifetime."
In an article in The Times (December 5, 1993), noted Constitutional expert Michael L. Nash, wrote that the "Queen has the last word in the custody upbringing, education and even the right of abode of the princes, even during the lifetime of their father, Prince Charles. As for their mother, the Princess of Wales, her say is a matter of discretion and negotiation."
This decision was legislated in 1717, and again in 1772, the same year as the Royal Marriages Act. These laws have not been superseded by newer legislation. [Update: The new succession law, which won't become official until after the Australian parliament passes it, and then the Queen will give her royal assent at a Privy Council meeting. This will probably take place some time during 2015. The succession to the crown has been changed to allow the succession of the eldest child regardless of sex, restores the succession rights of those who married Roman Catholics, and repeals the Royal Marriages Act, which has nothing to do with the custody legislation passed during the reign of George I]
When Prince William was an infant, he traveled with his parents to Scotland by plane. It was mentioned in the media at the time that Charles had needed his mother's permission to allow William to fly with him. This was noted by the broadsheets.
Diana's Will has a clause where she stated "Should any child of mine be under age at the date of the survivor of myself and my husband, I APPOINT my brother EARL SPENCER to be guardians of that child and I express the wish that should I predecease my husband he will consult with my mother with regarding to the upbringing and welfare of our children."
This clause would have not held up in court because Diana, who separated from Charles when her Will was written, had no legal right to decide about her children's upbringing and education. Queen Elizabeth II has had a far better relationship with her children, but it is unlikely that she would have allowed Lord Spencer to be her grandsons' legal guardian in the event that both Charles and Diana died before their sons' majority. More likely, the orphaned princes would have been raised by one of the Queen's younger children.
At the time of her death, Diana was largely estranged from her mother and her brother. After Diana's death, Charles invited his former mother-in-law to spend time with her grandsons, although she had no real role in their education and upbringing.
After the official separation, Charles and Diana each had about 40 days a year with their children.