Prince Harry has said he and Meghan were portrayed as the "new kids on the block" who threatened to "steal the limelight" from other royals - and that led to problems in those relationships.
Of brother Prince William and his wife Catherine, he said: "I always hoped that the four of us would get on.
"But very quickly it became Meghan versus Kate.
"And that, when it plays out so publicly, you can't hide from that," he told Tom Bradby, in an ITV interview.
In the 95-minute interview to promote his memoir, titled Spare, he recalled the days when he was the "third wheel" at official engagements and other outings - but his relationship with Prince William and Catherine was particularly warm at that time.
He said he imagined that would continue when he found his own partner and they became a foursome, but it was not to be.
Catherine, Princess of Wales, Prince William, Prince of Wales, Prince Harry, Duke of Sussex, and Meghan, Duchess of Sussex, on the long Walk at Windsor Castle on September 10, 2022, before meeting well-wishers after the death of the Queen. Photo: Kirsty O'Connor / POOL / AFP
He said negative attitudes about Meghan - particularly around her being an American actress - were heightened by a hostile press.
Those stereotypes about the new woman in his life had been a barrier to his brother and sister-in-law "welcoming her in" to the family, he said.
Prince Harry said while Prince William never tried to dissuade him from marrying Meghan, he did "air some concerns" and warned Harry: "This is gonna be really hard for you".
"I still to this day don't truly understand which part… he was talking about. But maybe, you know, maybe he predicted what the British press's reaction was gonna be."
In the interview, Harry also spoke of the emotional impact of the death of his mother and in his book describes returning to the scene of the car accident in Paris - and asking to be driven through it at the same speed as she was in 1997.
"I'd always imagined the tunnel was some treacherous passageway, inherently dangerous but it was just a short, simple, no-frills tunnel. No reason anyone should ever die inside it," he said.
He saw no point in reopening inquiries into the car accident - but questioned the official conclusions about the night of the crash.
A 2008 inquest found Diana was unlawfully killed partly due to the "gross negligence" of her driver who had been drunk and driving at excessive speed.
In the interview, Prince Harry said that the tunnel posed "no danger of anybody losing control even after a drink or a couple of drinks… [It was] almost physically impossible to lose control of a vehicle unless you are completely blinded at the wheel," implying that distraction from the paparazzi may have caused the driver to crash.
"The people that were predominantly responsible for it, all got away with it," he said.
Prince Harry also said that, after Diana's death, he and his brother William were sat down in a room and told that the events leading to the car crash were "like a bicycle chain".
"If you remove just one of those links from the chain, the end result doesn't happen," Harry recalled being told.
When William asked what might have happened had the paparazzi not been chasing Diana's car, he was told that "the result wouldn't have been the same."
During the interview, Harry also expressed his hope that he might one day reconcile with his father and brother.
"Forgiveness is 100 percent a possibility, because I would like to get my father back. I would like to have my brother back," he told interviewer Tom Bradby.
"At the moment, I don't recognise them, as much as they probably don't recognise me."
He blamed the tabloid press as the "antagonist", who he said "want to create as much conflict as possible".
The ITV interview is the first of four broadcast appearances to be aired over the coming days to promote Spare.
Prince Harry also spoke to two US TV networks. He gave an interview to Anderson Cooper for 60 Minutes on CBS News, and also spoke to Michael Strahan of Good Morning America, which will be broadcast on Monday. He will also appear as a guest on Stephen Colbert's Late Show on CBS on Tuesday.
Although Spare is not due to be published until Tuesday, extracts were leaked after some copies went on sale early in Spain. BBC News has obtained a copy and has been translating it.
Kensington Palace and Buckingham Palace have both said they will not comment on the contents of the book.
Prince Harry criticises lack of royal defence for Meghan after Jeremy Clarkson column
Prince Harry has accused the royal family of failing to defend his wife Meghan in the controversy over a Jeremy Clarkson newspaper column.
In the ITV interview, the Duke of Sussex said the "silence is deafening" about the "horrific" Sun article last month.
He contrasted this with the quick action taken after a race row at a Buckingham Palace reception.
Prince Harry also said he did not believe comments about his son's skin tone, by an unnamed royal, were racist.
The Clarkson article about Meghan had described how the columnist was "dreaming of the day when she is made to parade naked through the streets of every town in Britain while the crowds chant 'Shame!' and throw lumps of excrement at her".
Later taken down by the Sun and prompting an apology from the paper and Clarkson, the article was described by Prince Harry as "horrific and hurtful and cruel towards my wife".
Prince Harry and Meghan Markle speak during the 2021 Global Citizen Live festival at the Great Lawn, Central Park on 25 September, 2021 in New York City. Photo: AFP / Angela Weiss
"The world is asking for some form of comment from the monarchy. But the silence is deafening. To put it mildly," he said.
"Everything to do with my wife, after six years, they haven't said a single thing."
The prince contrasted the lack of a royal response to that article with the events that followed an encounter at Buckingham Palace between Lady Susan Hussey and Ngozi Fulani, just three weeks earlier.
Fulani, a black British charity founder was a guest at an event hosted by Camilla, the Queen Consort.
She was challenged repeatedly by Lady Hussey about where she was "really from" - and Fulani complained about how the exchange had offended her.
The controversy prompted a rapid apology from the Palace and a personal meeting was arranged between the two women.
The palace issued a statement in which it described the remarks as "unacceptable and deeply regrettable". Lady Hussey ultimately resigned as a lady of the household.
In his interview with Tom Bradby, Prince Harry defended Lady Hussey, saying "she had never meant any harm at all".
But he pointed to the contrast between the treatment given to Fulani and to his wife after the language used about her by Clarkson.
The ITV interview returned to Prince Harry and Meghan's previous claim - made in a 2021 interview with Oprah Winfrey - that a member of the royal family had raised questions about the skin colour of their future child.
Prince Harry again did not name the individual - and suggested this might have been a case of "unconscious bias" rather than racism.
Asked if he would see the questioning as racist, he said: "I wouldn't, not having lived within that family."
He rejected that he had accused members of the royal family of racism in the Oprah interview, saying the "British press had said that".
The wide-ranging interview also discussed the tensions that followed Meghan's introduction into the royal family.
Bradby said there was an impression that Prince William and Catherine "almost from the get-go" did not "get on" with Meghan.
Asked if that was a fair observation, Prince Harry replied: "Yeah, fair."
-BBC