26/07/2017 7:24 AM AEST | Updated 26/07/2017 7:27 AM AEST

Robert Pattinson Is Over The 'Troop Of Crazies' Who Judge His Love Life

If anyone knows a thing or two about fandom, it’s Robert Pattinson.

The now 31-year-old actor was hounded by fans, paparazzi and the press during his days starring as vampire Edward Cullen in the “Twilight” franchise from 2008 to 2012 ― mostly due to his real-life romance with co-star Kristen Stewart. And let’s just say he hasn’t, nor will he ever, forget it. 

When discussing his relationship with singer FKA Twigs (Tahliah Debrett Barnett) on the Howard Stern Show Tuesday, Pattinson called out Twihards for invading his privacy and personal life. 

“For whatever reason with ‘Twilight,’ I don’t know whether it is the specific thing with the audience, but there’s like a crack troop of crazies who think every single decision you’re making is creating some kind of conspiracy,” he said. “So to protect [your relationship], you kind of think, ‘I want to create a big boundary between it.’ But then it makes it difficult for your actual relationship.”

Pattinson dealt with the “crazies” when he was in a relationship with Stewart, specifically when she was photographed kissing her “Snow White and the Huntsman” director, Rupert Sanders, while dating the actor. At the time, the photos were spread across tabloids and on websites. The attention surrounding the scandal proved to be too much for both Pattinson and Stewart, who eventually split for good in 2013. (Stewart is now dating model Stella Maxwell.)

BEN STANSALL via Getty Images

In order to protect his new life with FKA Twigs, they’ve decided to keep things private and not discuss their romance in detail. 

“It’s one of the most frustrating things in the world because you want to be able to do that,” Pattinson said of flaunting their relationship. “You have to make the decision whether you want to let the kind of crazy people in.” 

Still, Pattinson did confirm that, yes, he’s (“kind of”) engaged to FKA Twigs and he’s thrilled about it. 

“Yeah, she’s amazing,” the “Good Time” star gushed. “She’s super talented and it’s kind of a totally different world [to dating an actress].”