Milo Ventimiglia Says 'Gilmore Girls' ' Jess Was a 'Child' Who 'Had a Lot of Life to Live'

Sorry to members of Gilmore Girls' Team Jess, Milo Ventimiglia is not with you.

After playing one man in a years-long love battle for the heart of Rory Gilmore (Alexis Bledel), the actor now says he wouldn't support Jess as a boyfriend for Rory because the bad boy of Stars Hollow was "a child."

"I think Jess had a lot of life to live," Ventimiglia told InStyle. "And I don't think he was quite set up the best way in his younger years — not having a father, not having a parent. He had to find his way on his own a little bit. And I think he got to it a little quicker, just a place of ... acceptance of who he was, and what he wanted to be, and who he wanted to be around."

Now that he's older, Ventimiglia said he's learned more about life: "The older I've gotten, you know, [at age] 45, emotions kind of hit you differently. You care about life a little bit more. You care about people a lot more."

While emotional maturity wasn't one of Jess's strengths, Ventimiglia did say the role prepared him in one specific way to play another TV heartthrob — This Is Us patriarch Jack Pearson — years later.

"I think playing Jess helped me understand [the concept of an] 'internet boyfriend' a little bit," he quipped.

Contrasting Jack and Jess, he revealed where he really found inspiration during his six seasons playing Jack.

"Jack and Jess are two very different people to me," he explained. "I never really saw [similarities], other than the kind of absence of a father for Jess, and I know Luke [Scott Patterson] represented that male role model figure who was there to really shepherd Jess out of some troubling times. But Jack was more inspired by my own dad."

Unlike Jess, Ventimiglia said he felt like Jack was an "attainable hero" — someone people could "strive to be."

"It goes beyond just men, women, old, young — any background. It's like, 'Let's just strive to have a golden heart,'" Ventimiglia said. "We all have our hangups. We all make our mistakes. But fundamentally, who Jack was? He was just a good, solid man."

After playing two greatly desired "internet boyfriends," Ventimiglia is pivoting into a completely different role as a smooth-talking swindler in ABC's The Company You Keep.

Perhaps showing he's more of a Jack than a Jess, the actor made a point to bring along several folks from his This Is Us team to his new series.

"This Is Us was such a wonderful set to be on," he told emmy magazime. "How could I not try to put that magic into a new endeavor that we have? So far, it has really saved us, helped us."

He adds, "I know a lot of talented folks. When you can work with your friends, I just think that's the best recipe. You're going to capture something great and have a good time while you're doing it."

And though his new character Charlie Nicoletti falls mostly on the side of Bad Boy, he'll probably have a few hints of Jack in him as well. "I'm sure Charlie will pop up with a mustache, with some facial hair," Ventimiglia teases, "and maybe blonde at some point or a little wild hair."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

The Company You Keep premieres Sunday at 10 p.m. ET on ABC.