LOS ANGELES (AP) — Twenty-some years ago, when Jason Mantzoukas was a struggling actor and comedian in New York, a casting agent told him that he wasn’t going to work for a while. She said that while he was funny, talented and one of her favorites, he’s not what people pictured when writing roles and was falling through the cracks.

Tall, with a shock of thick, curly black hair, deep set brown eyes and an olive complexion courtesy of his Greek heritage, she said he was both “handsome” and “not handsome enough” and both “too ethnic” and “not ethnic enough.” But she also knew that once someone cast him, he’d work forever.

Mantzoukas insists it was a generous reckoning.

“It was true,” Mantzoukas, 44, said in between sips of mint tea on a sunny evening in the Silver Lake neighborhood of Los Angeles, just a few days before the release of his latest film, “The House.”

The show that “categorically changed the equation” for him was the FX fantasy football sitcom “The League,” which ran for 6 seasons. Mantzoukas played the recurring role of Rafi, a truly wild, deranged and disgusting creation who fans couldn’t help but love. Mantzoukas might not be a household name yet, but Rafi is.

Mantzoukas is emphatically not a scumbag, but he’s not surprised that people think he might be.

“People predominantly assume I’m that person — that I’m a monster,” he said. “Part of me is like, ‘Great, I’m doing a good job convincing you I am this person and I also have not overexposed myself as who I actually am so you know differently.'”

He frequently guest appears on the improv podcast “Comedy Bang Bang” and co-hosts the wonderfully funny movie podcast “How Did This Get Made” with Paul Scheer and June Diane Raphael, yet he’s still not really being himself.

“I feel like I’m playing the villain of podcasts, the heel of podcasts,” he said. “I like antagonizing the audience, poking at them.”

This persona, the Rafi-effect and his unique look has impacted, and sometimes limited, the kind of roles he’s offered.

“I get offered a lot of scumbags, lots of ‘weird uncle at the wedding,’ lots of creepy massage guys. But listen, I’ve done that to myself,” he said.

His complexion has resulted in him being put up for roles that span a variety of ethnicities, from Greek to Middle Eastern to Hispanic.

“At a certain point I had to be like, ‘I will not do any more auditions with a Middle Eastern accent.'”

In “The House,” out Friday, Mantzoukas plays the third lead to Will Ferrell and Amy Poehler. A scumbag-lite, he’s the guy who convinces them to start an underground casino in their sleepy suburban town. After stealing scenes for years in supporting roles in everything from “Parks and Recreation,” ”Broad City,” ”Enlightened” and “Brooklyn 99” to “The Dictator” and “Sleeping with Other People,” it’s arguably his highest profile part to date. On another level, it’s also just him acting alongside the people he’s been doing improv with for 20 years.

Nick Kroll used to watch him in awe in the early days of the now-legendary comedy breeding ground that is Upright Citizen’s Brigade before they became friends and collaborators.

“He’s a truly gifted improviser. He’s able to on the spot write jokes that are as good as anyone sitting there working on something for three weeks,” Kroll said. “People see him as really fun and dynamic to watch but he’s unbelievably smart and the way he approaches characters and stories is unbelievably nuanced.”

Mantzoukas grew up in the small “Norman Rockwell-ian” Massachusetts town of Nahant. His dad worked in health care and his mom ran a gourmet gift basket business when he got older. And there were no actors in the bunch.

“I’m the anomaly,” he said. “My family is very quiet and shy and reserved in a very funny way when you think about who I am.”

A lifetime comedy fan who learned from “The Carol Burnett Show,” VCR-taped episodes of “Saturday Night Live” that he would watch on Sundays, Monty Python’s Flying Circus, Woody Allen’s “Bananas,” and Elaine May and Mike Nichols albums, Mantzoukas started performing improv in earnest at Middlebury College, and then in New York in the very early days of the Upright Citizen’s Brigade.

“It’s really nice that my generation of the UCB team have all gotten to be in positions where they’re making things. What’s wonderful about the scene is that it is so everyone supports each other,” he said. “The ethos of improv — make your scene partner look good — has produced a generation of people who are very generous to their friends, which is pretty rad in a town that can feel very cutthroat and full of rejection and like nobody is on your side a lot of the time.”

But it wasn’t all serendipity. Mantzoukas has made his own opportunities, by being a go-to on every level — from acting, to writing, to guesting on podcasts and helping friends and peers work through their script problems.

His podcast co-hosts Scheer and Raphael say that he has the “brilliantly attuned mind of a writer.” And all agree he’s the first person you call when you’re stuck on a script.

As far as the future, Mantzoukas is adapting the graphic novel “Battling Boy” for Paramount, and has a film called “The Long Dumb Road” that will likely come out next year. He wants do drama, genre stuff, sci-fi, and maybe even direct something that he writes eventually too. He was, however, disappointed that he was unsuccessful in lobbying for a part in “John Wick 2.”

“I don’t have a threshold. I think of myself as successful now.” he said. “I’m interested in just continuing to consistently work in all the mediums I do.”

LINDSEY BAHR, AP Film Writer