Josh Brolin is accustomed to man-of-action roles. In his last film, “Oldboy,” he handled interpersonal disputes with a blood-spattered claw hammer. His latest, “Labor Day,” repositions him as the lead of a love story. In his big action scene, he sinks his hands into a bowl of sliced, sugared peaches and bakes Kate Winslet a pie.

“I think I’ve missed out on something, because I enjoyed it thoroughly,” Brolin said in an interview after the film’s debut at the Toronto International Film Festival. “It’s weird, man. You start to get a complex after a time. It’s like, am I not the handsome type? Do they want me for all the ‘Aarghhh’?” he said, putting on a menacing expression.

“This seemed like a good hybrid between the two,” offering him a role as a formidable, mysterious escaped prisoner with a wide romantic streak. “I couldn’t hope for a better type of movie to have a romance in.”

Jason Reitman, who adapted Joyce Maynard’s novel and directed, presented Brolin with an unusual challenge. Reitman continually told him to rein himself in.

“It was an uneasy time, even though we had a blast,” he said. “It was unnerving. Kate and I had a similar question about, ‘Will we fill every moment?’ ” with actorly business. “Jason was all about bringing it back, bringing it back. Kate and I both give 1,000 percent. We huddled up together in our insecurity. Not against Jason, but just like, ‘This is so uncomfortable sometimes.’ ”

“I just worked with Paul Thomas Anderson [on the film of Thomas Pynchon’s psychedelic private-eye romp “Inherent Vice”], and it was just as satisfying but totally chaotic. On the edge, walking this cliff line all the time. Whereas with Jason, it’s a much more still experience. He was always tweaking me back to doing less and doing less and doing less.

“As an actor, or maybe it’s just me, the paranoia was that it was just going to be boring as all hell. That people are going to say, ‘I love what they’re doing, but why is he not moving?’ I’d move my hand like that,” shifting it an inch, “and the next day he’d say, ‘Don’t do that with your hand.’ ”

Ultimately, he realized that Reitman wanted him to be still because he trusted “the light in the eyes” to express Brolin’s character.

“It was more exposing for me than I’ve ever felt. It’s like doing an interview with you if I just stood here,” he said, standing up, “and didn’t move and answered your questions and all that. I can’t do that. Especially me,” he said, returning to the sofa. “I talk a lot. I’m animated.”

With his director limiting his gestures and advising him to tone it down, “I think I was working much harder interiorly than I normally would.”

What most intrigued him about Maynard’s novel, he said, was that the menacing convict was “elusive. There was always this feeling that this guy could turn around at any moment, that it’s all been a manipulation. Thinking about things like that [while in character] is important, especially if you’re not doing anything. Otherwise it’s dead eyes and you’re thinking about lunch. What’s it going to be today, lobster or steak?”