Generations of viewers first got to know Gene Wilder as Willy Wonka, and with a new documentary about the beloved actor, director Ron Frank argues that that’s as good a place to start as any.

The twinkling strains of Pure Imagination open the affectionately wrought Remembering Gene Wilder, transporting the audience back into fond memories of the candyman’s immortal introduction: he hobbles out to meet his adoring public with cane in hand, staggers a bit, starts to stumble, then somersaults himself into a sprightly upright stance. Like so many of Wilder’s finest moments, it was a surprise to his scene partners proving his nimble versatility as performer. He could mine humor from tension, aggravation or anxiety, but his desire to keep the public on their toes always gave way to a welcoming friendliness in his art as in his life.

In surveying the broad strokes of Wilder’s biography, however, Frank’s film acknowledges that his brilliance often posed challenges for which America wasn’t always ready. The spiky take on Roald Dahl’s famed chocolatier enchanted Roger Ebert, whose four-star review declared the adaptation the finest kids’ picture since The Wizard of Oz. But parents bristled at this Wonka, a mercurial oddball delighting in bad things happening to bad apples. “When [Willy Wonka] came out in the 70s, they thought that what it did to children – and Gene outlined this in his own book – was cruel,” Frank tells the Guardian from his home in Connecticut. “One kid disappears in a chocolate tube, another one blows up, one shrinks down. Mothers thought it wasn’t good for their children, and it died in the box office. It was only revived with the home video sales.”

In ways that weren’t always comfortable for the public to accept, the antic and serious coexisted in uneasy cooperation in the work of Wilder, starting from childhood. He often repeated a boyhood memory in which a doctor informed him that with his mother’s heart condition, he had to make her laugh instead of making her angry, or she could drop dead. Wilder grew up under the premise that comedy and pain were close cousins, evident in the development of a screen persona constantly teetering on the brink of a breakdown. (“He was good as the ‘Why is this happening to me?’ guy,” Frank says.) Family stoked his passions along with his nerves, as Wilder set a track for the footlights of New York to follow his sister. “The first time he saw his sister act, he was 11 years old, she was onstage doing a solo act,” Frank says. “The lights went down, the audience applauded, she commanded the stage, and Gene was swept away by it all. He was overcome with emotion and saw himself up there, too.”

He entered the entertainment biz just as the hilarity of neurosis was breaking out of Borscht Belt standup stages and into the mainstream, his combination of shtick and personal dysfunction a hit with an America dipping its toe into therapy and pop-psych. After cutting his teeth on and off Broadway, he landed his first film role in Bonnie and Clyde as a hostage who develops Stockholm syndrome in record time before his captors give him the heave-ho; he played a doctor wrapped up in amour fou with a sheep for nebbish extraordinaire Woody Allen in Everything You Ever Wanted to Know About Sex* (But Were Afraid to Ask). At the same time, beneath the mania, “he had a very sincere gentle quality”, Frank says.

Wilder got to show off both sides to himself with his name-making role in The Producers, the first collaboration with frequent director and lifelong friend Mel Brooks. Frank structures his film with Wilder as narrator, providing voiceover from beyond the grave via the audiobook of his memoir. “I did not want to see another interview-type film with Gene,” he says. “He died in 2016, so we didn’t even have that option. We let him tell his own story, first-person.” In one such anecdote, the Embassy Pictures head, Joseph Levine, ordered Brooks to fire Wilder, who had been deemed off-putting and insufficiently famous. “You bet”, replied Brooks, only to continue shooting with Wilder until they had so much footage, it would’ve been impossible to start over. “That’s how Mel worked,” Frank laughs. “He would listen to producers, nod in agreement, and then not do anything they said.”

In addition to sharing a great love of French wines, Wilder and Brooks gave the world some of the most indelible comedies defining a bawdy, subversive era. Young Frankenstein – thrown together by Wilder’s agent, casting his only three clients in the lead roles – allowed both men to indulge their love of old Hollywood by meticulously recreating it, the studio suits persuaded to allow black-and-white cinematography. Blazing Saddles gestured to a different sector of the past while looking forward, its satire on race relations and the western using now-outmoded language to put forth progressive ideas about prejudice. “I get asked a lot if Blazing Saddles could be made today,” Frank says. “I think it depends more on who the audience is. Some tastes and sensitivities are so different now, that even though it pokes fun at racism, there’s a risk that it might not be seen that way.”

Wilder’s buddy-buddy dynamic with co-star Cleavon Little on Blazing Saddles pointed to the next partnership that would shape his career, with that film’s co-writer and initial lead Richard Pryor. (The executives at Warner Bros declared him uninsurable and insisted on a replacement.) Over the course of four films together, they’d forge a close bond as friends even as Pryor struggled with addictions that often threw wrenches into the production process. Wilder harbored a humanitarian streak that also colored how he chose his roles, as in their third picture together, the crime caper See No Evil, Hear No Evil. He worked closely with the New York League for the Hard of Hearing to portray a deaf man with honesty and sensitivity, and found his fourth wife in lip-reading coach Karen Webb. “[His work] was an extension of his character,” Frank says. “He was a gentle soul. He wouldn’t hurt a fly – literally, according to Karen. He got along with people, fabulously, even ones who were said to be difficult.”

Wilder’s final team-up with Pryor on the roundly rejected Another You also marked his final film appearance, his later years filled by less ambitious projects. He did a short-lived sitcom called Something Wilder and a two-episode stint on Will & Grace, but didn’t much like making TV, finding the pace of shooting more hurried than he’d gotten used to in the days of riffing with his buddies. He found pleasure in writing both memoir and fiction novels, painting, playing music and returning to his roots in the theatre. Never much of a Hollywood guy – he never got a star on the Walk of Fame boulevard – he felt more in his element at the old Connecticut farmhouse he shared with his wife until his death. Frank’s documentary suggests that acting proved most valuable to Wilder as a means of connecting to his fellow human beings, whether on set or through the screen.

“When we talked to Alan Alda, he shared a story about worrying before one of the films he directed came out, what the critics might say,” Frank recalls. “He was commiserating with Gene, who told him, ‘What difference will it make? If they pan the film, so what? Big deal! You made it, it’s finished, it’s over. Be proud of it.’ He shared that comfort with him. Gene knew how to live life well.”

• Remembering Gene Wilder is out in US cinemas on 15 March with UK and Australia dates to be announced

• This article was amended on 14 March 2024. An earlier version stated that Gene Wilder improvised his entrance in Willy Wonka and the Chocolate Factory; it has now been changed to reflect that Wilder planned the scene (his autobiography describes how he brought his idea for it to the director) but without the knowledge of other actors.