John McCain was being filmed on the porch of his home in Sedona, Arizona, looking happy and relaxed despite the mortal battle he is waging with brain cancer, when he uttered the words that are certain to go down in the annals of American political history. “I should have said: ‘Look, Joe Lieberman is my best friend, we should take him. But I was persuaded by my political advisers it would be harmful, and that was another mistake that I made.”

Peter Kunhardt, an Emmy-winning director, was inside the house at that precise moment sitting with McCain’s wife Cindy and daughter Meghan, who were watching the taping as it was being relayed live on a TV monitor. “They actually gasped when he said he wished he had picked Lieberman,” Kunhardt said. “After the filming, Cindy came up to us and said that was the first time he’d ever said that.”

It’s not so much that McCain had finally admitted regret over his decision to forgo choosing Lieberman to be his running mate in the 2008 presidential election. It was, after all, an unlikely pairing – McCain was the Republican nominee and Lieberman was at that time a Democrat.

It was what flowed from that fateful decision that made the comment so riveting: the rise of Sarah Palin. To choose the gun-totin’ mama grizzly from Alaska as his alternative vice-presidential pick arguably did more than anything to let the populist genie out of the bottle, unleashing the terrifying consequences that we are still wrestling with today.

The on-camera confession is the most electrifying part of John McCain: For Whom the Bell Tolls, the upcoming HBO documentary that Kunhardt has directed with his sons George and Teddy. The family trio spent six hours interviewing the US senator from Arizona – three in Sedona, three on Capitol Hill – and with the footage have put together a retrospective of his life on the verge of death.

It makes for a strange cinematic animal. The Kunhardts stressed that they consciously set out to locate the film in the present tense – “We wanted to capture the living John McCain, not the dead John McCain,” as Teddy put it. But McCain’s diagnosis last July of a deadly brain tumor inevitably hangs heavy in the Sedona air, giving the documentary the feel of an obituary produced and broadcast while the subject is still with us.

That slightly awkward sense of an eulogy to a living man comes across most strongly from McCain himself. Early on in the film the 81-year-old makes a grand gesture direct to camera, saying: “I have lived an honorable life, and I am proud of my life.” He sounds eerily as though he were dictating his own epitaph.

The idea for a film came about with lightning speed after McCain’s diagnosis with glioblastoma, a very aggressive form of brain tumor, was made public. Teddy Kunhardt proposed it on the same day as the announcement, and they took it days later to HBO, who immediately gave it a green light.

Days after that they pitched it to McCain himself and he went for it immediately. “John is cognizant this is a terrible disease and he wanted to get his story down as much as we wanted to tell it. When we approached him there was an immediate, ‘Yes, let’s do it and let’s do it now’,” said Teddy.

The father-sons team are unapologetic about the favorable light in which they cast their subject – literally so: they filmed him under Sedona’s famously soft natural desert light. Though they gave McCain no editorial control over the documentary, they openly hail him as an “American hero” and buy uncritically into the mythology of the “maverick” politician that McCain has studiously cultivated over many years.

Certainly, the case for John McCain as hero and maverick can be made, and the Kunhardts do so over almost two hours of entertaining viewing. No matter what Donald Trump may say about it, McCain’s fortitude over five years of torture and prolonged solitary confinement in the “Hanoi Hilton” as a prisoner of war in Vietnam had heroic qualities.

At several junctures in his political career he resisted peer pressure within the Republican party and went his own “maverick” way. Even this month, while sick at home in Arizona, he opposed the confirmation of Gina Haspel as director of the CIA given her refusal to disown torture, prompting the Trump White House to stoop to new lows when an aide said his view could be discounted as he was “dying anyway”.

The senator’s political friendships too reflect a willingness to cross the aisle that these days seem almost quaintly from another age. The supporting cast on the HBO documentary makes the point – Joe Biden, Bill and Hillary Clinton, John Kerry, Joe Lieberman, Barack Obama: his Democratic admirers outnumber the Republican.

“When we made the initial list for the film we were surprised by how many Democrats there were,” Teddy Kunhardt said. “It hit home his point of bipartisanship: Republicans or Democrats, they are not your enemies.”

At times McCain should be credited with both acting heroically and as a maverick. Take 25 July last year, just days after his diagnosis, when he ignored doctors’ warnings and flew back from Arizona to the US Senate, the blood barely dried in an open gash from surgery above his left eye, to cast his vote on a Republican plan to scrap Obamacare.

The documentary lingers over that exceptional moment to powerful effect. It follows McCain as he received a standing ovation entering the Senate chamber. Then it pans in as, later that night, he stood in front of his stony-faced colleagues, held out his right arm for what seems an aeon, then let his thumb drop to confine the Republican bill to the dustbin of history.

There were audible gasps on that occasion too.

But the problem with this kind of glowing eulogy – whether composed before his death or after – is that it obscures, amid the adulation, deeper and in some regards darker aspects of the individual and the life.

McCain’s conception of heroism is in itself inextricably linked with his embrace of militarism. He uses the language of “sacrifice” and serving a “greater cause” – a worldview handed down to him by his naval father and grandfather.

Describing his captivity in Vietnam, McCain explains his stoicism to the Kunhardts thus: “McCains were doing what McCains were bred to do, and if it brings you into harm’s way, that was our profession.”

“He is the representative of one type of heroic American character,” said Peter Kunhardt, whose film-making career has specialised to some extent in profiling major American politicians including Abraham Lincoln and John F Kennedy. “John McCain is distinctive in that he takes this code of conduct from his training in the military and he overlays it with his own personal view of how to serve as an American.”

But there are consequences. In foreign policy terms his default position is that of uber-hawk. After his release from Vietnamese captivity in 1973, he was one of a dwindling band of those who backed Richard Nixon in stepping up bombing of Vietnam and Cambodia.

He continued the theme with firm support for George Bush’s reckless invasion of Iraq, going on to remark in 2008 that if US troops stayed inside the country for 100 years that would be “fine by me”. More recently he has struck similarly bellicose postures on Syria and North Korea.

In his politics at home he has also displayed critical weakness when he most needed to tap that legendary heroism. In his first presidential run in 2000, he blatantly pandered to racism to save his own electoral prospects – he told voters in South Carolina that he regarded the Confederate battle flag then flying from the state capitol as an acceptable “symbol of heritage”. (He later reversed his position, but by then the damage had arguably been done.)

A similar pattern of political expediency emerged in his second and final run on the White House against Barack Obama in 2008. In that race he shed his maverick skin and conducted himself as a conventional grubby candidate prepared to do whatever it took to get elected – on tax cuts, immigration reform, abortion and other key issues he abandoned earlier principled stances to take the populist route.

The Kunhardts said that they addressed key moments in which McCain failed to meet his own moral standards as a theme in the film. “We wanted to show his past failures and how he hit them head on and learned from them and moved forward – the idea that you can recognize what you’ve done wrong and try and do right,” Teddy said.

Which brings us neatly back to Sarah Palin and that controversial VP pick.

On 29 August 2008 McCain stood in front of a large crowd in Dayton, Ohio, and introduced “the next vice-president of the United States, Governor Sarah Palin of the great state of Alaska”. The HBO documentary lingers over this moment as well, zooming in on the man in what must have been a singularly lonely instance.

The scene is terrible to behold. There he stands, his face locked in an unconvincing smile, like a child desperately feigning delight as he opens an unwanted birthday present.

“I don’t think he could have known it at the time,” the New York Times columnist David Brooks tells the Kunhardts. “But he took a disease that was running through the Republican party – anti-intellectualism, disrespect for facts – and he put it right at the center of the party.”

That’s an important caveat as we assess John McCain on the point of his own mortality. He can plausibly claim to represent the antithesis of the current incumbent of the White House: he is as open-minded as Trump is partisan, as public-spirited as Trump is self-serving, he speaks honor to Trump’s pussy grabbing.

But he also has to face tough questions that the Kunhardts pointedly skirt around. Did he do enough when he effectively led the Republican party to hold back the dark forces welling up within it? Must he accept a share of blame for the ugly, ominous world he will probably soon leave behind?

• John McCain: For Whom the Bell Tolls, will debut on HBO on 28 May with a UK date yet to be announced