This summer, the East Hampton mansion Grey Gardens is 121 years old – exactly half the age of America itself. Its 28 rooms have welcomed the titans who’ve shaped the country: coal barons and newspaper titans and, most famously, the wildling relatives of Jackie Kennedy. Edith Ewing Bouvier Beale and her daughter Edith Bouvier Beale moved into Grey Gardens in 1924 and stayed there for over five decades. Outside, the United States was upended by massive cultural shifts: the Great Depression catapulting into the second world war; the civil rights protests and Haight-Ashbury fever sweating out its fury into disco decadence. Inside, wayward vines blocked out the sun – really, the world – as Big Edie and Little Edie gathered cats, fed raccoons and curled into themselves, abandoning the uninhabitable parts of the house room by room until they were crowded into just three. The two beauties moldered, as did the walls around them.

Until June 1972 – the That Summer of Göran Olsson’s new documentary – when the Studio 54 era came to them. Big Edie’s niece Lee Radziwill (Jackie Kennedy’s sister) and her photographer boyfriend Peter Beard vacationed at the beach with their famous friends such as Truman Capote and Bianca Jagger. (Andy Warhol, too, in one of his rare jaunts out of the city, seen nervously peering at the sun from underneath a wide-brimmed hat.) Radziwill and Beard visited Grey Gardens with the vague idea of making a documentary about the history of the Hamptons, perhaps narrated by, as Radziwill puts it, “my extremely eccentric aunt”.

To help, Radziwill hired Albert and David Maysles, two brothers she knew from the Rolling Stones scene who’d just shot the concert film Gimme Shelter. Over a couple weekends, they shot an hour of Big Edie and Little Edie squabbling with each other and the health officials threatening eviction. Ultimately Radziwill saved the house and scrapped the movie. Rumor was she was embarrassed by the evidence of the Bouvier clan’s disintegration. “No, not at all,” deflects Olsson, on the phone from Switzerland. “They were super busy. They went to Europe and Africa, they did exhibitions, they didn’t know exactly what to do about it, and they had other things on their mind.” As for the Maysles brothers, they’d return to Grey Gardens for the 1975 documentary (called Grey Gardens) that would make them, and the Beales became legends in a film so groundbreaking it announces its own existence in a newspaper clipping in the opening credits.

That Summer, Olsson’s new film, represents that lost footage. Consider it a Grey Gardens prequel. Big Edie is more physically and mentally intact; her home is more trashed with cat urine and rot. We see the inspectors barge in brandishing court papers as Radziwill frets over what to save. The rattan furniture, yes. Anything upholstered, absolutely not. And there’s Little Edie – fashion icon, delirious dancer, homebody goddess – leaning into the camera with never-before heard pronouncements, delivered in that urgent whisper that seems to bore through the lens and into your skull. “Cats don’t fit into having a house redecorated,” she hisses. And later: “I think any of us would be happy to have raccoons who look upon us as friends.”

“The Beales really paved the way for YouTubers – I certainly know that they paved the way for the Kardashians,” says Olsson. “Documentary has always been way ahead of the mainstream media or social phenomena.”

True, but this deeper excavation of Grey Gardens still seems like a surprising fit for the Swedish-born filmmaker of the Sundance award-winning documentary The Black Power Mixtape 1967–1975, which charted the rise of black nationalism as an army fighting three wars: Vietnam, poverty and heroin addiction. In those same years, the Beales were, well, at home fighting over lipstick and who needs to feed the cats. Same America, different universes.

Yet to Olsson, both films are about freedom – and they’ve always been intertwined. As a boy growing up in Lund, he’d bicycle from his anti-apartheid club meetings to the public library to flip through Interview magazine. “Everything about the art scene and music scene in Manhattan in the 70s, Bianca, Halston, the Factory, Max’s Kansas City, it was an obsession of mine, and I was totally alone in that interest,” says Olsson. When Peter Beard invited him to see his old Hamptons tapes and photographs, which open That Summer and sets up the spotlight everyone in the film is, or will be, under, Olsson was delighted that he recognized everyone in the background. Finally, his childhood fixation paid off. “I know that I romanticize it, and that it was not true because of the drugs and the economic problems, but still to me it’s the greatest place in terms of tolerance, where people could blend, meet other class, and gender, and ethnic backgrounds.”

Freedom eludes Little Edie, who spends That Summer and Grey Gardens yowling her desperation to escape to New York. Cut into four raw reels, this early footage of the then 55-year-old stings. Big Edie nags her to wear makeup, stop crossing her arms and change her costumes every two hours so she can look pretty for the construction workers tramping through the house – potential husbands we know she’ll reject. In a startling aside, the two chat about Little Edie committing “incest” with her Uncle Jack. Big Edie shrugs: “Well, you had to find out about men sometime,” and the conversation charges on, cross-talk, arguments and interruptions in dissonant harmony. In the film’s cruelest line, Big Edie cackles: “The only vermin was you, Edie!”

Watching That Summer, I thought of Albert Maysles’ 2014 Grey Gardens Q&A where he said: “I hope that most people who see it are shocked by it and don’t want to see any more.” Or as Little Edie says here: “I think its very cruel to dig up the past.” Yet, we do want to see more – so much that Grey Gardens has already been spun into a TV movie, a play and a musical.

“I’m not saying they’re exploiting them in Grey Gardens, but I think a beautiful thing about this footage is that Lee is there,” says Olsson. Radziwill isn’t just the link between the Beales and the Maysles – she’s the last in a line of people who knew them as more than kitsch icons. She’s That Summer’s soft-spoken, camera-indifferent, stabilizing force – she gets the electricity and the hot water working, she charms the inspectors into leaving them alone, and she reminds you that the Edies were genuinely loved. In a casual moment, one of the Edies reaches out to stroke her hair. “It’s so nice, it’s so intimate,” says Olsson. “It’s an organic connection to this environment.”

Radziwill didn’t participate in That Summer, though she did allow Olsson to use an audio interview she’d previously recorded with Sofia Coppola. Olsson shrugs it off as “bad timing”, though it’s hard not to wonder if she was spooked by her aunt and cousin’s example that audiences are startled to see glamorous women age. (Though Radziwill remains stunning.) But This Summer adds her quiet, steely spirit to home already “oozing with romance, ghosts and other things,” as Little Edie says in Grey Gardens. This mansion still has corners to explore.

• That Summer is released in the US on 18 May and in the UK on 1 June