With his new film, Charlie Kaufman again proves that if you want something to make you feel trapped in a terrifying claustrophobic nightmare for ever and ever ... well, he’s your guy. His latest movie is about a student, played by Jessie Buckley, who for six weeks has been dating a dullish man called Jake (Jesse Plemons) for reasons that that she can’t put her finger on. She is now going to meet his parents, an important next step that she has sleepwalked into, like everything else in their relationship.

It means a long, uncomfortable, cold drive out to these old folks’ creepy farm in the countryside – just at the moment when she realises that she maybe wants to end things with Jake, which gives her an anti-epiphany about her life and his. (“I’ve never mentioned Jake to my parents and I guess I never will,” she murmurs silently.)

She has a clear, dispassionate glimpse of someone whose existence she really isn’t committed to and also of their supposed future lives together: a pointless, arbitrary entanglement, like all coupledom. A warning and despairing voice whispers in her head that all this is pointless but also breaking up could be pointless, too. Should she be ending things in a more radical way? Soon she is to be vouchsafed a haunting vision of the things that are important to poor, boring Jake – his mum and dad (wonderfully played by Toni Collette and David Thewlis), a local ice-cream store and his high school, whose sad old janitor (Guy Boyd) haunts this movie’s dreams.

I’m Thinking of Ending Things is adapted by Kaufman from the novel by Canadian author Iain Reid and it’s really scary in a way that conventional scary movies really aren’t scary: insidiously disquieting and yet also somehow poignant and sad, a secondary mood that finally, inexplicably emerges from an unending rhapsody of directionless weirdness. Who is this woman? Who is Jake? Who is the janitor? It’s an arresting drama, although I was sorry that straight-up comedy (which Kaufman can really do) played little part.

This film happens to be coming out right on top of Kaufman’s other new work – an epically bizarre autofictional novel called Antkind – and there are points in common, namely, Kaufman’s extraordinary, compulsive way of creating an alternative world of loneliness and sadness, accumulating tension and anxiety and suppressed panic minute by minute. You can spend up to an hour wondering uneasily when this film is going to start, while also realising that you have been on the edge of your seat.

As ever, Kaufman shores up his world with pop-culture fragments. Real people get roped in. He has a digression-daydream featuring a made-up romcom directed by Robert Zemeckis. Are we supposed to look down on Zemeckis, therefore? Not necessarily. The couple’s road trip also features a long and rather bad-tempered discussion of the film A Woman Under the Influence by John Cassavetes, although the directors that have surely influenced I’m Thinking of Ending Things, such as Buñuel and Lynch, go unmentioned.

The centre of the horror is that farm, stark and cold, where Colette and Thewlis’s strange pair host a decidedly unrelaxed supper party for the unhappy couple. As their appearance starts changing, with the furnishings and photos changing, too, Buckley starts politely asking Jake if they can go home now, and Kaufman demonstrates his unearthly skill in creating scenarios like lucid dreams. Molly Hughes’s production design and Łukasz Żal’s cinematography are also vital in fabricating this world.

And what does it all mean? Kaufman is content to swirl away from actual revelations, but his evocation of high-school life occupies the position where in any other kind of fiction an explanation might go. It is a place where Jake seems to have picked up an uncool fondness for musicals and Broadway show tunes, and Kaufman imagines a kind of fantasy dance sequence for Jake and his new girlfriend in the corridors of the school. But perhaps it is more that for many (or maybe even everyone) high school felt shatteringly real in a way that nothing else would, and success there really does feel as good as winning the Nobel prize. Not that Jake has ever achieved anything of the sort, then or now.

Plemons stolidly occupies the role of Jake – it is the kind of role that the late Philip Seymour Hoffman (star of Kaufman’s masterpiece, Synecdoche, New York) might also have played. And Buckley is impeccably cast as the young woman adrift in a bad dream someone else is having.

• I’m Thinking of Ending Things is on Netflix from 4 September