Two classic New York musicals open with aerial footage of the city. West Side Story comes with an abstract title sequence by Saul Bass and a glorious overture by Leonard Bernstein. The Muppets Take Manhattan features a scat-singing, happy-go-lucky frog. But Kermit and his gang similarly encounter a grimy concrete jungle that can be unpredictable and cruel as they endeavour to make it big on Broadway.

The theatre has inspired arthouse cinema classics by Cassavetes, Bergman and Rivette but I’m ending this series with the Muppets. Before Statler and Waldorf pipe up with the heckles, let me put the case for Frank Oz’s underappreciated 1984 comedy. This is a film that bursts with exuberant hope – just listen to the wonder with which Miss Piggy says the word “Broadway!” – but also has the bittersweet tang of knockbacks. I love the montage that accompanies You Can’t Take No for An Answer, sung by indefatigable jazz messenger Dr Teeth, in which the Muppets spend the summer shuffling around talent agencies and producers’ offices, Kermit wielding his script for a musical and Fozzy clutching a copy of Variety, only to be met with derision. It ends with the rare sight of Kermit losing it with his pals and accepting defeat – before rediscovering his trouper spirit.

The story follows Kermit and co as they graduate from their liberal arts course (Poughkeepsie’s Vassar College was used as a location) and decide to take their school production, Manhattan Melodies, to New York. Hey, it worked for Godspell. Full of dreams but short on cash, they wind up sleeping in lockers at the train station. When the others give up and leave town, Kermit joins the ranks of the city’s musical-theatre hopefuls working in restaurants. He gets a job doing dishes in a rat-infested kitchen, although these rodents are actually hired help, who skate on slabs of melting butter in the pan and swim in the coffee.

The film is both a stagey spin on the plot of The Muppet Movie, in which the gang take on Hollywood, and an extended riff on the average Muppets TV episode, which came with its own weekly backstage dramas. The Muppet Show featured a who’s who of US entertainment stars in guest roles; there are cameos here for Brooke Shields, Elliott Gould and Gregory Hines. At one point even Laurence Olivier had been lined up, along with Dustin Hoffman who was going to spoof the film producer Robert Evans. Instead, we have John Landis as a high-powered agent, Frances Bergen as his receptionist and Joan Rivers giving a makeover to Miss Piggy (both of them in pink frills) in a scene reportedly fuelled by a few bloody marys.

Filmed with extensive use of New York locations, there are appearances by the city’s then mayor, Ed Koch, and the unofficial mayor of Broadway, Vincent Sardi Jr, who founded late-night theatre eaterie Sardi’s. It is here that Kermit, in fake moustache and cravat, poses as a producer and lets loose a team of rats to start a “whispering campaign” promoting his show. He swaps a Liza Minnelli portrait on the wall of fame for his own picture, much to the chagrin of Minnelli herself (yet another cameo).

There are some nice gags about Variety’s distinctive “slanguage”, which Kermit borrows to talk up his boffo socko script, and the film has fun with the art of selling a show much like The Band Wagon did. It shares, too, a fond awareness for the absurdities of the industry. “Just because this whole thing is ridiculous doesn’t mean that it might not make it on Broadway,” observes the producer who puts on Kermit’s show as a favour to his son. That line of satire stretches to Madison Avenue’s advertisers and how the whole city sells its image.

Oscar-nominated for Jeff Moss’s song score (beaten to the award by Purple Rain), the film switches between musical styles including brassy jazz, lush show tunes and, in I’m Gonna Always Love You, doo wop and rock’n’roll for a fantasy sequence that introduced the Muppet Babies and led to their TV spin-off. Jim Henson had always dreamed of a theatrical career alongside his TV and film success, and since his death in 1990 the Muppets have “taken” the Hollywood Bowl and the O2 Arena in London, if not yet the West End and Broadway. I found it hard to watch the film without thinking of the Bad Idea Bears, Trekkie Monster and other residents of Avenue Q, the filthy, puppet-powered stage musical that was a hit in the US and the UK. But it also has a lot in common with the brilliant animated film Sing, in which a koala hosts a talent contest to save his unloved playhouse.

The musical within this musical, shopped around town by Kermit, is about New York City life. “Cops, shootings, car chases?” assumes Dabney Coleman when they pitch it to him. “More like songs and dances,” Kermit explains. “Songs and dances?” Coleman replies, as if the two are subject to rationing. After months of theatre closures, audiences might greet that prospect with disbelief, too. Oz’s film entwines the idealism of Kermit’s show with the optimism that is inherent to the musical genre and that essentially drives any production that has made it in front of audiences – now more than ever. Theatre is a collective act of imagination, uniting those making it and those watching it with a sense of hope. Like the frog says: “The show’s not dead … as long as I believe in it.”

• The Muppets Take Manhattan is on Amazon Prime, YouTube and GooglePlay