John Akomfrah, widely recognised as one of Britain's most expansive and intellectually rewarding film-makers, has never been afraid of a battle. Back in the 1970s, when he was barely out of his teens, he tried to screen Derek Jarman's homoerotic Sebastiane at the film club of the Southwark further education college, where he was studying. "There were rows. Black kids were throwing chairs everywhere. They were saying 'you can't show this'. So we stopped the film and had a discussion: what do you mean, 'We can't show this film'? It was clear there were forms of propriety for black spectatorship. Rather than run back into the field, I thought: let's just accelerate it. Let's push these boundaries a little bit more."

In 1982 he began to do just that, co-founding the Black Audio Film Collective with a group of friends he'd grown up with in London or met at Portsmouth Polytechnic. Over the next 16 years, moving as seamlessly between the form of cine-essay and what they called slide-tape texts as they did between the worlds of the gallery and the newly created Channel 4, they sought to create a new language for migrant cinema. They crafted diasporic films that eschewed social realism and agitprop, took aesthetics as seriously as they did subject matter, and were informed by the canon of non-western moviemakers (Ritwik Ghatak from Bengal, Senegalese Ousmane Sembène, Santiago Álvarez from Cuba), as much as they were by the giants of the Atlantic avant garde.

Not everyone thought this was possible. Akomfrah recalls that when he first approached the Arts Council for money for an avant-garde group, "they told us straight: you can't be avant garde because blacks can't be avant-garde film-makers". Others, such as Salman Rushdie, who wrote a much-debated essay about Handsworth Songs (1986) for the Guardian, felt the collective's work was pretentious and more preoccupied with theories of representation than with representing the second-generation migrants whose rioting had occasioned the film. In the US, according to the New York-based artist and writer Coco Fusco, the collective's work was almost inexplicable. "There was shock, incredulity, negativity. They encountered chauvinism from African Americans who wanted to know: who were these blacks with funny accents? They were thought to be too intelligent for 'the people'."

Akomfrah finds the charge of being an apolitical dilettante absurd. He was born in Ghana in 1957, and both his parents were involved with anti-colonial activism. "My dad was a member of the cabinet of Kwame Nkrumah's party. My mum had met Malcolm X in Accra in 1965. We left Ghana because my mum's life was in danger after the coup of 1966, and my father died in part because of the struggle that led up to the coup. In 1976 my friends and I seriously considered going to enlist in the MPLA to fight in Angola. We wanted to be of use. I'm glad we didn't. The generation before us spoke of going home; we realised what fighting we were going to do in both a literal and a metaphorical sense had to take place here."

For Akomfrah, that fighting initially took the form of politics: "As a kid I went to an Althusserian study group; there were lots of young black kids there trying to get their heads around Althusser!" As a student activist he was a "serial occupier" and sit-in organiser, and consequently got expelled from many FE colleges. But the fight, for him, was also carried on in the realm of the imagination: he regularly sneaked in to see Tarkovsky and Fassbinder films at the Paris Pullman Cinema on the Fulham Palace Road. "I could see all these people were as fascinated with me as they were with the screen. They couldn't work out why this kid was so transfixed."

This commitment to a radicalism both of politics and of cinematic form finds expression in all his films. Seven Songs for Malcolm X (1993) draws on the photographer James Van Der Zee's The Harlem Book of the Dead and Sergei Paradjanov's The Color of Pomegranates (1968) to fashion a probing, internationalist vision of the black radical leader that is far removed from the conventional hero projected in Spike Lee's biopic the previous year; The Last Angel of History (1995) advanced the concept of the "data thief" as part of its argument about science-fiction elements in the music of Sun Ra, George Clinton and Lee Scratch Perry.

Akomfrah's new film, The Nine Muses, is a multilayered, gorgeously shot and affecting work that interweaves archival footage of black and Asian people travelling to and working in Britain with moody, elliptical shots of an anonymous black figure alone in the Alaskan wilderness. Split into nine chapters, each of which is dedicated to one of the Greek muses, and sprinkled with quotations ranging from the Odyssey to The Waste Land, it suggests that stories normally seen through the lens of postcolonialism could just as easily be seen in existential or mythic terms. In doing so, it invites viewers to reflect on the labels by which history – especially diasporic history – is framed and categorised.

"It's important to read images in the archive for their ambiguity and open-endedness," Akomfrah argues. "Migrants were often filmed in relation to debates about crime or social problems, so that's how they get fixed in official memory. But that Caribbean woman standing in a 60s factory isn't thinking about how she's a migrant or a burden on the British state; she's as likely to be thinking about what she's going to eat that evening or about her lover."

One of the most striking features of The Nine Muses is its sound design by Akomfrah's fellow Black Audio founder, Trevor Mathison, which meshes Arvo Pärt liturgical pieces with spirituals and Indian courtly music. The desire to create new kinds of Afro-Asian ambience stems partly from Akomfrah's youthful enthusiasm for post-punk bands such as Test Department and Cabaret Voltaire, which explored the subversive potential of noise. "The avant garde saw our emphasis on the audio as a thought crime, a heresy. It was all about the image for them. They frowned on the sonic, treating it as an impure intrusion into a hallowed field. It was a weird hangover from high modernism, especially as if you watch a Dziga Vertov film you'll see the early avant garde was as interested in sound as in images."

The work of Black Audio and Akomfrah has been increasingly celebrated by the art world recently. Kodwo Eshun, one half of the Turner prize-nominated Otolith Group whose experimental films owe a debt to Black Audio, says: "From the moment I first saw Handsworth Songs it became impossible for me to settle for anything less." Moreover, a new generation of film-makers and students, frustrated by the format-driven orthodoxies of mainstream TV and weaned on the serendipities of YouTube, is also discovering the work.

In the wake of last summer's riots, a screening of Handsworth Songs at Tate Modern attracted a huge audience, which then stayed for a three-hour conversation. "Especially after having spent much of the last 20 years being told by apparatchiks that there's no life in experimental film, it was shocking," Akomfrah says. "These were inquisitive, intellectual magpies confronted by the same questions as we were in the 1980s: how much can we call political, given that the powers that be are saying it's all criminal or without any basis in politics? They believed, like I do, that the moving image has a role to play in galvanising these debates."

• The Nine Muses was released on Friday 20 January.