In a brief biographical sketch, the crime writer Christopher Fowler, who has died aged 69, claimed he had achieved several of his “pathetic schoolboy fantasies”: releasing an “appalling” Christmas pop single; working as a male model; posing as the villain in a Batman graphic novel; running a Soho night club; appearing in The Pan Book of Horror series; and standing in for James Bond. Rather than examples of Fowler’s wicked sense of humour, all these claims were true. Time Out, meanwhile, called him “an award-winning novelist who would make a good serial killer”.

He was best known for his Bryant & May thrillers, featuring the elderly, venerable detectives Arthur Bryant and John May, who head up the Peculiar Crimes Unit, a department of the London police set up during the second world war to investigate cases that might cause a national scandal or public unrest. The pair had made appearances in some of Fowler’s early novels – Rune (1990), Darkest Day (1993) and Soho Black (1998) – before breaking out into their own long-running series, starting with Full Dark House in 2003.

Bryant, an irascible technophobe, and May, more accepting of the ways of modern life, have been partners for more than 60 years, having first teamed up in 1940 when they were 23 and 19 respectively. The first novel begins with the death by explosion of Bryant; while May investigates, he discovers connections with the pair’s very first case, the killing of a dancer during the blitz, which is told in flashback.

The Gothic air and clashing personalities continued into sequels, each intended to explore a different sub-genre of crime fiction, described by Fowler as “all the devices of classic murder mysteries, including disguised identities, locked room puzzles, surprise ending and nick-of-time rescues”.

The intended one-off novel became six books, then 12 and eventually 20, including two collections of short stories. Central to the series – and many of his other works – was Fowler’s love of London. He found the city endlessly fascinating and delighted in discovering its odder corners and the strangeness of its inhabitants, past and present. “The most bizarre facts in this book are the truest,” he noted in the acknowledgments of The Water Room, the second in the series, published in 2004.

Fowler began writing books alongside a career in the film industry, having co-founded the film promotion agency the Creative Partnership while in his 20s. This gave him the freedom to write what he liked, which made him a publisher’s nightmare: the unclassifiable author. After producing two novels that failed to launch and publishing two humorous titles, How to Impersonate Famous People (1984) and The Ultimate Party Book (1985), Fowler sold two collections of horror stories – City Jitters (1986) and More City Jitters (1988) – to the publisher of his near-neighbour, the novelist and playwright Clive Barker.

He followed this with Roofworld (1988), an urban fantasy novel about battling gangs and secret occult societies who use zipwires and bungee cords to travel above the city – inspired by the interconnected rooftops of London’s densely packed streets and his knowledge of how burglars would break into buildings through the top floor.

Roofworld established some of Fowler’s common themes – warfare between classes, battling evil corporations, and secret worlds existing within touching distance of unsuspecting Londoners – and introduced recurring characters, including DS Janice Longbright, who later served in the Peculiar Crimes Unit.

The real myths and mysteries of London held more interest to him than the supernatural horror of his earlier novels, although that did not stop him writing disturbing and blackly humorous takes on old themes: Rune’s subliminal messages echo MR James’ Casting the Runes; in Red Bride (1992) an ordinary man is seduced by a succubus; and in Spanky (1994) the hero engages in a Faustian pact with a demon.

The family at the heart of Psychoville (1995) are driven out of the suburbs by its change-resistant middle-class community; Disturbia (1996) has its hero uncover the machinations of a secret society; Hell Train (2011) is a homage to Hammer Films; Nyctophobia (2014) is a haunted house story; Little Boy Found (2017, as LK Fox) a psychological thriller; and Hot Water (2022) a murder mystery.

Fowler’s influences included Mervyn Peake’s Gormenghast trilogy (which he called “powerful, black and funny”), John Dickson Carr’s The Hollow Man (“the ultimate in sleight-of-hand”) and JG Ballard (“his ‘five minutes into the future’ books were startling in their prescience”), the latter strongly influencing The Sand Men (2015).

His short story Left Hand Drive was filmed in 1993, and The Master Builder was made into the TV movie Through the Eyes of a Killer (1992), with Tippi Hedren. He won five British Fantasy awards, including for Full Dark House. Other Bryant & May novels won prizes and he was awarded the 2015 Crime Writers’ Association Dagger in the Library award for his body of work. His memoir Paperboy (2009) won the inaugural Green Carnation award.

Fowler was born in Greenwich, south-east London, the son of Lilian (nee Upton), a legal secretary, and William Fowler, a glass-blower and designer of scientific instruments with whom Fowler had a troubled relationship. He grew up reading fantasy, horror and science fiction, graduating from Superman comics to Tolkien and Ballard in his teens.

He was educated at Colfe’s grammar school in Lee before enrolling to study art at Goldsmiths College in 1972, although he left to take up work as a copywriter in various advertising agencies. At the age of 26 he set up the Creative Partnership with the producer Jim Sturgeon to market films, becoming a one-stop shop producing trailers, posters and commercials for radio and TV.

The company worked on campaigns for Bernardo Bertolucci, Peter Greenaway, Ridley Scott, Quentin Tarantino, David Cronenberg, Mike Leigh and many others, on movies ranging from Alien – they were paid £20 for the line “In space no one can hear you scream” – and The Cook, the Thief, His Wife and Her Lover, to GoldenEye and Reservoir Dogs.

The Creative Partnership opened an office in Beverly Hills and Fowler spent the early 80s in the US, but could find only badly paid, uncreative work. With time on his hands he began writing, finding success after returning to London. He was able to maintain a steady output, including a graphic novel (Menz Insana, 1997), a stage play (Celebrity, produced at the Phoenix, London, in 2010), and the War of the Worlds video game (2011) for Paramount.

He also wrote a 319-episode series of short essays on forgotten authors for the Independent on Sunday, partly collected as Invisible Ink (2012) and The Book of Forgotten Authors (2017). Essay 319 featured Christopher Fowler, described as “a typical example of the late 20th-century midlist author ... [whose] ability of turning his hand to most literary forms granted him the honorary title of ‘Wordslut’”.

Fowler was diagnosed with a rare form of cancer in March 2020. He worked on his 20th Bryant & May novel, London Bridge is Falling Down (2021), while undergoing chemotherapy in hospital, and the following year published Bryant & May’s Peculiar London, in which his characters explore the true life eccentricities and elusive byways of the city. His third volume of memoirs, Word Monkey (following Paperboy and Film Freak, 2013), is due out in August.

Fowler formed a civil partnership in 2007 with the TV executive Peter Chapman, and 10 years later they married. Peter survives him, along with his younger brother, Steven.

• Christopher Robert Fowler, author and screenwriter, born 26 March 1953; died 2 March 2023