With a sigh, Ahmir Thompson – better known as Questlove – turns his laptop around, so I can see the inside of his apartment, rather than the beautiful view of the New York skyline through the window behind him. It is a chaos of overflowing boxes and furniture covered with papers. “An ex-publicist of mine decided that they didn’t need their 8x10 photographs and old articles from the NME any more, so they gifted them to me,” he shrugs.

Thompson seems equivocal about this state of affairs. On the one hand, he can barely contain his delight: “Look at this!” he enthuses, showing me a newly acquired invite to the 1984 premiere of Prince’s Purple Rain movie. But, on the other: well, look at the place. “People are saying: ‘I got kids, but they won’t care about this stuff like you will. If this needs to go in a museum or something, I can trust you with history.’ The universe has put me in the position of keeper of the record. So, you know, be careful what you wish for.”

This is apparently what happens when you direct one of the most acclaimed music documentaries in recent memory. Summer of Soul, 2021’s Oscar-winning exhumation of forgotten footage of the 1969 Harlem Cultural festival, was a film that also had stern things to say about how African American culture is commemorated and curated. “We hold these truths to be self-evident – that Black history is going to be erased,” says a voiceover near the film’s conclusion. In its aftermath, Thompson says, he became, by default, a “historical hoarder”.

“I had a normal record collection,” he says, although it’s worth noting that, by “normal”, he means a mere “60,000 to 70,000 records. But one by one, radio stations started digitising everything, because they wanted to make more space, and they were like: ‘What do you do with the physical copies? Throw them away?’ So I started getting calls from this jazz station in Alabama, or this former soul station in West Virginia: ‘Hey man, we got 12,000 pieces that are probably going to go in the garbage …’ I’m up to 200,000 records now. I have four gargantuan storage units.” He looks pained. “Just to insure those records, and find someone to categorise them, is almost like buying another house. I don’t know if it’ll get done in my lifetime.”

Which brings us to the other thing that happens when you direct one of the most acclaimed music documentaries in recent memory: you become incredibly busy.

Thompson is an incurable music nerd. Over the course of an hour, our conversation covers everything from free jazz to his love of knotty British prog rockers Gentle Giant, to the way you can divide Michael Jackson’s influence on the choreography of other pop stars into those who saw him before the arrival of the video recorder and those who encountered him afterwards (he thinks the moves of the latter category are more precise and mechanical, as a result of pause and rewind footage of Jackson in full flight).

He has also long been noted for a painstaking more-is-more approach to his work. His band, the Roots, were famed for playing live shows that often stretched over three hours. A blithe decision to start live-streaming themed DJ sets during lockdown became all-consuming: he would spend days “taking meticulous notes of every song I know, then putting them into categories”. Charged with overseeing a 13-minute Grammy awards tribute for the 50th anniversary of hip-hop, he put so much effort into getting it exactly right that the resulting stress caused his teeth to start falling out. But after Summer of Soul’s success, he says, “it’s literally just been raining projects from heaven … I was like ‘Yes. Yes. Yes. Yes. Yes. Yes.’”

The net result is a work schedule so intense it makes your head hurt just thinking about it. There is the Roots’ five-nights-a-week gig as the house band on Jimmy Fallon’s talkshow, a role they’ve held since 2009 (although they were nearly fired early on for playing an instrumental version of Fishbone’s Lyin’ Ass Bitch as walk-on music for right-wing politician Michele Bachmann). Their annual festival, the Roots Picnic, is this weekend; and a new Roots album, their first since 2014’s …And Then You Shoot Your Cousin, is being worked on. There is his podcast, and his DJing – he estimates that over the last 10 years, he has DJ’d for “at least 40,000 hours” – and the six film projects he’s currently involved with directing. One is rumoured to be a live-action remake of Disney classic The Aristocats, but the only one he’s allowed to discuss is a nearly completed documentary about Sly Stone, “a 20,000-piece puzzle that took a lot of meticulous tightrope-walking to put together”, that delves not just into his music, but his capacity for self-sabotage.

“Woodstock turned Sly Stone into a household word, and the first figure in Black entertainment to achieve that level of celebrity post-civil rights movement,” he says. “Which makes his celebrity different to that of Ray Charles, James Brown, Chuck Berry, where they actively knew what segregation was while they were famous – you know, James Brown can’t eat at the supper club he’s performing in. But what happens when you get everything you ever wanted? It hit me that Sly Stone is the first domino of a billion dominoes of self-sabotage moves that happen with artists, sometimes intentional, a lot of them unintentional or subconscious. Aside from being a musical pioneer, I wanted to know whether or not his life is also the blueprint for every Black artist that goes through that obstacle course and doesn’t make it out the other side.”

And there’s also his latest book, the eighth in a bibliography that includes everything from cookbooks to bestselling explorations of the meaning of creativity, to young adult sci-fi novels. Co-written with his regular literary collaborator Ben Greenman, Hip-Hop Is History melds a detailed chronological retelling of the genre’s story with occasionally hair-raising memoir, recalling the likes of the 1995 Source Awards, when the East Coast-West Coast rivalry that eventually claimed the lives of Tupac Shakur and Biggie Smalls first erupted into public view and the Roots found themselves uncomfortably seated directly between the two warring factions.

There’s also some impressively original theorising. He views the Wu-Tang Clan as a pop band, gushing out the kind of culture that “latchkey Black kids” of their generation absorbed through endless afternoons watching TV; and views the mush-mouthed, drugged-out “SoundCloud rap” of the 2010s as the sound of young Black America anaesthetising itself as the promise of the Obama years dissipated.

He says he was driven to write it by a sense that hip-hop had struggled to make sense of its own history, partly because no one involved in its early years was much interested in posterity. “Its initial participants entered this culture in an atmosphere of fight or flight, survival hustle, ‘by any means necessary, I have to make it through the next 24 hours’. It’s not like I got into this because it was my only way to survive – it was a way to get out of what my parents wanted me to do, which was go to some classical conservatory. But when the Roots started, we had to busk on the street corners to get attention. We had to move to London, where at least the idea of a band wasn’t like a novelty idea, with three gigs booked at the Jazz Café and enough record label support to last five days. People were living for the day, living for the moment, not thinking about history. So I stress to artists now: dude, every sketch pad, every direction you take from a telephone, save all your history because you never know when you’re going to need it.”

Hip-hop grabbed Thompson from the start. He was already a music fan and, indeed, a drummer – his parents were both musicians – but hearing Rapper’s Delight on the radio as an eight-year-old changed everything. Characteristically, his immediate reaction to the bizarre sound of “someone talking on a record instead of a melody” was to rush for his cassette recorder to tape the remainder of the song, then sit up half the night memorising the lyrics. “By lunchtime the next day I was a celebrity at school – I was performing it for girls who were impressed, and that’s when I realised the magic of: ‘I perform and people pay attention, and that’s a good feeling.’ It’s kind of weird how Rapper’s Delight wound up planting a seed for the rest of my performative life.”

Video recorder or not, you could never match Michael Jackson’s genius – but there was something attainable about hip-hop. “Run-DMC,” he nods. “Before then, the idea of a star was peacocking; you were supposed to wear something outlandish. Parliament did not look like my uncles, the Temptations didn’t, Sly and the Family Stone, none of them. Whereas the idea of wearing Adidas was obtainable. You could buy a pair of shell toes, some black Lee jeans and a black T-shirt and you can look like Run-DMC.”

He doesn’t see hip-hop as this attainable today. “In 1997, hip-hop realises, 20 years into its existence, ‘Oh, we are now the establishment that we were once against’,” he says. “And suddenly, the narrative was ‘Hey, this is my lifestyle and you don’t have access to it’. So it became like an envy thing.”

It’s not the first time Thompson has expressed reservations about the direction in which hip-hop is moving: Hip-Hop Is History is liberally sprinkled with moments when, confounded by a new album or sub-genre, he wonders aloud whether this music is still for him, before invariably reconsidering.

It happened recently, he says, with the beef between Drake and Kendrick Lamar: the day a glut of diss tracks appeared, he found himself sitting up at 1am, playing them at “twice slower speed” on YouTube and checking the lyrics online. “You know that moment in the movies, when the old guy’s at the club and he’s like: ‘Maybe I shouldn’t be here?’” he laughs. “I’m not trying to be hyperbolic or anything, but I thought: am I done? The way my dad was done when he heard Stevie Wonder’s Journey Through the Secret Life of Plants: we used to binge-shop records and listen to them together, him schooling me on music. But when he heard that album, he laid down his sword and shield, and just walked away.”

He frets about “the mud-slinging angle” of the Drake-Kendrick saga: “I’m not trying to be all moral; I know the sphere and I love them all. But if I were controlling the narrative, I would have loved to see this literally be, like: ‘Alright, I’m going to go write the song that will make me a lyrical god, here’s my skill level, you don’t even have to be the subject of it.’”

But of course Thompson won’t actually walk away, any more than he can turn down the radio stations’ records or the publicists’ archives or the offers to make more documentary films. Not long after we speak, he posts his thoughts on the Kendrick-Drake beef to Instagram – essentially a more forceful version of the reservations he expressed to me. It’s enough to cause controversy in the comments section, to which he responds, then invites people to comment further. Ahmir “Questlove” Thompson is in the thick of it once more – which is, you suspect, where he prefers to be.

• Hip-Hop Is History is published by White Rabbit Books on 11 June. The Roots Picnic is at Fairmount Park, Philadelphia, 1 & 2 June

• A picture caption was amended on 31 May 2024 because an earlier version misnamed the film the Summer of Soul as the Summer of Love.