Watch Barack Obama's speech on race

Tony Peterson was visiting his younger brother, Keith, in Boulder, Colorado, in the late 1990s. They were in a bookshop together when Keith plucked a paperback out of the remainder bin and said: “I’m buying this, do you want one?” Tony looked at the book, which was called Dreams from My Father. It meant nothing to him, but his brother said: “Well, look who wrote it.” The author’s name was Barack Obama. “That’s Barry!” said Keith.

In the spring of 1976, Tony and Barry had been pupils together at one of America’s biggest private schools, Punahou, on the Hawaiian island of Honolulu. The school boasted 76 acres of green lawns and woodlands dotted with low buildings topped by squat roofs. There were 3,600 pupils and, that year, only four of them were black males. It was an elite school that pupils from poorer families, no matter how gifted, could only access with the help of the school’s “financial aid” endowments. Barry had been one of them. Everyone knew him as Barry. Tony had never even known his real name.

In this state of racial isolation, Tony and Barry and one of the other black pupils, Rik Smith, had begun to meet once a week on the steps outside the attendance office, where they would sit and talk. Nearby was the senior bench, a stone seat that encircled the ancient Banyan tree, where athletes and their acolytes, including cheerleaders, would gather to see and be seen. Tony, Barry and Rik could not sit on the senior bench unless they had been invited, but here on the steps they could sit and listen to the attendance-office radio playing the songs of Elton John, the Doobie Brothers and the Eagles.

As Tony recalls after 30 years, their talks were rarely earnest. He could not remember Barry expressing the agonies of racial identity, the struggle to become “a black man in America” that he would write about later in that book. They used to talk about all kinds of things. In March 1976, Tony had tape-recorded one of their conversations for an English project. He still had it in March 2008.

Rik: “Have you guys ever thought about time?”

Barry: “Yeah.”

Tony: “I thought about it.”

Rik: “Think about time, okay. What is it? What is time?”

Tony: “I don’t know.”

Barry: “Eh. Time is just a collection of human… listen, this is gonna sound good, boy! See, time is just a collection of human experiences combined so that they make a long, flowing stream of thought.”

Barry was 14, younger than the other two.

He was bright, Tony recalled, but not especially charismatic, so Tony was amazed later to see how magnetic his friend had become, watching him on TV. They talked about girls and about race. Would the white and Asian girls date them, those black boys? (They would.) Why was a young black man with a book perceived to be “acting white”? Would there ever be a black president of the United States? Not in their lifetime, they concluded. They couldn’t imagine it.

Tony was watching television in July 2004 when Barry, by now a senator, still barely known outside his immediate Chicago circle, stepped up to the podium to give the keynote address to the Democratic National Convention. The speech is now ranked alongside the stirring political oratory of Kennedy and Martin Luther King. Obama evoked the civil-rights era. He spoke of the need to support the inner cities and “eradicate the slander that says a black youth with a book is acting white”.

Tony sat there listening and swelling with pride, hearing echoes of their conversations from all those years ago. That was the guy he knew, the same old Barry. It was an audacious speech. Tony just hoped his friend knew what he was doing.

Beyond the tourist beaches and the skyscraping hotels of Waikiki, Hawaii is a melting pot of Japanese, Chinese, Korean, Filipino, Samoan, native islanders, Caucasians and various others, known in local slang as hapa – of mixed heritage. Only African-Americans are missing in significant numbers.

A young Kenyan had arrived on the island of Honolulu in around 1960 to study economics at the University of Hawaii. He was Barack Obama Sr, a member of the Luo tribe, from a rural background in western Kenya, near Kisumu. He was only in his mid-twenties, but already had a wife and children back home. On the Honolulu campus, Barack Sr met a young white girl, Ann, whose parents had only recently settled on the island after her father, Stanley, had taken a job. The couple were originally from Kansas but had been travelling around America trying to find work and a place to make home.

In his book Dreams from My Father, Obama writes that his parents were married in a civil service but that he has never been able to get to the bottom of that “murky bill of particulars”. Though records of births, deaths and marriages are subject to much stricter data protection in America, I was unable even to find a record of the wedding – or subsequent divorce – and wonder if, in fact, the couple were ever married.

Ann was still only 18 when Barack Jr was born in August 1961. Barack Sr left Hawaii less than a year later, after giving an interview to the local paper to celebrate his place in Hawaiian history as the first African graduate of the university: “the 26-year-old, straight-A student from Kenya is heading for Harvard to work on his PhD in economics”.

A friend from university, Neil Abercrombie, now a Democratic congressman and keen Obama supporter, said that Barack Sr was a great pontificator, supremely self-confident and blessed with razor-sharp intelligence. They’d sit down and talk over beer and pizza. Neil felt Barack Sr’s ambitions outstripped his commitment to Ann and his newborn child. There was no discernible friction in the separation, as far as Neil could see. Barack Sr simply left and never came back.

Ann met another overseas student at the university, Lolo Soetoro from Indonesia, and went to live with him in Jakarta, where she home-schooled her son for a time as she could not afford to send him to the local international school. Barack Jr spent a couple of years at an Islamic school in Jakarta – Lolo was a Muslim.

Ann sent Barack back to Honolulu when he was 10 (shortly after she had given birth to a second child, Maya, in 1970), to live with his maternal grandparents, Madelyn and Stanley – whom he knew as Tut and Gramps. They lived in a 10th-floor, two-bedroomed apartment a few blocks from Punahou School. Madelyn is still living there now, in her eighties and in failing health. It was his grandparents who helped Obama to get his school place.

According to Kelli Furushima, one of the organisers of the succession of annual reunions for Punahou’s graduating class of 1979, Barack Obama has not replied to a single invitation. Perhaps that tells its own story about his memory of his school days, though he did accept an invitation from the school itself in 2004, to return as a guest speaker. In his book he describes how he was made to feel an outsider at Punahou from the start, the children laughing at his “funny name” and tribal origins, so he took the name Barry and kept his head down, later seeking out the company of the few other black students. He did not deny his own mixed heritage, but identified himself as a black man. His own white mother had been a keen and ready teacher of the history of the civil-rights movement.

There was a succession of racist incidents: he had punched a boy who called him a coon, threatened to report the tennis coach, who told him not to touch the match schedule in case his colour rubbed off, and challenged the basketball coach, who called an opposing team a bunch of niggers. “There are black people and there are niggers,” the coach had explained, “and those guys were niggers.” Obama had said in return: “There are white folks and then there are ignorant motherf***ers like you.” But, of course, it was difficult to be suspicious of “white folks” when you were half-white yourself. That was part of the dilemma.

He forged a black identity through sport, taking up basketball, playing on the school’s mixed team. His school team-mates recalled an energetic, highly competitive player whom they nicknamed Barry O’Bomber, on account of his enthusiasm for long shots. He achieved impressive elevation with his double-pump jump and was a talented player, though not good enough to make the starting line-up on the championship-winning team of his senior years – a detail oddly absent from his book.

A fellow player, Alan Lum, now a coach at the school himself, recalled him as being smooth and silky in the way he walked and talked. He remembered the rest of the team teasing Obama once on a flight to a tournament in Maui, as he was sitting reading a book. It was unusual for athletes to be keen readers too.

Lum said that Obama would always be ready to speak up, where Lum would have kept quiet, especially when it came to urging the coach to give them a turn on court. He didn’t see anything significant in Barry not being called Barack. “We didn’t have too many African-Americans, but we really didn’t make race an issue. When we got to the court we were all team-mates, brothers.

Lum said he was sad when he read in the book about Obama’s internal struggles. “It was a lost opportunity. I could probably have helped him if I knew he felt that way.”

Obama’s teacher for four years, Eric Kusunoki, said much the same thing, about how sad he had felt, suggesting perhaps that this was a party line agreed by the school, who were rightly sensitive to the criticism that they had not cared enough about a boy’s racial identity. The school had been overwhelmed by media interest in the months after Obama began his run for the presidency.

Kusunoki could not recall Obama ever getting into trouble or being obnoxious, nor even being sulky or brooding. He had always been respectful and polite, and seemed articulate and well read; just another potential success in a school that created many. He had certainly never noticed or suspected that Obama was under the influence of drugs or alcohol. That really had been a revelation to Kusunoki, when he read it in his former pupil’s book. Obama had described “the swagger that carries me into a classroom drunk or high, knowing that my teachers will smell beer or reefer on my breath, just daring them to say something”. His drug and alcohol use, on his own account, had been intended to obliterate questions of identity, to “push questions of who I was out my mind”. He had smoked marijuana and snorted a little cocaine but resisted a one-time-only offer to try heroin.

By common agreement, there was widespread local use of pacololo – Hawaiian slang for marijuana – among young people in the 1970s. Obama had written his book in the mid-1990s, long before his political ascent began, when there was less need to be guarded about the minor sins of his past. Nobody had come forward since to challenge his assertion that his drug abuse had been limited to those two years at the end of his school days. Well, nobody except some YouTube freak who claimed to have enjoyed drugs and gay sex with Obama recently.

And among all his fellow alumni from Punahou, there was only one man who could say anything about the troubles Obama had experienced in those days. In the 1979 school yearbook, Obama had posted a photograph captioned Still Life, which showed a record deck, a bottle of beer and what appeared to be paraphernalia for rolling joints. Alongside it he had written: “Thanks to Tut, Gramps, Choom Gang and Ray for all the good times”. Choom Gang had been generally taken to refer to fellow marijuana smokers, even though Choom was most commonly a slang for vagina or “pussy”. Ray was the nickname – and a pseudonym in the book – for Keith Kakugawa, a fellow pupil, older than Obama, who had already left the school by 1979 but was still in his social circle.

I could not contact Keith for this article – a year earlier he had been living rough in a wrecked car in Los Angeles, having recently completed the latest of a succession of jail terms for drug offences. He had reportedly just tried to hustle his old friend Obama for money – an allegation he had denied to a reporter from The Wall Street Journal. He claimed not to know anything about Obama’s drug-taking but did recall his conflicts of identity and feelings of abandonment by his parents.

Tony Peterson had known Keith and said that, from the distance of 30 years, Keith had seemed the most troubled of boys about his own mixed identity. He was half-black and half-Japanese. It was, said Tony, extremely rare for anyone out of Punahou to take such a downward turn in their lives. But it was a salutary reminder, at a time when everyone was celebrating Obama’s success, of how things could go wrong and might conceivably have done so for Obama if he had taken a different path.

The very things that seem a virtue in his make-up now were problematic at the time. His mother’s adventurous drive and interest in other cultures made her a remote figure for many years. She had left Lolo and returned to Hawaii for a while, where Obama had lived with her and Maya. She had gone back to university to study, living a basic existence and surviving, I was told, on food stamps. Not many future presidents have been raised on food stamps.

Eventually, Ann had returned to southeast Asia, where she researched an 800-page thesis on Indonesian blacksmithing. She had been a passionate, wise woman and a devoted feminist, according to Georgia McAuley, who had been close to her in Indonesia and later in Hawaii. She had become ill with cancer and died a few days before her 53rd birthday in 1995. Ann had gracefully told Obama not to blame his father for their separation. It had been mutual, she said, not all one-sided. Barack Sr had left Harvard to return to Kenya, where he enjoyed a brief period in the sun as a senior economic adviser to the government, before falling foul of the tribal politics that have plagued Kenya once again recently – he was Luo and the government was mainly composed of Kikuyu.

Barack Sr was followed to Kenya by another white woman, Ruth, whom he had met at Harvard, and he had two children with her; as well as, apparently, more children with his early wife and another child – perhaps a grand total of seven in all (there is some dispute about the paternity of one or two) – with a fourth woman.

He had been a disappointed man, the veneer of self-confidence wiped away as he retreated into alcoholism. By the time of his death, in a car crash in Kenya in 1982, Obama had seen him again only once, when Barack Sr visited Hawaii for a month in the 1970s while recovering from an earlier car crash. He had given his son a basketball and taught him to dance, and Obama had carried those happy memories forward with him for years, until the less pretty truths about his father began to emerge from meetings with his extended family. Friends in Chicago recalled that Barack was driven in part by a determination not to suffer the same fate as his father, not to get washed up on the shore of failed dreams and ambitions.

It was Jerry Kellman who flew down from Chicago to meet Obama in a coffee shop in midtown Manhattan in 1985. Kellman was a community organiser for the church-based Developing Communities Project (DCP) in Chicago. Barack had seen the advertisement Kellman had posted in the journal Community Jobs. He had just finished college after four years of study, first at Occidental in LA and then at Columbia in New York. He had been looking for work as an organiser in New York but had ended up working as a reporter for a trade paper instead. He had thoughts of becoming a writer, and when he got to Chicago would be busy taking notes and drafting some short-story fiction based on people he met. As he explains in his book, his quest to find a place in black America was far from over. He was looking to live and work among African-Americans.

Kellman needed a black organiser to work in the African-American communities on the far south side of the city. The locals were getting fed up with Kellman’s promises that he would find someone soon. It wasn’t easy. Bright young African-Americans could find better paid jobs elsewhere. Organising was a thankless task, with only frustration and failure stretched out before it. As Kellman said, if you were bright enough to be an organiser, you were smart enough to know not to do it.

There was no political ambition at this stage and, as Kellman told me, nobody in their right minds would take an organising job as a stepping stone to politics. Organising had its origins in the labour movement, in opposition to state or federal governance. It was, pretty much, a stepping stone to nowhere, a means only to its own end, of serving and enabling disenfranchised people.

Kellman could see that Obama was politically naive but idealistic. He asked Obama what he knew about Chicago, and he replied that it was America’s most segregated city. He added something about the notoriously dirty politics of the city, its first black mayor, Harold Washington, then facing enormous hurdles from the white infrastructure, amid a long history of corruption, self-interest and racism against the south side, where “white flight” had isolated African-American communities such as Altgeld Gardens and Roseland. These faced hardships on all fronts from unemployment and housing to education and, inevitably, drugs and crime.

Kellman employed Obama initially on a probationary salary of $10,000 a year (it doubled after three months), plus a one-off resettlement payment of $2,000 that Obama used to buy an old Honda car and transport himself from New York to his new home. This was an apartment in the integrated district of Chicago known as Hyde Park, where Obama still lives, albeit now – several moves and two bestselling books later – in a large house on one of the grandest streets in the neighbourhood, with his African-American wife, Michelle, and their two daughters, Malia and Natasha.

In his early days in Chicago, according to Kellman, Obama had his own lessons in dirty politics, facing rumours and scurrilous comments that he was either “not black enough” (a charge that would resurface during his campaign for the Democratic nomination) or a pawn of the Jews and Catholics behind the DCP, and that he was just an overeducated university type who knew little of the world and nothing of the communities he had come to serve. These issues never seem to have caused him much trouble. Most people were won over and often awed by his charm and his skills and, especially, his lack of ego.

He worked closely with a trio of older women, Yvonne Lloyd, Margaret Bagby and Loretta Augustine-Herron, all with children of their own, who appointed themselves his “three black mommas” and christened him “baby-faced Obama” on account of his young age – he was 24 when he started working with them, and even younger in appearance. He was serious and punctual, and the women always seemed to be lagging behind, hurrying each other up, for fear of upsetting baby-faced Obama.

He never lost his temper but if, as sometimes happened in meetings, they would laugh and joke around, he would lower his head, slowly shake it from side to side, then look up and say: “All right, guys, this is serious, this is important.” And because they respected him, they would fall back into line.

According to Loretta, Obama just seemed to understand the people’s needs from the start.

He never tried to be a dictator but worked sensitively and patiently with the volunteers and activists, helping them to take the lead in meetings and campaigns, always taking a back seat himself, as he was supposed to do. He never seemed troubled, always confident, and that rubbed off on those around him, making them feel confident, or empowered, too.

Loretta remembered chairing a meeting with a high-handed woman official from Employment and Training who became livid when challenged, and started to patronise – “Do you even know what we do?!” – as she tried to shout everyone else down. Eventually, Obama’s voice was heard at the back of the room: “Let Loretta speak, we want to hear what Loretta has to say.” And the rest of the audience picked up the phrase and began repeating it, so the woman was forced to back down and allow Loretta to reclaim her own meeting. The victories were small, and few and far between, and often hard-earned in the face of determined opposition from rival community groups or city authorities.

Another activist, Linda Randle, remembered the beginnings of the campaign to get asbestos removed from homes in Altgeld and elsewhere. The houses had exposed pipes running through the rooms and they had been lagged with asbestos. One day they discovered that a maintenance team was going in to remove the asbestos from management offices on one of the estates, with no thought to do the same for all the residents.

Though it was not until after he had left for Harvard that the asbestos was finally removed, Obama and his colleagues played a central role in the campaign. The Altgeld homes were in a poor state of repair already, but Linda recalled the housing authority telling them that they had to make a choice – repairs or asbestos removal – as there wasn’t the money for both.

She recalled Obama as “well groomed and skinny”, always eating salads and always trying to include everyone in the meetings and campaigns, often in the face of hostile resistance. Linda had gone to a meeting once at Altgeld about the asbestos and been told how dare she come there to talk about it and take the credit, as if they had been first with asbestos; it had been their idea. It doesn’t matter who was first, Linda told them, we all need to work together – just as Obama was always saying.

Loretta said Obama seemed to work long hours and have little private life, which he did not much share with his colleagues. Loretta said she was going through a divorce then, and Yvonne and Margaret had their issues, but they would never sit and discuss them with Obama, so it seemed reasonable to assume he had his issues too, which he kept to himself.

He did talk to her about Punahou and how he worked round his cleverness so as not to arouse the resentment of others. She didn’t know of his growing disillusion with the community organising, but guessed he was not staying around for ever. Indeed, she could remember sitting with her friends at a training session once, the three of them watching Obama, and one saying they were sure he was going to be the first black president or something, because he just seemed so remarkable.

Of course, despite the long hours and the hard work in the community, Obama did have a personal life. He formed a close friendship with John Owens, who eventually took over Obama’s role at the DCP, having been trained and encouraged by Obama in preparation.

Owens had his own organising job before they met, but was immediately aware that Obama was different from the usual kind of person he encountered, and took pleasure in sharing the broad range of Obama’s interests. They went to a Cartier-Bresson photographic exhibition at the art institute and sometimes listened to jazz at a club in Hyde Park, not far from Obama’s apartment.

They went together to LA for two weeks’ leadership training and Owens saw how Obama kept to a strict schedule of work and fitness, swimming and working out in the gym. “This guy was like a machine.” When Owens went to order dessert at dinner, Obama would caution him – “Are you sure you deserve that?” – asking him to think twice about whether he had “earned” the pudding with exercise. He was still keen on basketball and played to win, giving no quarter to Owens even when it quickly became apparent that Obama was by far the better player and a good five years younger.

During their talks, Owens realised the profound importance that Obama attached to his absent father. It was something deep in his psyche that was missing, and he was trying to make a connection through the African-American community. He had shown great kindness to the ordinary working-class people in Altgeld and elsewhere, and had treated them, Owens said, like a good father. He believed it was marriage to Michelle that finally made that connection complete.

It amused Owens, too, that Obama showed other sophistications: keeping a bottle of wine and two glasses ready on the side at home and inviting his friend round for “chicken paprika”. When Owens asked what it was (it was not your typical African-American fare), Obama made a joke of the dish’s simplicity: “Well, you know, it’s chicken thighs with the skin off and paprika on it – and baked.” His apartment was somewhat bare, but heavily laden with books and music. The music was jazz and classical, and the books were philosophy, history, modern classics of black literature such as James Baldwin, and some works on revolution too.

Obama dated a young Chicago State University student for a couple of years and eventually moved in with her at her home, which was also in Hyde Park. Owens took over the lease on Obama’s apartment. Owens remembered the woman as white, though others thought she was of mixed white and Asian heritage. She was definitely not black and, Jerry Kellman recalled, there was some issue around her being threatened or disconcerted by Obama’s involvement with black culture. Owens knew the woman and never sensed she was awkward around him. She was slightly built, with close-cropped hair. Linda Randle met her a few times too, when she went to see Obama in Hyde Park for one of their regular meetings at Valois, the local restaurant.

The relationship ended just about the time Obama went off to Harvard. Owens recalled the woman being hurt “pretty badly” by the break-up and asking after Obama when Owens bumped into her in a store. But, according to Kellman, she too was going off in a new direction in her life, and the relationship had never looked like becoming permanent. The woman was not mentioned at all in Obama’s book – unless she is heavily disguised, as he does write about a woman from New York whom he loved but ended up arguing with over race.

Both Owens and Kellman could speak of Obama’s frustration with the slow-moving asbestos campaign, the feeling that he was letting folks down. Kellman thinks he was struggling to resist not throwing in the towel and abandoning community organising altogether.

Towards the end of the 1980s, Kellman and Obama went to a conference on the role of the black church, held at Harvard Divinity School. The church was at the heart of most African-American communities in America, and Obama himself had worked within it and now found faith and a home at the Trinity United Church, whose influential pastor, Jeremiah Wright, would marry Obama to Michelle in 1992. Kellman thought it likely that Obama himself had considered a ministry – a career in the church – at some stage of his life.

It was during their conference at Harvard that Obama told Kellman of his fears about ending up like his father, a political outcast and failure. He said he had decided to go to law school, and it was clear to Kellman that this was a step towards a political career. Kellman believed that Obama had seen the potential in grass-roots organising for galvanising a mass movement – something he would put to good use later. “This isn’t a campaign, it’s a movement!” was a chant I would hear at Obama rallies in Hawaii in 2008.

“Are you ready to lead?” Obama asked Owens on his return. Owens believed that Obama had been thinking about a bigger stage for a long time. They had sometimes talked about the dark days of Conservatism they were living through under President Ronald Reagan, and Obama had once said: “You know what, John? Things are gonna swing back the other way, and I’m going to be prepared when that time comes. And that’s what you need to be doing too.” Now Owens would take over Obama’s job, but Obama promised to return and not to abandon Chicago: not Owens and not the people nor the projects they had engaged with.

Loretta was sorry to lose him, but had always known he would be moving on. Inspired by his example, she too went back to school and became a teacher, while Obama went on to be a “superstar” (as Kellman put it) at Harvard, being elected the first-ever black president of the Harvard Law Review in 1990.

He had met Michelle Robinson, also a young Harvard-educated lawyer, but from a well-connected middle-class Chicago family, during a summer internship at a city law firm in 1989. When he was seeking a photographer for his wedding three years later, he asked John Owens, who was sharing the role of best man with Obama’s Kenyan brother, Roy. Owens invited his brother to take the pictures: “I want you to do this man, because this guy might be the first black president of the United States.”

Loretta sat with Yvonne and Margaret at the wedding and watched as Michelle and Barack worked their way round the room, greeting their guests. He was the same that day, 15 or more years ago, aged 31, as he was when she first knew him; the same as he is now, as far as Loretta was concerned. He was as considerate as ever, asking if everything was okay and if they needed anything. Loretta looked at him and said, not right now, but if you become the first black president we want to be at the inauguration ball… “We want to be there!” Barack laughed. “You’ve got it,” he said.

Times Recommends

• British ‘tortured and jailed Obama’s grandfather’
• BA in merger talks with Qantas
• Mumbai police were ill equipped for attacks