When Davis announced their engagement in June 1960, the couple received death threats and a slew of hate mail, according to ABC News. People even protested outside Davis’s concerts and appearances. As the target of such intense hatred, the couple hired security to protect them. Davis and Britt wed later that same year, at a time when interracial marriage was still illegal in 31 states.

Davis’s marriage to Britt proved to be so controversial that President John F. Kennedy turned the couple away from his inaugural gala in 1961 (via Smithsonian). The pair had been invited to the event, but Kennedy changed his mind shortly before the inauguration. Kennedy was concerned that the presence of the country’s most famous interracial couple would ruffle the feathers of southern politicians.

Davis and Britt had three children together before calling it quits in 1968. It wasn’t being the subject of such intense public hostility that tore them apart in the end; the couple just couldn’t overcome Davis’s commitment to his career (via ABC News). He spent most of his time away from home performing, and that took a toll on the relationship.

In the end, Davis found lasting love with African American dancer Altovise Gore (via Vanity Fair). The pair (pictured above) married in 1970, and this union lasted until Davis’s death in 1990, age 64 (via CNN). Gore died in 2009, age 65. Britt is still alive at this writing, age 88.