A month after he became President with the assassination of William McKinley in 1901, 42-year old Theodore Roosevelt invited 45-year old Booker T. Washington at the last minute to stay after an appointment and have dinner with his family in the White House. Washington, a prominent black educator and friend of Roosevelt’s, accepted. 

Several other Presidents had invited African-Americans to meetings at the White House, but never to a meal. Dining together signified that the two were social equals, something that was unacceptable to many Americans at the time. In fact, Roosevelt paused before making the invitation, according to Deborah Davis who wrote a book about the event.  “But Roosevelt was so ashamed that he hesitated, that he hastened to send the invitation out before he could change his mind.”

The dinner secured Washington’s status as the leading black spokesman in the United States, but the backlash was fierce, particularly in the South and a hit man was even hired to kill Washington, according to Davis.
See

“Teddy Roosevelt’s ‘Shocking’ Dinner With Washington” on National Public Radio’s website (2012)