You can't go up to someone and say something offensive, whether it is racist, sexist, or in any way inappropriate and say that nothing can happen to you because the 1st Amendment protects you. While you can't be arrested or charged with a crime, it is still a heinous act.
Remember, the NBA has their OWN Constitution that players, employees, owners, and everyone else involved with them have to follow. What Sterling did, whether it is was in private or in public, is a heinous act and deserved to be treated as one. I don't think the fine should've happened, unless the NBA plans to donate them to several minority associations. But the banning and almost guaranteed order of selling the team and ownership is completely fair.
It doesn't matter if is a prominently African-American league, or Caucasian league, or Hispanic league, or Asian league, or any other league. What he said was wrong, and utterly disturbing and he deserves to be punished because as an owner, a representative of the NBA, he is supposed to be held to high standards and represent the NBA well. While these comments were said privately per say, the ultimate deciding factor was his past run-ins with racism and unethical behavior. This isn't the first time this has happened with Donald Sterling and this was the straw that broke the camel's back.
He deserves to be banned and lose the PRIVILEGE to own a team in the NBA. And anyone who questions otherwise really needs to take a step back and realize the repercussions of Donald Sterling's actions and words.
It also doesn't help him that the President of NBA's Player Association is someone on the Clippers and is against what Sterling said, along with the coach of the team as well.
Adam Silver did the RIGHT thing in banning him and pushing for Sterling to be forced to sell the team and his ownership. I applaud him for doing this and so should everyone else.