That seems fair.
I just don't think it would set a good precedent in the locker room if the coach is giving up on his first starting lineup after 3 games. Basketball players are human beings, and they respond to trust/confidence and clear expectations/guidelines like anyone else. I don't think you can get away with yanking Kanter from the starting lineup until ~10 games in, but that's just my opinion, and I have absolutely no experience in a competitive team sports locker room.
That situation can cut both ways too. I agree with your point, and it is likely too soon to bench Kanter. On the other hand, players also know who is and is not performing, who can and cannot perform. They can be hurt by playing/starting a guy who should, just because of his draft position or contract size. I think that Kanter is getting to getting close to the point of proving that he can't do it.