Comment by Boaz Barak

I don't think that AI necessarily should always go with the general moral sentiment, but it definitely should guide it and it shouldn't adhere too much to a rigid, rigid ethical framework. AI should have a sense of ethics, which sometimes means making decisions that the normal person would find unreasonable, but this probably should be the exception rather than the rule. A good model is a courtroom, which has both a judge and a jury of one’s peers. We want judges to use the fact that they are human and have some moral intuition and common sense. But we also want them to follow the law, and sometimes following the law might lead to outcomes that people would find problematic, such as a killer being acquitted due to an illegal search.