The prime minister said the flag represents "our diverse country" and he will not allow people to feel intimidated.