because those that do not have any such beliefs are then having the will of others imposed on them. it's the inherent problem when using non-facts to make real world decisions that most certainly effect the quality of life of others.
All laws have the community beliefs imposed upon others. You are advocating anarchy, and no laws. I believe I should be allowed to trade stocks with insider information. Don't judge me! you are imposing your beliefs on me and effecting my quality of life!