Civilization is a propped up thing that requires buy in from its members. This means some will toil in the fields, and others will make decisions. You might not like all of those decisions, but in exchange you get a military, a road system, certain protections against famine, etc. In addition, you get an open exchange of ideas, goods, and other services from all these more people living in your society. Progress, right?
If we all go back to the "I'm only doing what I want to do dagnabit!" mentality, then society will start to break down. It's a compromise, but personally I chose some of that progress since it fills my life with all sorts of cool shit I couldn't have came up with myself.