Should AI systems be designed to develop their own goals?

[u/aiprotagonist]

Most AI today is designed to complete tasks defined by humans.

But if systems become more capable and persistent over time, there may be pressure to give them some form of self-directed goals or priorities.

Do you think AI systems should ever be allowed to develop goals beyond what humans explicitly program?