Giving of priesthood blessings is not one of the benefits of the priesthood. The benefits are that we may receive blessings, that we may act and speak for the Lord, that we may know that he will support us in our actions and that we may receive and know His will. The benefits are that we are bound to Christ for exaltation.
Men and women are literally clothed and we may cloth and unclothe ourselves at will literally and more importantly spiritually. We are told that how we wear (treat or respect) the garment is symbolic of how we honor (wear) our covenants of the temple. To remove the temple garment for functional things does not remove our priesthood authority. Nor does the mere wearing of the garment grant that authority. How we consistently treat and think of the garment is indicative of how we honor our covenants in thought and deed.