When in the gallery section of your admin, an entire gallery can be deleted by pressing the small "delete" text link to the right of the gallery name.