Beej

WordPress 3.0+ is great. It shows you your uploaded header images. 
It’s too bad it doesn’t let you delete them as well! 
The answer is here with a simple yet extremely useful little plugin called Delete Custom Header. 
Unlike WP’s “Remove Header” button, this plugin completely removes the header image data 
from the database and the files from the server.
For themes with a header option, like Twenty Ten, Twenty Eleven, etc.

Delete Custom Header

Delete uploaded header images and remove them from the database.

jonkemp

Removes unwanted tags from your WordPress header.
The function wp_head; found in the header.php template file adds unnecessary tags to your WordPress template. These tags are the RSD, or Really Simple Discovery, link, the Windows Live Writer Link and the WordPress version number. This plugin will stop all 3 tags from being automatically added to your WordPress header.

WordPress Head Cleaner

Removes unwanted tags from your WordPress header.

Delete Custom Header vs WordPress Head Cleaner