Simon Prosser

Debugging tool for theme authors and reviewers.
This tool intercepts all debug information and prints it all out neatly into the footer. It also checks each page for W3C validation. 
This plugin is released as a tool to aid the development of themes and plugins for WordPress and can be used to aid debugging your theme before submission to the themes directory.

Debogger

Debugging tool for theme authors and reviewers.

Brian Alexander

Adds the current page’s template name to the admin bar.
Because this plugin reveals potentially sensitive information about the active theme, it is recommended for development environments only, and should not be enabled on a production site.

What Template

Adds the current page's template name to the admin bar.

Debogger vs What Template