Why http://localhost does not work?

I encountered an issue recently where I was able to browse the Default Web Site using http://IPAddress but was not able to browse it using http://localhost
The reason for that was the Default Web Site Bindings.
If you click on Default Web Site, then click Bindings on the Actions pane, you will see the Site Bindings. Edit the Bindings and in the IP Address drop down, select "All Unassigned" instead of a specific IP Address.