On Windows 7 or 8:
Open the Start Menu and type “Control Panel”. Select the Control Panel entry from the popup list.
Type “Folder Options” into the search box in the Control Panel window. Select Folder Options when it appears.
Select the View tab.
Uncheck the box for “Hide extensions for known file types”.
Click OK until there are no more OKs to click.
On Windows 10:
Open the Start Menu (icon shaped like a Window at the bottom left, typically). Select the File Explorer option, right above Settings, from the popup list.
Click on this, and then select View from the “ribbon lists” of options along the top.
After clicking this, on the right there should be a check box for “File Name Extensions”. Click this box; that should do it.