Almost every panorama has a precise geographical location assigned for future reference. Location of capture position is set with possibly high accuracy (relative to spatial precision of Google Maps™ data). However there is several exceptions to this rule. If the picture is captured at private property or for some reason true location can not be provided, it is located not precisely or even mistakenly. In single cases location is impossible to define at all (eg. plane photo) so it’s not provided.
Color markers with numbers shows clustered picture locations for larger area. More saturated marker color means bigger picture count in the area. When you zoom in, they are displayed in more detailed manner, down to the single picture marker (white, no number).