the temp control thing on thermostats should only really be used as a rough guide if thats what you are going off? use a thermometer and turn the thermostat up or down accodingly (dont pay attention to what the thermostat is actually set at)
if your already aware of that and its something else.
is the wiring definatly correct. 240v input and the output suppling the heat source?
if thats correct...
can you hear it clicking on and off (for on off thermostats)
if you can, set the thermostat really low and try and keep the probe warm so the thermostat turns off, then check whether the heat sources cools down ( obviously youll be able to see if its a heat lamp)
otherwise shoot us some info on your set up
what type of thermostat is it? on/off, pulsing, dimming?
what type of heat source are you using?
and any more detail on what seems to be happening...?