Continue recording on same track?

Hi, how do I stop recording a track (When I make a mistake!) and continue recording on the same track? I am currently trying to record a whole keyboard track all the way through without mistakes! When I mess up, I have to delete the whole thing and try again. Frustrating! There must be a way to click just before the mistake and start from that point on the same track without having to go back and start again? Would need this for all tracks I guess, vocals, guitar, etc. Many thanks

If you are using the current Audacity 2.3.0, see here:

If you are using an older version, you will need to get the current version:

You may want to look at the Punch&Roll feature introduced with 2.3.0