I was recently asked this question. So I don’t have to type it all out again, these are my recommendations:
For even semi serious recording, it is vital that you have good quality equipment to play back on (monitoring) If your speakers are bass heavy, then all of your recordings will end up sounding weak when played on other systems. If your speakers (or headphones) clack clarity, or over emphasize the mid range, or have other failings, your recordings will suffer because you will not be able to tell what the recording is really like.
I much prefer to monitor with loudspeakers, and for this you should use studio monitor speakers. I use “Tannoy Reveal” monitors, which are at the budget end (around $400 ?). A good pair of headphones can be a much less expensive option, though I find it difficult to get a real sense of the recording without using speakers.
I would also strongly recommend getting a reasonable quality mixing desk. This will allow you to record from more than one source at a time (for example recording guitar and vocal) and will become the centre of your recording set-up. Behringer make small mixing desks that offer remarkable value for money.
At some point you will probably want to use a condenser microphone, so make sure that your mixing desk has “phantom power” as most condenser microphones require this. At around $300 the Behringer 1622FX would be an excellent choice (possibly a little more than you need straight away, but if you get anything smaller/cheaper it will not be long before you find yourself wishing that you had this feature or that feature and want to upgrade). These mixers also include a little USB sound card, so that will save you needing to upgrade your computer sound card.
Microphones - for close mic’ing amps, a good dynamic microphone is just the job. There is a mail order company in Germany that do an “own brand” microphone the “Thomann T-Bone MB85 beta”. It is similar to the Shure SM 58 beta, but a lot cheaper. You will probably be looking upward of $50 for a reasonable dynamic microphone.
You can probably get away with using a good dynamic microphone for recording full-on rock vocals, but for anything more detailed you will want to use a studio condenser. Note that USB microphones are not compatible with mixing desks.
A couple of microphone stands, leads, and some heavy blankets to hang around the room (for recording you will want to deaden the sound in the room - lots of soft furnishings, heavy curtains and so on help a lot).