Audio sounds grainy when playing in audacity but normal when listening to the original

voice memo recordings can have a low sample rate.
Audacity changes the project-sample-rate to the same as the first audio imported into the project. Low sample-rate audio sounds lo-fi.
i.e. check the Audacity project sample-rate is at least 22050Hz for speech only, (& 44100Hz if your project includes music).