This is a common problem when recording with two or more devices. The way that the professionals deal with this is to use hardware that can be physically connected to a common word clock so as to ensure that all devices remain synchronised. For consumer level equipment hardware synchronisation is rarely possible (because it is too expensive to include at that price point).
A solution that is usually quite effective is to turn to older technology: The “clapper board”. It does not need to be an actual “clapper board”, it is just the principle that is important. The idea is to create a common timing point on all recordings at the start of the recording, and another timing point at the end. For synchronising a video camera with a digital audio recorder, this can be as simple as clapping hands together (on camera and close to the mic). You;ve not described your set-up in detail so I don’t know what to suggest for your particular case, but you need to give yourself a clear and crisp signal on each recording device at the start and at the end so that you then have “timing marks” for synchronising. You will then be able to see if the audio is running slower or faster than the video, and, importantly, how much slower or faster.