You could get a saucer to sit the pot in and water it from the bottom, if the top few inches stays dry for a wile the gnats will disappear eventually.

put the bits in their water when you water Like cpw says it will make short work of those critters

yes i agree with the mosqito bits it does the job ...

Welcome to the portal! Can't add any advice that hasn't already been said so I just want to Welcome you!

See, I knew these guys would help.