Where indeed...
The answer to this will be different from airframe to airframe and several factors need to be considered before you decide.
The most ideal location for a battery would be at the centerline of the airframe with the centerof the battery tray right on the center of gravity. Due to structural requirements ( like say a wing tube that sits at the CoG perhaps ) and weight considerations, the ideal location is seldom available so usually the battery location ends up in front or behind the CoG which makes which weight of battery you use affect the CoG.
This means that the plane will have only a few sizes ( weights ) of batteries that will allow the model to achieve a proper CoG. If you select a battery that is heavier or lighter than the recommended sizes... that means you will have to offset that weight somewhere. Adding any additional weight will always affect the flight performance/characteristics as well as as well as increase the chance of damaging the landing gear and it's associated structures used to support the LG.
So first you want to find out why the model isn't balancing at the recommended CoG... Did you add too much weight in the tail... If so the answer is easy, permentantly add weight forward of the CoG, add it as far forward as you can so you don't have to add as much weight and simply glue it in with epoxy or similar glue.
If the issue is caused by you using a battery that is heavier than recommended, then the so called easy answer is get a lighter weight battery... If you can't, then you will need to add weight to a location you can get to easily ( like in under the wing area forward of the CoG ) in order that you can remove that weight later for times when you aren't using a heavy battery. You could perhaps attach the added weight with some velcro or whatever you can consider that can be easily removed later.
Good luck on whatever direction you decide on. It is something you need to think about before you decide.
Randy