I'm no expert, but I have been playing and setting up my guitars for many years (old guy). Yes the worn frets could be part of the issue, but I guess that depends how deep the gouges are. It also could be nut height, especially if the nut height is high, not low. And truss rod adjustment. I can't give you anything specific, but me personally, I would set the neck dead straight (or close) as a starting point with with the truss rod. Next I'd raise or lower the saddles until the action is good, however you like it. Then see if anything changes and go from there. If the neck is straight and the nut seems high, meaning the first few strings are too high off the fretboard, then the nut is the issue. All these things can be put back easily so nothing is permanent. You also may get a lot of advice on other internet forums, some are very active and can be useful. Good luck.