This was my rough idea of what the problem was. We had since been referred to another vet for the teeth filing and a check up. We had his teeth filed yesterday and got the full story of exactly what is happening in his mouth.
His back molars have grown in the wrong direction where the teeth are misaligned. The side of the molars that don't grind/come in contact with the other teeth have overgrown, causing a couple of "spikey parts" on one side of some of the molar teeth. This likely caused some pain as it poked into his gums and discouraged him from eating. Also, one of his back teeth looked like it got chipped by picking up something hard and trying to chew it (probably picked up a small stone from being on the grass). When the vet filed the front teeth, he also chipped of the overgrown parts of the molars, so that they are flat again.
Because the back teeth are naturally growing incorrectly, the vet recommended that we come back regularly to check on his teeth and hopefully, over time, get them 'trained' to grow the right way.
Apparently, the front teeth still needed filing but the main problem is with the molars.
We were also advised to keep his jaws moving, trying to get him to chew veggies again. Thanks for reading my long reply. I appreciate all the help.