is the headset properly snug and the wheel true with no problem in the bearings like side to side play? Just a long shot.
I think sometimes tires can cause a bit of a shimmy and on my nomad, while I don't ride without hands, I'd sure like to be able to because that shows me a bike that needs little attention to go in a straight line. With that in mind, I find the steering too light on my Nomad until I put racks, panniers and a few pounds on the front. It acts like a damper and significantly changes the feel of the steering. The Nomad is designed to ride optimally while carrying a decent (or insane) load. Where you have the load causes the opposite due to how high it rides and the fact that bikepacks always have some play in how they attach, so they would not dampen the occilantions the same way, in my mind, and I can see how they could even enhance that sensation.
One easy test is to take all the luggage off and see if the problem does not go away. If it does, then maybe re-distributing the load could perhaps solve this? Those guesses are about all I can think of. Good luck.
Oh, and I take it that there is no tire sealant used in your tubes?
