My personal experience is that an easy way to inadvertently upset a bike’s stability is to fit a handlebar bag. Without rear panniers it upsets the bike in a X-wind and it’s weight forward of the handlebar stem makes the bike more twitchy as the bag’s weight increases. Longer mounts that move the bag further forwards must make these effects worse.
I have never tried having a bag on a rack on top of the front wheel but I imagine this might cause the same problems?
You're absolutely right. Most cyclists with their brain in gear try to distribute the weight on a loaded bike equally between the wheels whether they know about the Centre of Gravity and the dynamic couple between the front and rear wheels or not. It's long-established common sense.
But very few know about another movable point, called the Centre of Aerodynamic Pressure, which acts, to simplify matters, through the side profile of the bike, which makes the distribution of surfaces extremely important because it is imperative for the stability of any vehicle that the centre of aerodynamic pressure be behind the centre of gravity.
If you contemplate the side surfaces of the front of the bike and of the rear, you will see that with the rider on board there is just about zero chance of the centre of aerodynamic pressure moving forward. Panniers are put first on the back for a good reason, so is any further loading of the rack. The height of any load with much surface at all also goes on the rear rack first rather than the front, and lowrider pannier are fitted low down because you don't want the dynamic centre of aero pressure to slope downwards towards the back of the bike.
If the cyclist as his only load on the bike except himself fits a rack bag and high up at that, he unsettles the distribution of side surfaces and the stability of the bike in side winds as small as that caused by a hatchback passing him. The bike wants to swap ends because it is quite possible that in addition the front to rear aero couple now runs downwards.
That's what your experienced. Note that in yaw all these forces (vectors) may be ameliorated or enhanced. A good example is a large flat side truck passing close by. With the bike's design side surface and consequently aero handling upset by a possible doubling of the front surface by the rack bag, a point may arrive where you don't know which way the front wheel will break, which adds another layer of uncertainty.
Once, at a crossroad with high hedges on all sides, a gust of wind knocked me off my bike. A trucker stopped and helped me (I hurt a lot -- I'd been travelling at speed) and said, "You're not a blow-in [a recent arrival in the countryside]. You should know better. All the flatsiders [closed truck drivers] know about this corner." I'd been riding there thirty years. I hadn't even thought of the aero CoP because a lot of the aero effects I was used to in motor racing don't matter with a four-wheel vehicle until you're travelling well over the ton and you anyway have large flat surfaces well behind the rear axle. I calculated up the side areas on my bike and instantly discovered that a handlebar bag, while very convenient, is an invitation to road rash in a windy area such as I live in on the River Bandon.
Jobst Brandt, probably the leading theoretician of bikes in the latter half of the 20th century, used to say, "If the front wheel goes, there's nothing you can do. You're gone." It's one reason the centre of pressure and reaction to invisible side forces like crosswinds must remain behind the centre of gravity of the bike under all dynamic conditions, because you can recover from a rear-end upset whereas at the front recovery falls somewhere between extremely difficult and goodbye.
By comparison I can not feel any negative handling effects with the tyre on the front 1/4 inch bigger than on the rear.
That's hardly surprising; it's a modest change, well within the design parameters for any competent touring bike, which has large reserves of dynamic safety built in. But make the front much wider than the rear, and take the bike out on a fast downhill with curves, and you'll frighten yourself. A bike that turns in sweetly at moderate speed in rush hour traffic will suddenly become a health hazard. And worse with a touring load on it.
For a while, because I just couldn't find a wide rim (24mm across the beads, minimum) actually in stock anywhere, I used a motor built into a narrow front rim with the fat tyre on the fat rim with the Rohloff at the rear. On hilly lanes I'd been riding for decades, it was pretty obvious that the bike was nervous enough for me to cut apex speed from 55kph on the sweeping downhill curves to just over 30kph because the 60mm Big Apples just weren't working the same predictable way any more. Until then I'd declared the difference in the rims "not such a big deal", which was true enough when the referring to riding on the level under 20kph. It was one reason that for my next venture into electrifying the bike, I bought a mid-motor, to move the weight and the motor aero area backwards and downwards -- and to return to my fabulous 24mm rim width all round.