Hi Matt!
Please, can you post a photo showing the top of the saddle?
When I've run into similar tension problems on friends' saddles, the root cause was usually a sway-backed leather cover. When that happens, the leather stretches and it can become difficult to get the tension bolt to remain tight. No, there is no "extra" nut that is missing. The pitch of the threads is usually enough to hold tension unless some other condition obtains.
The tension bolt tensions the leather and if the leather is stretched, it may not exert enough counter-tension on the bolt threads to keep the nut tight.
The underside of yours does not look like a B.17 to me because of the extra rail "return" loop behind the tension cap. Is it instead a Flyer with springs at the rear, Matt?
It is also possible the tension cap has become unstaked from the leather nose. Check carefully to make sure all the rivets going through the leather and into the steel tension cap are intact and not just partially engaged. The same thing can happen at the cantle plate at the rear, but is generally more obvious.
When you do tension the saddle, use some care. It usually doesn't require much and overdoing it can cause tears if the leather is dried-out.
I've re-riveted a number of B.17 saddles with success and have even restored several that have become swaybacked after the owners rode them while the saddles were rain-soaked. It is best to remove the leather cover on those and soak in water before reshaping and carefully drying, but it can also be done in situ if the tension bolt is backed off all the way or (preferably) disengaged first.
Best,
Dan.
