The TLDR version is this: Make expectations clear from the outset
Exactly this, though sometimes even then someone is likely to have misunderstood what was agreed.
The only group (3 riders +) tours I've done have been with the CTC club group I regularly ride with, those tours are run very much like the regular day rides, there's a clear leader, you're expected to stay together OR you're free to go off and do your own thing but it's very much in or out and the leader isn't expected to accommodate you while you're off. Most of those tours have been for just a few days and have run exactly like the day rides, not that there isn't sometimes some minor issue, but nothing to cause a serious fall out. The couple I've done that lasted longer, 10 - 20 days, were showing some signs of strain by the end, though it was mitigated by having a couple of none moving on days were people were free to do their own thing, individually or in smaller groups.
I've had mixed experience on rides with one or two others and no obvious leader. The riding pace is the least of it, that can be agreed and compromised on. But it's hard to match personalities - I like to be up and away in the morning, anything the bike needed would have been done at the end of the previous days ride and I can't believe that people set out on tour with a poorly maintained bike! I like to get 20 miles in before breakfast and the bulk of the day's riding done before lunch, then the afternoons can be more relaxed. My idea of a photo stop is less than a minute and usually doesn't involve getting off the bike. I don't mind deviating from the route, but dislike changing the objectives and goals. I could go on, but you get the idea...
The shared touring that has worked well for me is what Moronic describes as the "individual ride together" and there's only one rider with whom that's meshed perfectly. He's a much more powerful rider than me, but I'd likely be on the road an hour before him, he'd linger in a cafe and catch me up, or me him, he might shoot up a gratuitous hill to check the view while I stuck to the valley road. It was because we felt no obligation to stay together, that we enjoyed that time and often rode the bulk of the day together. The mobile phone has made this sort of touring much easier. The other advantage is that you've had a different experience, you have something to talk about in the evening.
The sort of touring I've done where it's been more about the riding than the touring, A to B routes on a schedule, I'd rather do solo. I don't think I could even agree a plan with myself!
Lastly, not really tour specific, it can sometimes feel that the rider shooting off into the distance is faster than they are. I get that on Audax quite a lot, some rider that passed an hour before is only a few places in front at the cafe! Other riders are just incapable of pacing themselves, they'll set of on a 200 mile ride as if it's a 20 and be struggling in the second half, either needing longer breaks or simply crawling along. There is an element of that with a tour, it isn't what you can do on day one that's important, but if you can still do it on day ten and twenty.