Hi Matt!
It may not be obvious, but while the chain is still in one big, connected loop...it is captured by the chainstay and cannot be removed from the bike. When you look carefully at the chain, you can see part of the loop is inside the chainstay and part of it is outside -- the chainstay passes through the center of the loop.
Yes, you can remove it from the cog and even the chainring, but you cannot remove it from the bike (say, to clean it) without splitting it. Once split, the length of chain will come free of the frame and can be cleaned.
You can do it the first time with a chain splitter and thereafter with a separate quick-link for convenience if you'd prefer.
If the chain stretches (it doesn't really "stretch", rather, it wears between links and bushings enough to become longer), you can take up the slack by loosening the bolts under the eccentric bottom bracket and rotating the eccentric forward. If this is still not enough to tension the chain properly, then you can remove a link (really a link-pair, of one inner and one outer) and again fiddle with the eccentric to get the tension right.
Hope this helps.
Best,
Dan. (...who finds not everything is obvious on a bicycle)