Many different forms of punishment have been invented and used throughout human history, but they're not all equally good, in either moral or pragmatic terms.
Has no moral purity issues, but unfortunately it's strictly negative-sum; it only punishes instead of extracting restitution. For that reason, it's generally preferable to use forced labor (see below) if possible.
Although forcing labor out of a criminal can require containing them, this method is kind of the opposite of imprisonment in two important ways. First, it forces the criminal to produce value that can be given to the victim as restitution or to the punisher to cover the costs of catching the criminal, rather than sitting in a cage all day doing nothing and eating food bought with taxpayer money. Also, and more importantly (yes I consider this more important) it reverses the moral backwardness of imprisonment, because instead of punishing the valuation of agency, it rewards repentance by making diligent labor the path to quicker release.
The only downside of this method is practicality: forcing labor out of a criminal generally requires the punisher to be in a position of power similar to a government.
This one is very similar to forced labor, and is a lot more practical. It's also the easiest one to do anonymously, avoiding any retaliation. So what's the downside?
Well, depending on the criminal, collateral damage. If the criminal has a family, the rest of them will probably feel the impact just as much, or even entirely instead of the criminal (for example if the criminal is a child and their parents will just replace whatever you took from them). That's unacceptable.
This one is unique in that it can't scale at all; it has a fixed magnitude. Death is mainly useful for pragmatic reasons; if the criminal(s) are an active threat, it's often the only realistic option.
Not much of a punishment as it doesn't actively hurt the criminal, but this one has some practicality upsides. It'll almost never be counter-punished, so it's well suited to situations where the criminal holds so much power that trying to really get back at them will just get you hurt as well and not succeed.
The major downside of ostracism is, of course, that it is as much a punishment to the punisher as it is to the sufferer.
Exile is a lot like ostracism, except it has a lot of problems. It's like imprisonment in that it severely punishes anyone who was benefitting from the criminal being in their life (their family, employer, etc) but also it can't be done without controlling the borders of the area you're trying to exclude them from. And good people almost never have that.
One more thing I want to say. There's a particular case that comes up very often, which is a parent seeking to punish their child. There are a few common ways of doing this. One is a "time out", which, being a form of imprisonment, is absolutely not an option for conscientious parents. Another is "spanking", which, while I did say many positive words about using physical pain, is also a non-option, because of the perverted nature of striking that particular body part. Ask yourself seriously and be honest: wouldn't that seem messed up if it was anyone other than a small child?
What to do instead? Obvious. The thing both children and parents seem to ignore is that each of the child's needs is at all times being provided for by the parent. What you should do as a parent is exploit that - don't feed them their next meal, or kick them out of the house for a while. This should make intuitive sense: if you want to punish someone who you're currently helping, first stop helping them, then move into actual infringement. What most parents do is like a government punishing someone who receives welfare by inflicting pain but continuing to give them welfare. Would anyone argue that that makes sense?
Another thing to consider in the parent case is that most likely, the child genuinely doesn't realize that the food they eat and the house they live in aren't free to their parents. Children truly have no way of knowing this for their first several years. And our culture makes a point to not give them any sense of duty or responsibility outside of school until they approach adult age. Even if they're too young to fully understand, using a form of punishment that involves demonstrating their dependence on your generosity instead of violating their rights is obviously a good idea.