Most people seem to recognize the government that rules over them as more or less "corrupt", but in fact the whole concept is morally depraved.
The moral problems with government
First, let me dispel the notion that they have some kind of valid authority. Morality comes from your conscience. You should always do what your conscience says, but there are three ways it can also be bad not to do what someone else says:
When handling someone else's property. While you might argue that such things as roads are the government's property (they aren't and we'll get to that), you surely know that your own body is not the government's property, hence laws prohibiting underage alcohol consumption or requiring you to wear your seat belt cannot be validated on this ground; and neither is your labor, so neither can taxation be validated on this ground.
When you owe the person a great debt and have no other way to repay it, such as a child to their parents. But you don't owe the government anything. You might argue that they protect you from crime (they don't and we'll get to that), but remember that they fund their activities by taking your money, so they aren't doing you a favor, just forcing a transaction on you that you cannot give valid consent to because they will put you in jail if you refuse. If you're being threatened, there is no such thing as consent.
When someone is asking a favor of you and it would cost the forces of good nothing to do it. While it might be selfish to refuse to do someone a favor, it does not justify taking their money by force or even worse, putting them in jail. Who's really selfish here?
Clearly, there is no basis for a goverment's authority.
You might still think it's possible to have a benevolent government if the only laws they enforce are just prohibiting objectively, universally immoral behavior such as theft. Though note that this means it isn't a democracy. If your laws are being determined by objective, universal principles, they are not simultaneously being determined by popular vote. Certainly an ideal person will use all the power at their disposal to punish such actions even though they know they don't have authority over anyone. But there are still three differences between such a government and a conscientious vigilante.
The obvious one is that governments also collect taxes. Do you disagree with some of their actions or their methods or just think they're ineffective and someone else would be a better protector of justice if given the money? Too bad, better pay anyway or you'll be put in jail. There's no way to come close to morally justifying this. No one would ever a defend a private vigilante doing it and saying "it's everyone's duty to fund the enforcement of justice", no matter how effective or how uncontroversial the vigilante was.
A more subtle difference is the justification they give for their actions. By setting the rules in their legislative process, governments claim to actually have authority over the rules, while conscientious vigilantes only claim to be enforcing the laws of conscience.
Finally, the most subtle one, is that a government doesn't distinguish between people who commit crimes out of malice and people who commit crimes earnestly believing they're doing the right thing. A conscientious vigilante will only apply retribution to the first case, while in the second case they will use at most the amount of force necessary to stop the crime and will express sympathy for the 'criminal'.
There's a commonly raised emotional objection to vigilantism that you might be raising right now, which tends to go something like "you think you alone have the right to decide who lives and who dies?" No "right" is needed to decide that; if I genuinely believe Alice murdered Bob, I must act on this knowledge even if I can't get a court to believe me (or the situation is too urgent). If I genuinely believe Alice is innocent, then I must help her escape even if everyone else is convinced she isn't. Moral behavior is objective; it does not depend on whether a court or a senate recognizes it. We all have conscience and even if you don't believe that conscience is infallible, you have to realize that any external authority is going off either their own conscience or just another authority. So in the end, all morality comes from conscience. That's why I say it's infallible: it's the only source there is. Nothing can trump it because there is nothing else.
So that's it for the principle-based arguments. Now there's a few concerns I want to put down about how a society without government would work out in practice. First of all, you might argue that if everyone acted with this vigilantist mindset there would be a lot of conflict in the world. Wouldn't everyone just resort to violence to punish anyone who does anything they don't like? That couldn't be more false. For one thing, that's exactly how we do things right now. Anyone who doesn't like an activity can vote for it to be made illegal, and if 51% of people agree with them, the government threatens everyone into submitting to those people. More importantly, it's fallacious to compare a governed society where most people are cooperative and won't break the law even if they can get away with it to an Anarchy where we assume everyone will always victimize the weak if they can. That's not how you do comparison in a rational manner. If you observe that most people in your country are at least somewhat conscientious, then you must consider a hypothetical Anarchy that is the same way. Ask yourself if the people you know would actually act any different if the government disappeared. (Obviously there will be criminals. I was just about to get to that.)
The only thing that most normal people in an Anarcho-vigilantist society would use violence to punish would be actual crimes (theft, assault, etc). The difference is that in anarchy, there isn't a centralized authority that decides for everyone what counts as a crime and (through the threat of more violence of course) forces everyone to fund the enforcement of their ideas. Furthermore, attempting to violently punish someone for something that shouldn't be violently punished is itself a crime, and so that person would be seen as a criminal to be stopped.
Who would actually do the punishing, though, you ask? Where would we get a sufficient mass of people who would reliably go out of their way to stop criminals, given that they're not being paid to do so? Simple: pay them to do so. You want someone else to protect you, you pay that person. It's called voluntary association and it has numerous benefits over forcing everyone to pay for an accept the "services" of one group:
More options. If protection from criminals were treated as a market service the way food and other things are, there would be many different groups offering protection services. If you think one protection agency is ineffective or you disagree with some of their methods, you can employ a different one, or none of them, or try to go into business as one yourself. With a government, you're forced to pay for their police regardless of whether they're doing their job well or providing any value to you.
Competition. Since each market firm that offers law enforcement services could only stay in business by being employed instead of by forcefully extracting taxes, they'd have to do their best to make people want to employ them. That means anyone who tries to enforce bizarre laws that nobody wants enforced or use cruel and disproportionate punishments on the criminals they catch would be likely to go out of business even without anyone fighting them. Similarly, they would have to keep their prices low or all the customers would just hire a different agency. Police, on the other hand, won't face any consequences if a citizen they're supposed to protect is dissatisfied, but the employing government isn't. When the police abuse their power and hurt innocent people, the only mechanism to hold them accountable is through government courts... doesn't that seem like a poor incentive system?
Disincentivization of violence. When two anarchist protection agencies disagree on a case, instead of fighting each other over it, they have every incentive to try their hardest to find a peaceful solution. Why? Because violence is expensive! If you go to war with another justice agency, your employees are going to die, and new people won't want to work for your organization because they don't want to die. Not to mention you'll lose customers because most people would rather hire someone who's going to try harder to solve disputes peacefully. Compare this to living under a government, where the police are a unified group with such a monopoly on military force that no one can effectively challenge them, so they don't really pay any cost for the violence they commit. And whatever costs they do rack up are distributed across the entire country through taxes instead of being centered on the people responsible.
So anarchist, market-based "law enforcement" is really a strict improvement over government police.
What about the poor?
There's just one more possible objection I can anticipate pertaining to law enforcement, which is, "what about the poor? How will people too poor to afford protection get by?" There's actually a really simple answer. It's called a loan. Let me explain.
Imagine someone steals all your money so you can't pay a justice agency to get it back (or it was a crime other than theft and you just never had the money). No problem; make an agreement that they'll get it back (or take money from the criminal proportional to the harm they caused you) and take the normal cost of their services out of it before giving you the remainder. This wouldn't even result in you still losing some of your property in the case of theft because it would be normal practice for these justice agencies to extract an additional punitive fine or penal labor from the criminal and not just the stolen property (otherwise there's no actual deterrent to steal), so they'd usually have enough to give you everything back and still profit themselves.
Also there are so many large-scale examples of voluntary charity in the real world that it's unreasonable to expect that there won't be people willing and able to protect the truly improverished for free.
By the way, what exactly is a government's answer to "who will protect the poor"? Of course, it's to force the rich to pay for the needs of the poor through taxes, and if they won't, then drag them off into cages. That doesn't sound like a very moral solution to me.
Still, I should point out that that same solution is compatible with Anarchism. Believing that the rich should be forced to pay for the needs of the poor is compatible with recognizing that state authority is wholly illegitimate. It is possible to have an anarchist society where it's considered moral to steal from the rich for this purpose, though I consider that far worse than an anarchist society that respects consent more.
Another thing you might be concerned about is the idea of "public property". How would we handle road maintenance? This question depends heavily on who we consider to own the roads at the start of the Anarchy's lifetime. Given that the people who built it did so with money stolen from millions of different people, I think our only reasonable option would be to consider them unowned natural resources. People could claim ownership of them by investing labor in them and improving them (the same way you can come to own any unowned natural resource). Once a person owns a section of a road in this manner, they could charge for its use, eventually making this a worthwhile business investment. This way you could privatize roads without forcing anyone to pay for what they aren't using or letting the roads decay and become useless. Even if this idea didn't work out, even if no idea worked out, this problem would be neither fatal nor a problem with Anarchy itself (only with converting a governed society to Anarchy). So it really doesn't threaten my arguments.
How would this kind of society fare against foreign invasion? Quite well, actually. First off, it would be much less enticing for a government to invade than another governed society. Think about why governments invade. Generally speaking, it's either for material gain (the natural resources of the land, the tax base), or because they think the target society is oppressed or impoverished and that it would benefit from being "annexed". Both of these reasons are shot down when one considers invading an Anarchist society. For the second reason, they'll see how things actually are there, and, unless you can defeat my other arguments above, they'll realize that the society is actually quite well off and does not need a foreign government's intervention. For the first reason, an invading government knows that even if they win the war, every citizen in the former Anarchy is going to be a criminal, breaking whatever laws they don't believe in and don't suit them whenever they think they can get away with it. The probability of rebellion, guerilla tactics, assassination of leaders, etc, is also astronomically high. The upshot is that conquering an Anarchist society would be more trouble than its worth for a government.
Finally, if a foreign government really did wage war on an Anarchy, the Anarchists would have some advantages:
No anti-gun laws. The ordinary citizens would be very likely to be armed and willing to defend their homes from the invader.
No single point of failure. They're not affected by taking out a few important individuals the way a government is.
No misalignment of incentives. On the country's side, the people are being taxed and possibly drafted for a war they don't have a personal stake in, and as soon as popular support falls under 50% (assuming a democracy), the government has to pull out. Whereas the Anarchists are defending their homes and thus very much do have a personal stake in this. All those different private justice agencies have every reason to work together to fight the common threat.
The biggest advantage of all: First strike. The government's goal isn't to wipe out the Anarchy, so they can't just march in and shoot everyone. Most likely their goal is to confiscate everyone's weapons. But this situation where only one side is unable to shoot someone from the opposing side on sight basically guarantees that the Anarchists will have first strike in every confrontation.
One more thing...
There's one last point I want to make. I made a bunch of arguments up there about how an Anarcho-vigilantist society would not be a world rife with violence despite the vigilantist mindset. But more importantly is that it doesn't matter if Anarcho-vigilantism would involve a lot of violence. Isn't people getting hurt because they disagree about what's right better than people doing what they believe is wrong? If there was a planet full of conscientious and benevolent but impoverished people and a planet of selfish hedonists living in near-utopia and you had to destroy one, which would you destroy? Would you rather have suffering or evil be rampant?
I'd refer you to more Anarchist reading if you're not fully convinced. Besides my other article where I detail many pragmatic downsides of government, Roderick T. Long has a PDF on Anarchism where he focuses more on defending the practical side. His version is slightly different from mine due to his stricter adherence to the non-aggression principle, but he makes lots of good points. ESR also has some really good articles on why he's an anarchist and the myth of man the killer. Backalley Philosophy is a youtube channel with some amazingly accurate videos on the topic. There's also zerothposition.com, a blog with a few writers whose philosophies seem mostly the same as Roderick's but with more content. They often post about current events and stuff. The fine points of their ideas are obviously broken, but they do often have insightful things to say.