Probably the most important lesson I can teach you - if you can be convinced - is that government is one of the evilest ideas people have ever come up with. If you're anything close to a good person, you probably already believe that the American government (or whatever government you live under) is highly corrupt, but that isn't enough.
The moral problems with government
First, let me dispel the notion that they have some kind of valid authority. Morality comes from your conscience. You should always do what your conscience says, but there are three ways it can also be bad not to do what someone else says. The first is when handling that person's property. While you might argue that things such as roads are the government's property (they aren't), you surely know that your own body is not the government's property, hence laws prohibiting underage alcohol consumption or requiring you to wear your seat belt cannot be validated on this ground. The second way is when someone is asking a favor of you and it would cost the forces of good nothing to do it. While it might be unkind to refuse a favor, it is not something you can ever punish someone with violence for. Remember that every law made by any government is backed by the threat of violence if you refuse. The third way is when you owe the person a great debt and have no other way to repay it, such as a child to their parents. But you don't owe the government anything. You might argue that they protect you from crime (they don't), but you have to remember that they fund their activities by taking your money, so they aren't doing you a favor, just forcing a transaction on you that you may or may not want. Therefore, no government has authority over anyone.
You might still think it's possible to have a benevolent government if the only laws they enforce are exactly those of conscience. Certainly a good person will use all the power at their disposal to punish evil even though they know they don't have authority over anyone. But there are still three differences between a government and a conscientious vigilante. The obvious one is that governments also collect taxes, forcing the populace to fund their activities instead of getting an honest job. It should be clear which has the favorable side of the comparison here. A more subtle difference is the justification they give for their actions. Governments claim to have authority, while conscientious vigilantes only claim to be enforcing the laws of conscience. Finally, the most subtle one, is that a government doesn't distinguish between people who commit crimes out of malice and people who commit crimes earnestly believing they're doing the right thing. A conscientious vigilante will only apply retribution to the first case, while in the second case they will use at most the amount of force necessary to stop the crime and will express sympathy for the 'criminal'.
There's a commonly raised objection against vigilantism that you might be raising right now, which tends to go something like "you think you alone have the right to decide who lives and who dies?" Rather, everyone has the right to decide that. We've all got conscience. We all have to listen to it. And even if you don't believe that our consciences are infallible, you have to realize that any external authority is going off either their own conscience or just another authority. So in the end, all morality comes from conscience. That's why I say it's infallible: it's the only source there is. Nothing can trump it because there is nothing else.
Another way to look at it is this: all of us on this planet have a conscience. All of us don't follow it. None of us have any more right to determine morality than any others - not even someone elected by a majority of flawed people. But someone has to make the decision. That's why everyone can and should make it for themselves. No one determines morality, everyone simply decides it.
So that's it for the principle-based arguments. Now there's a few concerns I want to put down about how this kind of society would work out in practice. First of all, you might argue that if everyone acted with this vigilantist mindset there would be a lot of conflict in the world. Wouldn't everyone just resort to violence to punish anyone who does anything they don't like? No. That couldn't be more false. For one thing, that's exactly how we do things right now. Anyone who doesn't like an activity can vote for it to be made illegal, and if 51% of people agree with them, the government threatens everyone into submitting to those people. More importantly, it's fallacious to compare a governed society where most people are cooperative and won't break the law even if they can get away with it to an Anarchy where we assume everyone will always victimize the weak if they can. That's not how you do comparison in a rational manner. If you observe that most people in your country are at least somewhat conscientious, then you must consider a hypothetical Anarchy that is the same way. Ask yourself if the people you know would actually act any different if the government disappeared. (Obviously there will be criminals. We'll get to that in a second.)
The only thing that most normal people in an Anarcho-Vigilantist society would use violence to punish would be actual crimes (theft, assault, etc). And the concept of punishing crimes with violence is anything but unique to Anarchy. It's just the timeless realization that criminals must be stopped to protect their victims. The only difference here is that there isn't a centralized authority that decides for everyone what counts as a crime and (through the threat of more violence of course) forces everyone to fund the enforcement of their ideas. Furthermore, any rational individual will realize that attempting to violently punish someone for something that shouldn't be violently punished is itself a crime, and so that person would be seen as a criminal to be stopped.
Who would actually do the punishing, though, you ask? Where would we get a sufficient mass of people who would reliably go out of their way to stop criminals, given that they're not being paid to do so? Simple: pay them to do so. You want someone else to protect you, you pay that person. This is the same way we do it in government, except for several advantages:
More options. Due to the decentralized nature of this kind of law enforcement, there would be many different groups who offer such protection services. If you think hate speech should be considered a crime and should be punished, you can employ the group that agrees with you, otherwise you can hire a different group. Or you can choose not to employ any of them and try to provide these services for yourself. You see? Everyone is better served by this system.
Competition. Since each group that offers law enforcement services stays in business by being employed instead of by forcefully extracting taxes, they'll have to do their best to make people want to employ them. That means anyone who tries to enforce bizarre laws that nobody wants enforced or use cruel and disproportionate punishments on the criminals they catch is likely to go out of business even without anyone fighting them. Similarly, they're forced to keep their prices low or all the customers will just hire a different agency. Police, on the other hand, won't face any consequences if a customer is dissatisfied, but the employing government isn't.
Disincentivization of violence. When two protection agencies disagree on a case, instead of fighting each other over it, they have every incentive to try their hardest to find a peaceful solution. Why? Because violence is expensive! If you go to war with another protection agency, your employees are going to die, and new people won't want to work for your organization because they don't want to die. Not to mention you'll lose customers because most people would rather hire someone who's going to try harder to solve disputes peacefully. Compare this to living under a government, where the police are a unified group with such a monopoly on military force that no one can effectively challenge them, so they don't really pay any cost for the violence they commit. And whatever costs they do rack up are distributed across the entire country through taxes.
There's just one more possible objection I can anticipate pertaining to law enforcement, which is, "what about the poor? How will people too poor to afford protection get by?" My answer is: ask yourself how government solves that problem (the rich pay for the needs of the poor), and then ask yourself if that solution might actually be worse than the problem. That's communism! Do you realize that? Solving poverty by pooling everyone's resources - that's the very definition of communism. It really blows my mind that most statists claim to abhor communism while somehow being entirely unaware of this hypocrisy. I don't know anything about you, reader, but if you're in the US, I'm 90% sure you don't support communism. That means you can't use that solution, making one more reason not to advocate government. However, I'll also be the first to admit that communism is not nearly as evil as most people think it is. I think it's possible to be a good and reasonable person while believing in communism. So I will point out that that very same solution can be implemented in Anarchy, if you want. You would implement it the exact same way as a government does - have a group of people whose job is to take from the rich and give to the poor (and of course, those people's wages come from the rich).
It's also worth pointing out that there are some really good examples of volunteers doing amazing things for the world without any selfish incentive. Wikipedia for starters stands as a tremendous testament to human selflessness. And of course the existence and quality of the entire Unix family of operating systems - they're developed entirely by volunteers. So I wouldn't be surprised if there were tons of people in an Anarchy willing to protect the poor for free.
Another thing you might be concerned about is the idea of "public property". How would we handle road maintenance? This question depends heavily on who we consider to own the roads at the start of the Anarchy's lifetime. Given that the people who built it did so with money stolen from millions of different people, I think our only reasonable option would be to consider them unowned natural resources. People could claim ownership of them by mixing their labor with them and improving them (the same way you can come to own any unowned natural resource). Once a person owns a section of a road in this manner, they could charge for its use, eventually making this a worthwhile business investment. This way you could privatize roads without forcing anyone to pay for what they aren't using or letting the roads decay and become useless. Even if this idea didn't work out, even if no idea worked out, this problem would be neither fatal nor a problem with Anarchy itself (only with converting a governed society to Anarchy). So it really doesn't threaten my arguments.
How would this kind of society fare against foreign invasion? Quite well, actually. First off, it would be much less enticing for a government to invade than another governed society. Think about why governments invade. Generally speaking, it's either for material gain (the natural resources of the land, the tax base), or because they think the target society is oppressed or impoverished and that it would benefit from being "annexed". Both of these reasons are shot down when one considers invading an Anarchist society. For the second reason, they'll see how things actually are there, and, unless you can defeat my other arguments above, they'll realize that the society is actually quite well off and does not need a foreign government's intervention. For the first reason, an invading government knows that even if they win the war, every citizen in the former Anarchy is going to be a criminal, breaking whatever laws they don't believe in and don't suit them whenever they think they can get away with it. The probability of rebellion, guerilla tactics, assassination of leaders, etc, is also astronomically high. The upshot is that conquering an Anarchist society would be more trouble than its worth for a government.
Finally, if a foreign government really did wage war on an Anarchy, the Anarchists would have some advantages:
No anti-gun laws. The ordinary citizens would be very likely to be armed and willing to defend their homes from the invader.
No single point of failure. They're not affected by taking out a few important individuals the way a government is.
No misalignment of incentives. On the country's side, the people are being taxed and possibly drafted for a war they don't have a personal stake in, and as soon as popular support falls under 50% (assuming a democracy), the government has to pull out. Whereas the Anarchists are defending their homes and thus very much do have a personal stake in this. All those different private protection agencies have every reason to work together to fight the common threat.
The biggest advantage of all: First strike. The government's goal isn't to wipe out the Anarchy, so they can't just march in and shoot everyone. Most likely their goal is to confiscate everyone's weapons. But this one-sided policy of being unable to shoot someone from the opposing side on sight basically guarantees that the Anarchists will have first strike in every confrontation.
One more thing...
There's one last point I want to make. I made a bunch of arguments up there about how an Anarcho-Vigilantist society would not be a world rife with violence despite the vigilantist mindset. But more importantly is that it doesn't matter if Anarcho-Vigilantism would involve a lot of violence. Isn't people getting hurt because they disagree about what's right better than people doing what they believe is wrong? If there was a planet full of conscientious and benevolent but impoverished and war-torn people and a planet of selfish hedonists living in near-utopia and you had to destroy one, which would you destroy? Would you rather have suffering or evil be rampant?
This is one of the only issues where I'd refer you to more Anarchist reading if you're not fully convinced. Besides my other article where I detail many pragmatic downsides of government, Roderick T. Long has a PDF on Anarchism where he focuses more on defending the practical side. His version is slightly different from mine due to his stricter adherence to the NAP, but he makes lots of good points.
There's also zerothposition.com, a blog with a few writers whose philosophies seem mostly the same as Roderick's but with more content. They often post about current events and stuff. Their philosophies are pretty solid and consistent, even when they differ from me (it's usually a matter of axiological dissonance rather than fallacies on their part).