The idea is very common that people can own land and this gives them the right to kick others off it for any reason or no reason. This is actually wrong (and not just because of the ownership backlash effect). Let me explain.
The underlying principle behind it is that if you can use someone else's property without interfering with their use of it in any way, then it doesn't count as infringing on someone else's rights. I stick to this principle in other situations. For example, if someone wanted to read a book I owned without paying me, I wouldn't try to stop them. The principle is ultimately based on a broader idea: refusing to help when you can do so without any cost to yourself is the same as hurting.
If you need more convincing than that, here's a more intuitive, conscience-based argument. Suppose you live in a small circle of land, less than an acre, and all the rest of the world is 'owned' by other people, so you can't leave your area. You are therefore in prison, are you not? And for what? What right do these people have to keep you in this little circle of land? Even if imprisonment were a valid punishment (it's not), you haven't done anything.
One caveat before I end this article: disrupting someone's privacy on their land counts as interfering with their use of it. So this belief isn't as widely-applicable as I might have made it sound. You can go on other people's land if you can do so without them having to see you or be in earshot of you.