I know it sucks letting someone track in the middle of season but I don't believe it's right to deny it. I don't believe it is the moral thing to do. Just leave an animal to suffer or go to waste so you have a chance at a big buck is kind of ridiculous. Hopefully that guy doesn't decide to be a dick back to you cause he could try to screw up the rest of your season in that area. Or he will just keep hunting, killing more deer and possibly the nice one your son's after.
Competition for big bucks may be why hunting will die someday. It will just become a rich man's game, like in other places of the world. I've been on a the side of the fence with 5 acres. We all need to start somewhere. Someone is not going to go out and buy 200 acres just to get into hunting. You may have made that guy quit hunting. That's just one guy but as things keep getting more competitive, the numbers quitting will increase because of denied access.
20 years ago, you needed to own no land to hunt, landowners readily gave permission. Currently you its very hard to get permission and people feel they have to the right to tell others they can't hunt on their own f'in land because they don't own enough. And even more this guys actually has permission to hunt 40, but its only 5 woods so you have made the executive decision that that 40 is too small to hunt. Pretty soon 40 acres of woods won't be big enough for someone to hunt because technically if you are sitting in the center of a 40 your are only 220 yards from the line and everyone here has tracked a deer further than that in their career.
Guys with a lot of land like to tell the little guys what they can and can't do on theirs. That is so over the top hypocritical because if someone told you what you can or can't do on yours's, you would be outraged. Like others have said, the shoe may be on the other foot someday and I hope they stick it to ya on a nice buck.
Sorry I'm being blunt but my emotions are high. I was in a similar situation with a neighbor recently. I own 5, have permission to hunt 40, about 30 of woods. He owns 120, about 60 of woods and said I don't have any business hunting that small of a place. You can tell I think what he did is a dick move and its very similar to your situation. Rant over.