I would try to work on planting some evergreens around the property border to give the deer a good visual screen from the neighbors. Wooded swamp is a great place for deer to go and spend most of their time in seclusion so that's a plus right there. Planting things like clover,chicory,rye,oats would give the deer something to eat most of the time and is easily maintained with simple mowings(except the oats). I would let the deer have the swamp area and don't go in there at all if possible. Beating your neighbors out to the stand is a big deal if you can do it. Let the neighbors bump and move the deer to you. Experiment with different ideas and keep what works for you.