I'm a little late to this party but I think having a good neighbor is far better than the other. you have to share the deer anyway as neither of you have enough property to essentially hold the deer captive. it sounds like the two of you can form a small cooperative and maybe build on that. My neighbor has much more cover than I do and they hunt brown is down, but they hunt on a fairly limited basis - I feed the deer they hold them. They don't hunt that smart either so that helps.
It takes a village - as they say. So unless you own the entire village you are going to have to share anyway. You have a partner now - see it as a positive.