I planted some elderberry shrubs and they were browsed so hard they could never grow. I caged some of them and one grew 9 feet high the first year it was caged. It had developed a good root system and that was why it could shoot up so quickly once protected. Now I'm seeing elderberry come up from seed, and many of them make it just fine.
The first Chinese Chestnuts I planted didn't seem to interest them for browsing. But some did get rubbed when they started getting bigger.
Now, I can't plant small Chinese Chestnut without a cage. They will live but are browsed so badly they look like a bush instead of a tree.
One thing I never saw touched was a Red Osier Dogwood. Every one I planted made it just fine, and those things are thick now.