I'm putting in Dogwoods (they've got some berries), am plum, hackberry, chokeberry, shadblow, all sorts of viburnums, and elderberry. I've got a fair amount of wild black and raspberrys, although they never get very tall, and are more of a groundcover. A lot of of these give soft mast in early summer through fall, and many of them are solid or great browse species. I'm also going to be putting in some DCOs and hazelnuts for some additional mast and browse.
Since I'm in an ag area, apples and oaks really make deer easier to hunt, rather than provide them with a very large missing dietary link. That being said, I think that these "other fruits" are key to a well balanced habitat, and in my habitat are one of the biggest things. They are one of the things that are missing in my autumn olive dominated area, and I truly feel that since I'm surrounded by ag and only have 20 acres, the biggest thing I can do for these deer is provide some security, and a very very high level of browse species. They can find their soybeans and corn in about 5 places within a mile of my place. I have water, my parcel is heavily pressured because of the surrounding properties state land and only having access from the West, there are swamps on my property and my neighbors 300 acres, my neighbor plots as well and has tons of different ag fields. The BIG components are there, so for me, I think that these species are as important if not more important for "my" herd's health.
Outside of deer, I think that these species have an incredibly high value to other wildlife. Birds use almost all of these species as nesting, or eat their berries or seeds. Rabbits love them, and they can provide food for them in the winter. Ninebark, dogwoods, viburnums and plum are, along with oak, some of the most important woody plants for our pollinators and the invertebrate ecosystem. Native bees use the hollow stems in many of these plants for their seasonal nests, OR they use the ground near their bases to make their burrows. Most of our native bees don't form hives or colonies, and are solitary... so these are extremely important for them. Lots of butterflies have larva that are only able to feed on certain types of host plants, which are in many cases these shrubs or similar species. I think that al long as you have oaks, and some native prairie (including some flowers), fruiting shrubs are more important to the overall mini-scale ecosystem than anything else. As far as oaks go... they support an incredible amount of life, so I'd put them before everything except a native prairie.... but other than that, I'd put all of these species up there as incredibly important.