Yeah... it is hard to hate something that tastes good and feeds the critters too. There were no apples here either until the Dutch and English settlers arrived, so there are "good" invasives and "bad" invasives in my mind. At my place, Autumn Olive and European Buckthorn seem to compete for the same space, so it's the Buckthorn that gets the saw or the spray, and the Autumn olive gets spared. The bears are not so kind, and they rip up Autumn olives pretty good getting their fill of the berries.
Good point chaded.