They are more than likely native and as such not hurting anything. If you have other....better plans for the area then fine, but don;t kill them just because they are not some prime deer food. Wildflowers and the like mean food and cover for many other critters.