I think we all have the notion to baby every tree we plant, but I think you have already answered your own question. If they are showing significant growth then why fertilize? Are they showing some kind of nutrient deficiency or are you harvesting a crop from them? If not and the trees are showing good growth then no reason to fertilize IMO. Over fertilizing can cause just as many issues as not fertilizing. Doing as you have done with weed control is a big factor in helping trees grow, reducing competition and returning the nutrients that were trapped in the weed previously.
Agree, I believe it all comes down the situation you are facing.
Over application of Nitrogen can hurt you in the long run.
Light sandy soil, that an inch of rain can move N through rather quickly, would not benefit from a major application up front, but rather two applications of the same amount over a months time.
But we also have to be realistic and understand that not all the fertilizer is going directly into the tree. P & K are relatively immobile nutrients an can take years to move into the second layer of the soil profile. Id venture a guess an say "weeds" pick up the majority of what is put down due to their shallow surface roots.
In my situation I view the shallow rooted grasses like smooth broom (my predominant cool season species) my nutrient converters. They take any excess applied fertilizer, turn into biomass, roots, stems, leaves. Then sometime in the future, they decompose into OM thus continuing a cycle of usable readily available nutrients to the tree. May take 10 years for some of that applied fert today to actually make its way into an apple and into a deer's mouth. All I am trying to do is create a nutrient sink at this time (while I am already there pruning, training, watering, spraying). So there may be a greater available source some time in the future.
Edit: Im in the same boat as H20, I am only fertilizing select plantings. No shrub or mass ceder plantings.