Just a handful of points:
Yearling racks mean next to nothing, outside of giving clues to how well his birth was timed, nutrition levels were available and how much stress was/wasn't endured. When a yearling is a spike, it's hard to pinpoint which is the reason, but it's almost always due to one of those 3 issues.
Let `em go so they can grow infers that all bucks continue making significant jumps each year. That's just not the case. In fact, I'd say somewhere around 25% or maybe even a bit more just don't do much past 3.5. An episode of Grow `em Big that was supposed to already have been posted (D&DH allowed it to slip through the cracks, until I caught it the other day) will be coming out in about 3 weeks now on aging deer. In it I have 2 pics of Heart Breaker at 3.5, 4.5, 5.5 & 6.5 (I've since got him at 7.5 this year, as well). Each year, he's within 5" of what he was at 3.5, including this year at 7.5.
A nasty tempered 8 I shot as a 6.5 year old did nothing but add some mass and maybe 2" of tine from 3.5. a buck that died from EHD at 9.5 added a couple inches of spread and maybe 5" of combined mass and tine length from 3.5. A buck I never completely made up my mind if he was a true stud 3.5 or a dang good 4.5 the first year I got him on cam actually stayed nearly identical the following year and lost 10ish inches the year I shot him as a either a 5.5 or 6.5. Cliff, a 160ish 4.5 last year, I actually told the owner I wouldn't be surprised if he flat lined on growth, is within 3" at 5.5.
On the flip side, I've had a bunch of bucks that didn't do all that much up to 3.5 make 20+ inch jumps going to 4.5 &/or 4.5 to 5.5, but it's fairly rare to not see tell tale signs that they very well may make that jump (bumps indicating future junk, more than a typical short browed 8, main beams that appear a smidge too long for the number of points and so on).
As a vast over generalization, perfectly clean, symmetrical 3.5 year old racks tend not to add nearly as much as those that are showing signs of adding more tines and junk in future years.
by all means, if you get a chance, kill the thing this year and say thank you, may I have another just like him to hunt next year, please. If he slips through the cracks and is there again next year, I think his jump will surprise you. When I look at this years pics, I don't see future booner no matter how old he gets (I very well could be wrong, but that's my honest opinion). However, I think he would be a pleasant surprise next year for you.