Unless you have completely crappy garden soil - and I would assume your garden is likely got good soil so I personally would say no - proper watering and insect control as well as mice, deer and rabbit protection is more important in my book. Accidental over fertilization could kill them. That would be my call.