There is no doubt that fertilizing will increase growth of not just the goldenrod, but also the other plants. However, in a forest opening (especially a small one) lack of nutrients is not usually the limiting factor for getting tall and thick weed cover. The first problem is lack of sunlight and the second problem is how that tree roots extend literally everywhere. They suck up moisture and nutrients - plus they limit root growth from plants below them on the successional ladder.
If I was standing there looking at the site, I could answer your question better. Fertilizer is expensive but you could try it one time and see what happens. If it doesn't help, you can stop doing it. My gut instinct is that it won't be worth it.