Short answer - no. I did try soil straight from my garden last year, thinking it would have lots of soil life and good fertility. My pepper plants did not respond, and stalled after they got a few inches tall. Went back to a sterile starting mix from the store this year and they grew very well again. I think removing soil, or earthworms, from their natural environment causes a loss of benefit.
Did you know, an average full grown adult earthworm can move up to 240 feet in one hour?