I think a lot of it is scent that is out of its normal. On the edge of my yard I have a large-ish area of brush and downed trees, I would say about 100 feet by 200 feet. When I had my yard cleared in 2019 they dozed a lot of it into a pile, then later I had a big wind storm, and a bunch of more trees fell in that area. Anyhow deer bed in there all the time, and even will lay on top of my septic which is right next to it. They watch me do my daily stuff, walk out to my wood boiler, drive cars in and out of the garage, walk out to the mail box, and they arent bothered at all, even my old dog doesnt bother them. But the minute I walk into the woods, they are gone. I have 3 apple trees right next to this area, they will walk out to the apple trees, grab a snack or 2, and walk back in the brush while I am outside.
Now put me out in the woods, over looking a food plot, and an apple orchard, those same deer that decide to leave the brush pile for greener pastures, will take off running as soon as they are down wind of me at all. They know their danger zones!
In the winters when the deer gather up, especially last winter with record snow, and 3+ feet of snow on the ground the entire winter, the does and fawns gathered in my plowed roads, and yard pretty much all winter. I had a plowed path going to my wood boiler, and wood yard, the deer were like cattle, and would be feet away from my on the tractor, and if they did jump off the plowed path, they would only go about 20 feet and stop and watch me then come right back as soon as I drove past them. It isnt really the scent, it is something different that doesnt belong that they are afraid of.
Usually about every other year I get a couple old does that bust me, and they will watch for me in the tree stands every time I go out, then sit back about 50 yards and blow at me. Some years those same does end up in my freezer, some years I move to other stands.