That is the current thinking. Time are research will tell for sure. Just based on timing of when it hit the human population significantly before we see the big spread in the deer population suggests that humans contract it first. However, we don't know for sure how the deer population as infected.
By the way, it is much more common that folks living together with close personal contact who are not taking precautions do pass the disease to their partner than cases where they don't. Symptoms from the disease vary quite a bit. Many folks have contracted the disease thinking it was a mild cold for flu. Without detailed lab testing, unless folks happened to get tested when they have an active case, they don't really know if they had it or not. That can easily explain how folks living together may not appear to transmit to one another. One person got exposed, have few if any symptoms and carried a low viral load that was enough to trigger an immune response. With little if any symptoms, they were likely much less infectious. Later when the partner did get a significant symptomatic infection they had a level of immune protection. It is hard to take anecdotal experiences and apply them as the typical case.
It doesn't appear that it has been in the deer population long enough for "natural immunity" to be the case, if by that you mean their immune systems have been exposed, recognize the specific virus, and are attacking it sufficiently before they become symptomatic. It is much more likely that COVID-19 simply affects different species differently as do many viruses. The researchers are saying they are not currently seeing significant adverse symptoms in deer.
Thanks,
Jack