Few are willing to welcome foreign workers or refugees into their countries over economic anxiety, says study.