Aye there was - I know of several people who voted leave because of free people movement rules.
People who had difficulty getting a doctor’s appointment or a school place would often blame the influx of migrants from the EU as the cause.
Reality is that the vast majority of these EU migrants are ‘economic migrants’ i.e. they came to the UK to work, therefore paying taxes and Nat Insurance - this extra tax input has not been put into Schools or Health care.
On the thread topic - here in NI the population is divided by ‘Nationalists’ and ‘Unionists’
Over the years the Unionists formed a majority - that is now in some doubt but the Union remained safe because a sizable chunk of Nationalists who view themselves as Irish were happy with the status quo so long as there is no border.
We have our Irish passport, fly from Dublin, identify as Irish but live in the UK (NI) and would vote to remain so.
But take us out of the EU against our wishes, put a hard border up and everything changes.
In my own community there are many who take this view.