Delay when dragging an HTML element when using touch

I have written a 1 page letters game in HTML/JS/CSS. I am using jQuery.

Part of the game is to drag a letter tile (<div>s) across the board over another tile, and the 2 tiles change place.

The drag and drop is working perfectly with the mouse but when I use touch screen there is a delay from the point of touching a tile before it will drag and if you attempt to drag too soon it simply fails. It appears to be between 500 and 800 ms. This does not appear to be the 300ms delay that is known to occur in browser devices, as I have tried specific fixes for this (e.g. using fastClick (see below).

I have tried multiple fixes suggested from my research like:

using jquery.ui using jquery,ui touch punch library using fastClick library modifying the CSS touch-action property setting the viewport width to different settings modifying the and jquery.draggable() properties to minimise delay

but none of these have worked. I have also tested on different devices.

The basic code to do with the drag and drop from my page is included below - there is a lot more code but this easily demonstrates this specific issue - if you run it you will see very easily that using a mouse you can click and drag the tile as soon as you click on it, but when using touch, you have to wait a short time with you finger on the tile before it will drag - if you try and move immediately it does not drag and you lose the connection.