Hi,
Do you have any other hosts configured to use the same swap file datastore?
If all VM's are no longer using that location as their swap file location, you should be able to configure the host to not use a swap file datastore anymore.
Have you disabled the swapfile location setting at the cluster level too?
I have had the issue before we I have had to reboot a host (even after un-configuring the swap datastore) to then be able to unmount it and remove it.
Happy to help!