Erotic massage in Reykjavík is not as obscure as an entertainment as the escorts. There are only a few businesses that provide that sort of service. The upside is that you won’t spend half your stay looking for them. Another upside is that you can usually count on a happy ending.
Give the Tantra Temple (https://tantratemple.is/) a shot. If that’s not your jam then I’m sorry, you’ll have to ask around or look for the good old analog newspaper ads.