You pretty much just wander around. Eventually you'll come to a building with the mani mani statue in front of it, but it's being blocked by some guy who's like "Bring me the guy with one eyebrow and a sailor suit" (or whatever).

Then you wander around some more, until you end up in a room with no exit. I think there's random normal person in there, and a guy that's invisible but easy to find because the room is so small. Talk to him a couple of times and he'll go with you. Then when you get back to the guy who wouldn't let you through before, the two of them will go off together and you can continue to the statue of Mani Mani.