The key may open the gate, but you must open the door.  Enter the city with no people.
Click the image above to enter.