Taxis are the most common form of in-town transportation and are used in most major cities of Saudi Arabia including Madinah. Taxis can be hailed from anywhere inside the city and can even be hired in advance from taxi drivers standing on the roadside. The rates offered have considerable room for negotiation/bargaining so one should keep that in mind.
Madinah also offers an interesting option of unofficial taxis where some of the locals will offer you rides. Discuss the price before getting in. In most cases prices for around the town are about SAR 5. This is a somewat safer and much cheaper option than a normal taxi. Plus you will get to experience some culture as well by interacting with your local driver.