How large is your space? For heating an entire room evenly, a fan is almost a necessity, and oscillation is very helpful. (Note: Some powerful heaters without a fan can create enough heat for a whole room, but you'll need a separate fan to circulate air in that case.) For heating just one person, a radiant heater is best. Overstock.com offers specifics on calculating your BTU needs.
Does noise matter to you? Not everyone wakes easily by quiet clicks or buzzing, and for some white noise is a pro. If the noise from an air conditioner doesn't bother you, you can likely deal with the noise of almost any heater. But if you're a light sleeper, or if you don't want a heater drowning out conversation or the TV, you should consider the heater's noise level.
Do you need consistent heat? Many owners rely on a space heater to deliver consistent heat, oftentimes while sleeping at night. If this is the case for you, a thermostat is an important feature. Thermostats are also helpful for saving on energy costs, according to Energy.gov. However, if you don't mind getting up to turn a heater on and off, you can save money by purchasing a simpler unit.
Who will be exposed to the space heater? If you need a small unit for use at the office, safety features may be less important than if the heater is for a child's room or bathroom. Cool-touch housings are especially good for kids, and tip-over switches are also important for homes with kids or pets. Drapes, bedding and clothing don't mix well with heaters either, and Harvard University recommends leaving at least 3 feet clearance in all directions -- making unit size a consideration, especially for use in bedrooms.
Do you care what the heater looks like? If a space heater is used only occasionally or in private areas of the home (or office), appearance might not matter much. For some, though, an unattractive heater isn't ideal. For aesthetes, design may be worth a higher price. At the very least, consider a low-profile heater that resembles built-in baseboard heating.