Do you want a traditional desktop computer or an all-in-one? An all-in-one is a modular desktop unit with an integrated display. They look terrific -- taking up less space and producing less cord and other clutter than traditional desktop setups. But an all-in-one computer can be pricier than a conventional desktop or sacrifice processing power for display quality. Also, you can't upgrade the monitor and can rarely upgrade components.
What extras will you need? You'll need a monitor, keyboard, mouse or trackpad, as well as cables for these unless they are wireless. Some computer systems ship with all or some of these items, but with others you are on your own. If speakers aren't provided, you'll have to supply those as well. Not all desktops come equipped with built-in optical drives, but you can connect an external optical drive or additional hard drive storage via the USB ports. The same goes for a memory card reader, too.
Is gaming a concern? If you are serious about gaming, be sure to choose your computer with that in mind. Most cheaper desktop computers rely on integrated graphics. Some have discrete graphics, but use a low-end graphics card. Some systems are upgradable at purchase -- or later by the owner -- to use more advanced graphics solutions, but that adds to the ultimate bottom line. However, all but the lowest-powered systems can handle less intense games without a hitch. If there's a specific game you know you'll want to play, check its hardware requirements before you buy.
Do you want the flexibility to upgrade your system as your needs grow? All-in-ones typically offer very limited expandability, or none at all. If you'd like to add more memory, additional hard drives, graphics cards, etc. later on, your best bet is a traditional full-sized tower. Some small form factor computers also limit expansion possibilities, but others are designed specifically to grow with the user as needs change, and some are designed for complete user customization from the start.