The very word "collecting", while perfectly legitimate and a very significant part of the art world, can also be unintentionally intimidating, and drive a conceptual wedge between newbies and art. There's great joy, much to be learned and shared through amassing an art collection, whether extraordinary or humble. Not everyone is at a point in their lives where they can afford expensive pieces on a regular basis, or have the room for them. Nor is everyone ready to commit to building a collection and/or investing the time and effort to do so. Don't let this keep you from owning art.
Where to start? Begin with something that you are already like and are familiar with, or be daring and let yourself be surprised. Realize that this doesn't mean you're committing to building a collection, or anything you don't feel like doing at this time. You're just buying one work of art.
Keep it simple: Buy something that connects with you and vice versa. Keep in mind that art moves into your house or workspace, like a new roommate, so choose with a long view. Set aside a sensible amount of money in concert with your economic station in life, as your budget, and make the rounds as you probably have been, except now the possibility of taking something home with you will change how you view art, because you are participating. If your budget is low, look for smaller works, and be sure to look through the cafes listed here, but some of the major galleries ocassionally have some inexpensive work. Ask questions about the work, go to the artists' websites or Facebook page and learn about them, their philosophy and what else they've done. Don't hesitate to email them if you want to. Attend the opening or closing reception and talk with them. No matter where you make your selection or what you spend, the thrill of adopting a work, moving it into your home & living with it, interacting on a daily basis, is at the very core of the art experience.