Whatever you get make sure it's a sit on top model.
Unless you consider an open cockpit type to be sit on top, I disagree with that statement 100%. I didn't like SOTs and the ones I've handled were heavy. Open cockpits are lighter and very stable. If you've dealt with a canoe, think of these as a 1 man canoe. I have a native watercraft Ultimate 12. Wilderness Systems Commander is slightly cheaper, my bro-in-law has one and it's very similar to the native I have.
Best advice I can give: As already stated by the Hawg, check craigslist. Look it over very well, but if they won't let you try it out, think twice.
Shop around and shop local - I found local dealers have better prices than online. Be patient - said local dealers often have deals or even better, events.
Try EVERYTHING out - you might be fine in a 10' kayak or you might hate it (I personally didn't like anything under 12'). The paddle size that is suggested for you may actually not be comfortable. Find a dealer that has the ability to let you try out kayaks.
This is your first yak, don't be afraid to go bare-bones to save cash instead of getting an angler. You can save a few hundred $. It's not hard to install a couple of rod holders. On many kayaks, the different angler options can also always be installed later.
Yak fishing is awesome. I don't get to do it much tho, as I rarely have time or ambition to go out myself and if I have a partner we use the canoe or we go out in a boat.