I had some problems with confidence when I was starting out. I think conceptually it's just a problem of being too serious, or to put it another way, putting too much 'mental weight' on everything you do. If something isn't a big deal, then there's no point in being anxious about it right? Not even a question of "confident or not confident" for simple stuff; it's just a matter of whether or not to do it. Simply choosing to do or not to do is easy.
More specific to drawing and art, I'd encourage you to watch a very good video from Matt Kohr of CTRLPAINT on this topic (http://www.ctrlpaint.com/videos/expectation-vs-reality
). The crux of it is to differentiate between "goal" and "expectation", and to set reasonable expectations.
When I first watched that video I thought his specific expectation of just having photoshop open was somewhat silly, so I did something different. I gathered all the best drawings I had done, and sorted them by mental category: "this is the best i have done on heads from the front, this is the best figure i've ever drawn", etc. Then whenever I wanted to draw in my head but felt uneasy in my heart, I'd go to that folder and remind myself what my capabilities really were. Rather than ask myself "Can I draw this really beautiful thing I'm seeing here?" and always answer with a resounding "No", I'd look at what I had picked out as the best I had done and asked, "Can I draw better than this thing I drew before? Can I make some improvement over that?". This is much easier to answer in the affirmative.
After several times I stopped going back to the folder, and haven't updated anything in it since. It's just obvious to me that I'll do better than last time, and if every time is better than last time, then that's good enough for me. It becomes how much more I can do better than my best last time, rather than how much more do I have to do to get to beauty.
Sometimes I hit big bumps in the road and do significantly more poorly, but since the overall trend in my mind is improvement, I just see them as evidence that I've just missed several big principles all this time, and use them as opportunities to make bigger leaps.