Where previously you were quite relaxed and focused, you now tense up and question your ability. Your energy is going by the second. This certain person chatters on at you about something, not listening to your responses, while only seeking to talk about their issues and how you can help them. You stammer some feeble responses by way of reply, feeling your strength ebb away until you find an excuse to leave.

Sitting in the toilet cubicle a few minutes later, you reflect on what has just happened. Why does this person have such an impact on you and how can you overcome it so that work ceases to be a place of torture?

What you can do
One solution is look at things from a military-style perspective - when you want to make a change to an environment or situation you can do it in two ways: infection or explosion. There are several practical things that you can do.

Explosion is radical and immediate - there are usually casualties and fall-out. Infection is about making slow, positive change - influencing others by your positive example. Its effects are more lasting and often more beneficial.
At work we usually wait until we've had enough of a particular situation and explode, leaving a lot of bad feeling and damage. If we can begin to infect our environment in a positive way on a daily basis, the changes will be lasting, less damaging and sustainable.

To start infecting your environment
Move around as though you're confident - and this goes for standing and sitting also. You'll feel confident as a result - it's that simple, but our minds can get in the way. Ignore your mind for a while. We are psycho-physical beings. If we change our physical behaviour we change our mindset; if we change our mindset we can change our behaviour.
Learn to listen. Nod and smile, question and encourage. Try to actually find out if this energy-taking person you find so hard to deal with has something to say - you might be pleasantly surprised. Also, if you give an example of how to do something well, others may be encouraged to follow your example.
When we find ourselves in a work environment that is predominantly and increasingly one of 'positive infection', this will lead to a general mood of confidence. Individual and group performance, and therefore productivity, will all improve.
In short, a confident company is a successful company. And companies are made up of every person that works there - so everybody's confidence is important to the work environment. Our work space is a constant bombardment of influences, both positive and negative from the physical environment, the people in it and our own self-confidence.
Be aware of this and remember that it's your responsibility to develop your confidence at work, otherwise you run the risk of becoming the energy-draining 'life taker' that forces people to flee to the nearest exit when you pass by.