I use gloves and I have no problem with it.
I dont believe they impact on grip strength at all - ultimately how can soft leather gloves somehow support the weight you are holding? Now straps, thats different.
I started using gloves because I had problems with my hands getting messed up with pull ups. I dont miss the callus's I used to get.
I once read a thread on here about people doing deadlifts in shorts and cutting up their shins and saying it was a badge of honour. In the same way I dont believe doing anything that injures you (not matter how small) is particularly bright and with that in mind I deadlift in sweat pants to protect my legs. One final point, I shake hands a lot in my job with professional people and I know they found it unusual to shake hands with me in the past when I had callus's so - its another reason why I'm glad I now use gloves.
Just do whats right for you and dont be bullied by people wanting to show how 'macho' they are. Just my opinion