i used to keep a fairly detailed log of every exercise, now i pretty much keep it to my progress on the big lifts. Personally, i think that it's very important to walk in to the gym with specific goals, and a notebook can help you keep track of those goals. You can specifically say, "i want to beat those reps from last week", or "beat the weight from last week."
whether you are always working hard or not, progression is very important. There are a lot of variables that can affect feelings of exertion. I know that some days i go in to the gym and 225 for deads feels heavy, but my notebook tells me i'm shooting for 350. If i didn't have the book there, or i was a little tired, or couldn't remember what i did the week before, i might make a day out of 225, and that would get me no where.
Actually writing it down though, i just label my big lift then "poundsxreps" for each set. Right now it's all structured because i'm following the juggernaut method, so it keeps things nice and easy for me.