Varies individually. I have found Pot does affect testosterone (as well as a multitude of other biological functions) initially by lowering as much as 10-20%, but the body adapts and compensates. SO ultimately the recreational smoker suffers more from the test drop. But another factor is the immune system (the CIA of our bodies) has issues with THC and although adapts so the disadvantages are minimum; times of immune distress (sickness, increased stress, anxiety, overtraining, extreme low calorie diets, other physical issues, etc) DOES affect gains and loses much more significantly.
So I suggest, smoke if u want, and no matter what anyone says there are disadvantages, but If u keep other key factors (diet, exercise, minimum fair conditioning, etc) these will be minimum.
Other factors to consider: Genetic and environmental.