Welp, you've just opened the can of worms where the two opposing schools of thought with raising chameleons are coming out. They are:
1. Minimal handlers- Fact is, due to capability of brain and whatnot- chameleons really aren't affectionate. It's true, they tolerate us. In a sense, that's the closest to love and affection you'll get, is the respect of tolerating you. This school of thought typically leans towards conservatively handling chams to minimize stress and vulnerability to stressful illnesses that can arise. These folks follow the, they are pets to observe, not handle.
2. Regular handlers- Handle chameleons regularly to keep them socialized. This school of thought assumes that regular handling will equate less stress ultimately when the owner has unavoidable contact, like: to make cage adjustments, take them out for shower or other treatments, take them to the vet, or any other inevitable situations with cham ownership. There can also be the idea of, might as well enjoy my pet and having not a strong belief the handling will harm the cham's long-term health.
I personally fall closer to the #2 realm, as I've found handling chams regularly does socialize them in a way that makes a lot of chameleon care soooo much easier. When I take them to the vet or have to administer drops- soooo much easier when your cham is used to your hands. They tend to be less aggressive and I enjoy my time with them! I do agree with some aspects of #1, as they are not soley for the entertainment of the owner. I think of raising chameleons as like a hobby- like growing difficult roses. Raising them takes a lot of dedication and effort as chameleons can get sick and die too easily.