To keep the weeds and grass from growing back, you need to put down a blocking material.
There are probably better materials for this purpose, but in the end, I just grabbed a trash bag from the kitchen.
We poured about in inch of the aggregate gravel to weigh down the plastic. We will probably finish or replace the gravel with some landscape stone of a dark brown color to better blend in with the surroundings. This may be silly, though, as a 43 foot stick coming out of the ground does not blend in.
You will note the black plastic is taut around the pipe. I will go back and open this up a bit so I can add gravel to the hole as it settles over time.