A direct response

I like how Joseph acknowledged the rumors and respond directly to the rather than dodging them. He had been called a money digger and he admitted that he was among the hired hands who dug as requested in search of a silver mine but he was also instrumental in convincing his boss to give up the search for treasure. He also gives context to the situation by explaining that due to poverty his whole family was obliged to take such work as they could get; they were not in a position to pick and choose what jobs looked or sounded good or respectable so long as they were legal and ethical.