They didn't realize that Joseph could understand what they were saying. He was using someone else to explain their words to him in the Egyptian language.