He was watching her eye while giving her a verbose speech. Since she was gazing only, it proved that she really doesn't speak that language. The giant considers it a proof that she is really the sister of Janit.