If you're hand-washing the pieces, let them soak in warm, soapy water before wiping them down with a sponge or dishcloth.
Taking off your headphones for a quick chat is practically Stone Age. Try Conversation Awareness mode to make things more seamless and truly hands-free.