As companies transition from data retrieval to AI generating data on the spot, tokens are more important than ever.