In reality, yes; you can control a Smart Plug without Internet. Here is the story of how I found out and how it works.
Unfortunately, that's not the case, and even unused chargers tend to draw power even when no device is connected, like all electrical kit on the grid. The wasted power from idle phone chargers is ...