I would hazard a guess that you actually have no interior fan running at all, and can't hear it when switched on. When the vehicle moves, fresh air is forced in by movement, giving some air flow. While there could be problems at the fuse box, there could be other places as well.

The front fan (just behind the grille) will come on from time to time when AC is operating to cool down compressor if needed. I believe it comes on for a short period on start up. If it wasn't coming on at all AC performace would be poor at idle or stop and go traffic, but OK at regular road speeds.

 

I had the same issue where the blower stop working. The problem was as simple as a bad connector to the blower, the one that is actually attached to it behind the glove box. Connector was fixed so was the problem.