This paper investigates the optimal tracking problem for discrete-time (DT) autonomous nonlinear switched systems with the switching cost. To avoid excessive switching frequency, the switching cost between modes is considered in the performance index, which means that the optimal switching policy is not only related to the tracking error but also the mode applied at the previous instant. The objective is to make the system state track the reference signal while minimizing the defined performance function. A model-free Q-learning algorithm that learns the optimal switching policy from real system data is developed. Furthermore, it is proved by mathematical induction that the iterative Q-functions generated by the proposed Qlearning algorithm will converge to the optimum. To implement the Q-learning algorithm, fuzzy logic systems (FLSs) are applied to approximate the iterative Q-functions. A novel structure of FLSs is designed to ensure the validity of Q-function approximation. Finally, simulation results demonstrate the effectiveness and advantages of the algorithm.