This work proposes the application of reinforcement learning approaches to vehicle automatic control. It addresses yaw motion stability and lane-keeping maneuvers. Reinforcement learning is used to tune PID parameters using LQG cost function as a reward function to optimize the PID parameters. The cost combines tracking error reduction and actuator effort limitation. Two learning agent performances are explored: the simple deep deterministic policy gradient (DDPG), and a more sophisticated one called Twin Delayed Deep Deterministic (TD3). The learning procedure checks the closed-loop stability before parameters update. DDPG and TD3 are compared in terms of performance to the Matlab-Simulink control parameters tuner. Yaw rate profile tracking and lane-keeping maneuvers are used to compare the behaviour of the achieved controllers. TD3 is found to be able to provide faster response while ensuring sufficient phase margin.