The former Stuttgart boss was given his marching orders as the Wolves fell to 10th on the table after a loss to Cologne on the weekend and the sacking was confirmed by supervisory board chairman Francisco Javier Garcia Sanz.
"This decision is not easy for us on the board," he said.
"However, we came to the realisation that, after the many negative experiences of the past few weeks, we had to act in order to set in motion a new start."
General manager Dieter Hoeness added: "We were no longer of the opinion that we could have changed the situation in the foreseeable future with Armin Veh."
Reserves coach Lorenz-Guenther Koestner will be in charge temporarily and will oversee Friday's Bundesliga match at Hamburg.