The old question “can we really trust the central bankers to set economic policy” has come up again. The latest person to raise it is Mohamed El-Erian, the former CEO of Pimco who now acts as an economic advisor to insurance giant Allianz.

In a new book called The Only Game in Town, El-Erian argues that the leaders in the central banks are emperors with no clothes. He makes some good points that everybody concerned about the economy and democracy should consider.

“They have been forced to make things up on the spot,” El-Erian wrote of the world’s central bankers. “Repeatedly, they have been compelled to resort to untested policy instruments. And, with their expectations for better outcomes often disappointed, many have felt (and still feel) the need to venture ever deeper into unknown and unfamiliar policy terrains.”

Many of the central bankers are really academics, political hacks, and bureaucrats who have no special knowledge or ability, El-Erian charges, yet they now make economic and increasingly political and social policy as well.

The reliance on central banks could even be responsible for slow economic growth, increasing income inequality, growing market volatility, and political inaction, El-Erian’s admirer, LinkedIn writer Daniel Roth, wrote in a recent op-ed piece. Roth raises a familiar argument that elected political leaders have effectively surrendered some of their power and responsibilities to the central bankers.

How Central Bankers became Superheroes

El-Erian’s argument is an important one because the policy setting role of central banks is actually a very recent phenomenon. Even though central banks have existed in some countries (Sweden and England) since the 17th century, their role in setting economic policy is recent.

During the 1930s and 1940s, the chief architect of Britain’s economic policy, John Maynard Keynes, was an investor and academic who had no real connection with the Bank of England. At the same time, the U.S. Federal Reserve played a very limited role in the U.S. economy. It did not even come up in the policy debates over the Great Depression in the 1930s.

The modern cult of the central banker arose in the United States in the 1980s. One chairman of the Federal Reserve, Paul Volcker, enacted policies that effectively tamed double-digit inflation. Volcker achieved what three presidents, Nixon, Ford, and Carter, had failed to accomplish – he ended inflation.

Volcker was succeeded by Alan Greenspan, who was widely viewed as the architect of America’s economic recovery in the 1980s and 1990s. Greenspan’s cheap money polices helped the U.S. recovery from some economic storms, including the dot.com bubble of 2000.

Whether Volcker and Greenspan were geniuses or simply lucky is open to debate. The problem is that their success and the attention the media paid to them greatly increased the power and influence of central bankers. During the 1990s, reporters actually followed Greenspan around and watched how he carried his briefcase in an attempt to see what inflation rates would be.

A side effect of this success was that American politicians effectively turned decision making about economic policy over to the Federal Reserve. Politicians did this because it shifts the blame for economic troubles to the Fed and it frees them of having to spend time on economic issues they neither understand nor care about.

In recent years, the American reliance on central bankers has been repeated all over the world – in Europe, Great Britain, China, and even Russia. Almost every central bank president is now a celebrity under constant media scrutiny.

Should Central Bankers Powers be curbed?

There have always been skeptics in the power of central banks and some politicians that wanted to curb them, yet only a few people have actively challenged the new status quo.

In the United States, former Congressman Ron Paul (R-Texas), who is a libertarian, and U.S. Senator Bernie Sanders (I-Vermont), who is a socialist, joined forces to promote “Audit the Fed Legislation” in the 2000s. The two wanted to subject the Federal Reserve to the same scrutiny from auditors that regular banks receive and limit its powers.

Some economists, such as American Nobel Prize winner and New York Times writer Paul Krugman, have also been highly critical of the reliance on central banks to promote economic growth. Krugman would like to see government take a more active role by increasing spending and working for active stimulus, in a return to classic Keynesian economic policies, which rely on direct government spending rather than market and interest rate manipulation for stimulus.

Recent events, such as the decline in household income in the United States, the global financial meltdown of 2007-2008, the continuing downturn in Europe, and the volatility in China, have led to increased criticism. The U.S. has become ground zero for criticism of central banks.

It is easy to see why the wages of college graduates in the United States have fallen by nearly $4 an hour, dropping from around $19 in 2000 to $15.29 in 2014, according to the Economic Policy Institute. At the same time, income inequality increased dramatically, with incomes for the wealthiest 1% of Americans growing by 138%, while pay for the bottom 90% grew by 15%.

Revolt against the Central Banks

This has fueled popular anger against the financial system, and by inference, the Federal Reserve in the U.S. Bernie Sanders, who was not even on the list of credible presidential candidates in 2014, has mounted a successful campaign for the Democratic nomination and won contests in several states, including Michigan, much to the chagrin of popular frontrunner Hillary Clinton. Clinton is widely seen as a champion of the economic status quo, with Sanders as its archenemy.

Sanders’ success and the positive response to books like El-Erian’s indicate that faith in central bankers is in decline. What is not clear is if it will lead to popular movements to limit central banks’ powers or abolish them. There was such a movement in the United States in the 1820s and 1830s, which led to the election of a popular President Andrew Jackson and the demise of America’s first central bank, the second Bank of the United States.

It remains to be seen whether people like El-Erian and Lebanese-American philosopher Nassim Taleb, who has gone so far as to demand the abolition of the Nobel Prize in Economics, can spark a revolt against central banks. Taleb, who is the creator of The Black Swan, contends that economics is not a science and that faith in economists, including central bankers, was responsible for the crisis of 2008.

Perhaps a more important question to ask is if central bankers will willingly give up the power, prestige, and influence that they have achieved in the last few decades. Will central bankers fight to retain their power, and will that battle devastate the economy? That is exactly what happened the last time a nation tried to get rid of its central bank.

During the 1830s, Jackson waged an all-out war to strip the Bank of the United States of its power to control the money supply. Jackson won, but some historians think his actions led to the panic of 1837. A financial collapse that some critics blame on an explosion in the money supply that occurred after the bank lost its power. Hopefully, history will not repeat itself on a global scale.

There is one certainty here. Central banks and their power are going to be a major political issue in many nations for the foreseeable future. The popular view of central bankers could soon be transformed from economic wizards to scapegoats for economic problems. The results of that could make central bankers wish they never became celebrities.