In Ted Butler's Archive

The Real Speculators

(This essay was written by silver analyst Theodore Butler, an independent consultant. Investment Rarities does not necessarily endorse these views, which may or may not prove to be correct.)

The unprecedented price volatility in crude oil, grain and other commodities, has focused our attention and galvanized a collective opinion. “Too much speculation” is the cry of the day. There appears to be much truth in that statement, since few can point to supply and demand factors that account for the shocking price moves. But maybe we are not looking closely enough at the speculation angle.

The most visible culprit for the excessive speculation is said to be the index funds. These are huge institutional funds that hold significant long positions in many commodity futures markets (but not in COMEX gold or silver futures). I have previously written about index funds. This is an important topic, although I have been clear to state that I have no vested interest in whether they continue to hold their big long positions or not.

Presently, there is a political frenzy developing to more closely regulate the index funds, and perhaps even force them to sell their long positions, thereby lowering the price of oil and other commodities. While I question whether these index funds should have been allowed to amass such a large position they were permitted to amass their positions legally and openly.

Should the index funds be forced to dump their long positions, that would likely pressure, at least temporarily, oil and other commodity prices. Perhaps a temporary lowering of prices is all the politicians are interested in. That way they could declare victory over the evil speculators and go back to their business of efficiently running (ruining?) the country.

But before the index funds are tarred and feathered and run out of town on a rail, let’s clear up a common misperception that it has been a sudden influx of index fund buying that has caused the recent dramatic increase in the price of crude oil. That is simply not true. The index funds are holding the same size, or smaller, long position in crude oil than they held 10 months ago, when crude oil was $70/barrel. Ditto for the large long speculators and smaller (unreported) traders on the NYMEX, according to CFTC data in the Commitment of Traders Report (COT). The data clearly shows that long traders on the NYMEX have not been buying aggressively and running up the price of crude. Well, if speculators are behind the recent sharp run-up in oil prices and the long-side traders haven‘t been buying, then who has been buying oil?

The answer is painfully obvious – the speculative shorts have been doing the buying. Public COT data proves this. The buying back of previously sold short futures contracts, primarily in the commercial category, account for the bulk of the buying over the past eight months or so, when oil was trading at $70.

There is always a short for every long position in every commodity futures contract. When enough longs panic and sell aggressively, prices plummet. When enough shorts panic and buy back their short positions aggressively, prices soar. Oil prices didn’t jump sharply because many new longs came into the market. They jumped because, at the margin, enough shorts panicked and bought back contracts they previously sold short, to prevent their losses from getting larger.

So while I agree that speculation caused oil prices to jump sharply, at least we should correctly identify which speculators did the buying. It was the shorts, not the longs. In fact, the data shows that the longs were selling. That’s not to say that oil prices won’t plunge in the future. They will, when enough longs panic and sell. To a large extent, this is the trading pattern of most markets.

By correctly identifying the real cause of the recent price spike caused by the speculative oil buying, we come to the real hidden problem with speculation. That problem is that large numbers of shorts are, effectively, trapped with their short positions. The shorts are trapped because the index funds buy and hold for the long term. That doesn’t mean prices can’t go down sharply while the index funds are long. For example, the wheat market rose almost 100% and then fell by 40% with hardly a change in the index funds’ large position. But because the index funds hold and don’t sell, regardless of whether prices rise or fall, large numbers of shorts can’t exit their short positions, even if prices fall. And when prices do fall, there are no complaints about index funds, just when prices rise.

Recently, some commentators have labeled the index funds as not playing fairly, because they don’t sell, but instead invest for the long term. But there is no rule that anyone can’t invest in futures for the long term. The index funds were clear in their intentions as they came into the futures market over the past several years. Everyone knew beforehand how they behaved and they certainly didn’t sneak into the market; because they were so big, you could see them coming a mile away. The shorts initially licked their chops, because they knew the index funds wouldn’t demand delivery and therefore attempt to squeeze the shorts. The shorts also knew the index funds would have to roll over their positions constantly, giving the shorts an opportunity to extort spread advantages due to the mandatory roll-over behavior of the index funds.

But there is such a thing as the law of unintended consequences, and that law has prevailed in the trading dance between the index funds and the shorts. When the index funds initially established their positions in oil or grain futures, there was no extreme tight supply/demand situation. That’s why great numbers of shorts sold into the index fund buying. But then conditions tightened up and the shorts appear to be on the wrong side and are looking for a way out. The easiest solution for the shorts is to have the regulators mandate that the index funds sell.

The real story should be told. It doesn’t seem fair to me to label the index funds as the real speculators when they back their purchases with the full cash value of the contracts and hold for the long term, while casting the short speculators masquerading as commercials, who are out for a quick buck, as innocent victims. If the regulators want to change the rules against the index funds, let them do so. Just don’t pretend these funds are evil and the short speculators are without blame. If we get shortages in oil or grain or anything else, prices will go higher, with or without the index funds.

I do have an interest in silver (and gold), so I would like to relate what I think all this index fund business means to those metals. There is no index fund participation in COMEX gold and silver futures (the index funds buy gold and silver through the ETFs and directly). This can be confirmed by observing the consistently small gross and net (ex spreads) long positions in the commercial category of the COT reports (the index funds are included in this category for all commodities). So, first and foremost, any arbitrary edict to limit index fund long commodity futures positions will not involve liquidation of gold and silver futures, because there are none to liquidate.

In fact, any such across the board order of index fund futures contract liquidation may so limit the choices of where to invest by large participants, that it could result in more, not less, buying in precious metals. Increasingly, I have been struck with the thought, independent of the index fund discussion, of just how few good real alternatives are available for investment, other than silver.

Although there in no index fund participation in gold and silver futures trading, there is a somewhat similar short situation connecting all the markets. There is a true speculative connection present in most markets that is hidden and excluded from current debate. That connection is the existence of a large number of shorts who are trapped and can’t easily fulfill the contract delivery requirements, nor extricate themselves from their short obligations by buying back their contracts. This is the real, but unspoken, motivation in the current index fund debate. How can the shorts be secretly rescued from the folly of their own creation that threatens to send many prices explosively higher?

Nowhere is the problem of the trapped shorts more extreme than in COMEX silver (and secondarily, gold). Precisely because there is no index fund long presence in COMEX silver futures, the problem for the shorts is worse. That’s because the corresponding long position is relatively diverse and not subject to an arbitrary edict of forced liquidation. The big shorts in COMEX silver and gold probably wish there was an index fund, or some other big concentrated long position, that they could attack and lobby against to get the shorts off the hook. But the real situation, to the shorts’ dismay, is as opposite as it can get.

While it is my contention that there is a large contingent of short positions trapped in many commodities, only in COMEX silver and gold is that trapped position held in a super-concentrated form. This elevates and intensifies the problem to the highest level. Whereas there is much debate about too much speculation in our markets, like oil, there is no talk of concentration or the intent to manipulate, two vital components in manipulation. That’s because there is no concentration or intent to manipulate in most markets. Except, of course, in silver (and gold).

In other words, while I think the shorts should be more readily blamed for the sudden spike in oil prices, for example, I don’t think that they intentionally manipulated prices upward, or that they held a concentrated position. Common sense and public data confirm this. But that same common sense and public data confirm the opposite in silver and gold, namely, an intentional and documented short-side manipulation.

The data contained in the COTs clearly indicates that the concentration held on the short side in COMEX silver and gold is head and shoulders above the concentration on the short side of oil or any market that has come under the accusation of speculative manipulation. And this is true whether you look at it either in percentage of the entire market terms or in terms of days of world production.

In the most recent COT for positions held as of June 3, the percentage of the entire NYMEX crude oil futures market held net by the 8 largest shorts was 12.8%. This concentration percentage is generally low compared to most other futures markets, mainly because the crude oil market is one of the largest futures markets around. But it is striking compared to the concentrations in silver and gold. The reported concentration of the 8 largest short traders in silver is 53.8% and 57.2% in gold, each more than 4 times the reported short concentration in oil.

And remember, these reported figures grossly understate the real concentrations in these markets, once you remove all the spread transactions, and make the comparisons more stark. Removing all the spreads in crude oil raises the true net concentration of the 8 largest short traders to maybe 19% of the entire market, while the silver percentage jumps to 79% and gold jumps to a new record of 84%, How 8 traders controlling 79% and 84% of an entire market can not be manipulation, in and of itself, is beyond me.

In terms of equivalent days of world production, the comparisons are off the charts. In crude oil, the 8 largest short traders represent 2 days of world oil production (174 million barrels held short vs. 85 million barrels daily production). In gold, the 8 traders hold short 103 days of world mine production (22.8 million ounces vs. 220,000 daily world mine production). In silver, the 8 largest traders hold short 183 days of world mine production (330 million ounces vs. 1.8 million ounces daily mine production). Under this comparison, gold has a concentrated short position more than 50 times the concentration in oil, while silver is 90 times more concentrated than oil. This is simply astounding.

Now here comes the most important message of this piece. If you think I’m just complaining about the super short concentration in silver and gold in terms of proving they are manipulated in price, you are only partially correct. I want to convey something else. If you agree with my premise that the most plausible explanation for the sudden sharp jump in crude oil prices was due to some panicky short covering, then I ask you to contemplate just what is likely to be the price result when some big shorts try to buy back silver?

Yes, I rant and rave about the manipulative and depressing impact of the concentrated short position in silver, as I believe I should, but there are big benefits in this manipulation. The price-support this short position places below the market and the explosive effect it will have on prices yet to be, must not be underappreciated. If such a small amount of short-covering in such a large market, like oil, can have such a big impact on price, it is hard to imagine what the impact on price might be from a large amount of short covering in such a small market as silver.

This is the bullish beauty of the short concentration in silver (and gold). Because the concentrated position is so large (on both a percentage and real world basis) and held by so few participants, any short covering by any of these short traders is virtually guaranteed to impact prices profoundly. Much more profoundly than what we have witnessed in oil. In fact, it is the growing extreme concentration that should tell everyone that the game is coming to an end. That fewer and fewer traders want anything to do with the short side in silver (and gold) means that the manipulators are growing more isolated and desperate. If silver and gold were such attractive free market short candidates, more and more participants would be shorting them, not less.

And to those who think these short traders are so powerful and in control, that they can extend the manipulation in silver indefinitely, please think again. What assures that the short manipulators will fail for sure, at some point, are the realities of the physical realm. The shorts can play all the paper games in the world, but the moment a wholesale physical shortage becomes evident, the shorts are toast. I intend to publish information in the near future which should provide such evidence.

In the meantime, we must try to decipher and understand and learn from the events of the day as they occur. I think oil prices recently shot up, just like wheat and cotton did not so long ago, because a number of shorts, at the margin, decided to buy back short positions in a hurry. I know that the short position in silver is held by very few participants, so when they cover, it will not be an event measured at the margin. It will be an event characterized by a change at the core of the market. The short covering in oil, wheat and cotton are just a hint of what’s to come when the shorts cover in silver.

For subscription info please go to

Start typing and press Enter to search