The History & Evolution of Data in Commodities

In the early days of commodities trading, access to information relied heavily on personal networks, political influence, and on-the-ground logistics. Availability of capital created an asymmetry of information, where the largest companies knew more than anyone else. But now, things have changed. 

Today, with the rise of alternative data sources, anyone with an internet connection can explore details on everything from crop yields to shipping movements. Information that was once exclusive is now readily available. This new era of transparency, combined with increased volatility and opportunity in the commodities space, has led to the proliferation of entrants, including hedge funds, banks, and tech-focused trading players.

Humble Beginnings

Throughout history, traders have been on a quest for Alpha – their competitive edge. This relentless pursuit has driven innovation and transformation throughout the industry. 

In the 19th century, Paul Reuter used homing pigeons to transmit stock prices between Brussels and Aachen, Germany, allowing him to receive market updates faster than anyone else. More recently, in 2009, Spread Networks undertook a $300 million project to lay 1,331 kilometres of fibre-optic cable from Chicago to New Jersey, all in an effort to reduce data transmission time by four milliseconds. Today, the pursuit of Alpha continues, with data being the latest hot commodity.

The data landscape in commodities has changed dramatically. Back in the 80s and 90s, says Jack Nugent, director of the commodities and tech strategic advisory firm Tradavex, trading companies relied on contacts in satellite offices around the world for their intelligence. They would then position trades within their company to monetise this information.

“Fast forward to the 2000s, trading firms began to reduce their reliance on satellite offices, generating their own tacit data by acquiring assets such as storage facilities, refineries, fleets, and terminals, allowing them to directly observe what was happening through their own infrastructure. And today, things have changed yet again”, says Nugent.

But the road to a more transparent commodities industry didn’t happen overnight. Pivotal changes in legislation, as well as technological advancement have been fundamental in the shift.

Tragedy begets opportunity

On 24 March 1989, the Exxon Valdez spilled 250,000 barrels of crude oil into U.S. waters. In the wake of this disaster, the United States Congress passed the Oil Pollution Act, which mandated a vessel tracking system for tankers entering Alaskan waters. By 2004, all large ships were required to carry Automatic Identification System (AIS) transponders, providing real-time data on ship identity, location, speed, and direction. This not only made real-time ship tracking accessible but also introduced transparency underpinned by law.

The commodities industry quickly capitalised on this data. AIS signals, broadcast over a high-frequency radio network, can be received by anyone with the appropriate equipment. This allows traders to monitor how frequently a ship arrives at or departs from a port, and by analysing the type of vessel, they can estimate the volume of cargo being transported.

To infinity & beyond

As the cost of launching spacecraft has become more affordable, so too has the price of accessing satellite data. Arthur Richier, Head of Strategic Partnerships at Vortexa, a company specialising in using satellite data to track energy flows, mentions that in the past, large trading houses would spend millions of dollars renting satellites from governments to access crucial data. This satellite data is used to monitor everything from crop health and the movement of oil shipments to mining activities. Whether through real-time data or extrapolated trends, this information helps firms anticipate price movements and optimise their trading strategies.

Owing to the substantial costs involved, the early years of satellite intelligence were characterised by a lack of transparency, with only the biggest players affording exclusive access. However, the orbital landscape has evolved significantly since the days of Sputnik, with nearly 10,000 satellites currently in orbit and almost 60,000 expected by 2030. See all the satellites currently in orbit here.

Types of alternative data sources

These are just two examples of how the rise of alternative data sources has made the commodities industry more transparent. Generally, there are now four main types of alternative data providers:

1. Government Agencies: They manage and disseminate official data from their regions, including statistics on production, inventory, and economic indicators. For example, the Energy Information Administration publishes data on the amount of oil stored in Cushing, Oklahoma, often referred to as the “pipeline crossroads of the world.”
2. Commercial Organisations and Big Data Providers: These include firms specialising in weather or news data that collect and structure large datasets, offering insights into market conditions, climate patterns, and news sentiment. Vortexa is an example of such a company.
3. International Monitoring Organisations: These include entities like the International Maritime Organisation, which provides data such as as AIS information for shipping and maritime activities.
4. Exchanges: These platforms offer trading data and market insights relevant to price movements, trading volumes, and other financial metrics.

With the competition heating up, and data becoming increasingly more democratised, there’s one thing on everyone’s mind: Where can we find our competitive advantage?

Looking for more insights?

Read the first edition of our eBook series: The Democratisation of Data in Commodities – Foundations of Transparency

Featuring insights from current and former employees at: Trafigura – Danske Commodities – Vortexa – Tradavex – ChAI – Tallarium