Predicting is about telling how things will look in the future within certain error. The error must be small enough to make the prediction meaningful. Forecasting asset prices is a dilemma that has fascinated investors since the very advent of financial markets, as accurate predictions for the movement of financial markets could lead to fast and substantial trading profits.

Attempts to forecast stock prices have been numerous. Investors have used regression charts, fundamental analysis, and quantities known as indicators in an attempt to predict future stock prices. Most of these approaches are often not repeatable, and lack a systematic way for measuring its accuracy.
Other approaches, as ours, model the financial markets from first principles.
We built a simplified market mechanism. We assume two different sources of power that move the market.
First, the news. Rational expectations theory tells us that this is the only source for a change in prices.
Second, we suppose that prices do drive themselves.

If prices are too high, sellers will drive the price back down, if prices are too low, buyers will push prices back up, even in absence of news.
Market oscillations are the result of a detailed balance of expectations, generated both by news arrival and by changes in prices, of each individual that conforms the financial market.

Those who buy, do it thinking that prices will rise  further, and those who sell believe that prices will continue to fall.

When the equilibrium between those forces is broken a trend shows up. More subtle mechanisms are also at work: some investors, knowing that persistent movements do exist, act in consequence, causing financial markets movements to be exaggerated.

This mechanism comprises a series of interaction rules between agents that make up the basic unit necessary to create a model of stock markets dynamics.

To find the relationship between the price of a given asset at any moment, and the subsequent price, we can try to measure the balance of expectations, but we cannot include the change in expectations due to the news, since their arrival is in a random fashion.

At Trading Pro, we verified that the role of news in the market behavior is often overestimated, mostly because investors need an explanation for a loss or a gain.

Financial markets are complex auto organized systems composed of similar individuals, each trying to maximize their income

The arrival of news is sudden, and usually drive the market almost instantaneously to a new situation, which is the starting point of a longer lasting autonomous complex evolution which rules are written in the same prices.

Furthermore, we suppose that there are certain rules that govern the way in which the price of certain asset could evolve,plus random shocks provided by outside influences (news and events that we cannot foresee).

However, if the shocks are not continuously disturbing the market, we can glimpse into the future.

With the arrival of greater computer power, new methods are available to understand the intrinsic dynamics of the financial markets.

Tools like chaos theory, the study of complex systems, and dimensional shrinkage are new approaches to study them. There are several numbers of diverse systems in nature that exhibit very complex, apparently random behaviors that can be appropriately described by simple equations.

As an example of those systems, consider the human heart. It is a cellular swarm of individuals, very similar to each other, although not identical, interacting together all the time.

However if one observes the heart as a whole, it possess a harmonic behavior that can be described by a few equations.

This drastic reduction of the number of variables needed to describe a phenomenon is called 'dimensionality reduction'. These kind of emergent cooperative behaviors is typical in systems driven by the aggregate of a lot of interacting individuals.

Making the assumption that the financial markets are complex auto organized systems composed of similar individuals, each trying to maximize their income, we can hope that a description of low dimensionality may be suitable, and that a certain forecasting capacity is possible.

We assume that stock prices reflects all the available information, so price series should be enough to study the market.

If we accept that the news arrives in a random fashion, we cannot waste any effort trying to predict these events.

We must restrict our system, and consider that it is randomly shocked externally.

Starting from the time series of prices, following a method proposed by F. Takens, a multidimensional trajectory can be built up. Then we suppose that the observed evolution is determined by a system driven by news events and the free evolution of the intrinsic dynamics of the markets. Therefore, the actual movement is:

News-> Free evolution-> News-> Free evolution-> News-> Free evolution->

Our task is to try to understand the free evolution rules. Unfortunately, we cannot completely separate the data corresponding to free evolution of the market from the movement that is news driven. We must consider that the time series is very polluted with external noise that we must try to clean.

Our method of achieving this goal is by ignoring the details of the movement, and concentrating just on the major turning points of the price evolution in different time frames. Then we numerically look for a set of equations that can account for this behavior and we extrapolate this behavior into the near future.

In doing so we can achieve accurate forecasts of the major market turning points without being side tracked by external events.