Humility, creativity and curiosity
Humility, creativity and curiosity are, I believe, qualities to aspire to when navigating today’s evolving field of information analysis. One needs to embrace the complexity of the scope and embrace the many dimensions of the challenge.
I spent the past 25 years in the financial industry, involved in developing quantitative models, managing software development project, creating new financial product to help people save and invest. Here are some topics which I have been interested to address and continuously learn about as new approaches and exciting new tools and frameworks emerge:
If you have a question, ask it: Especially true in the quantitative modelling world key ideas and frameworks are often times assumed to be known and understood. For instance, in finance, risk is the standard deviation of returns, Modern Portfolio Theory and Black and Scholes option pricing model are part of introduction financial modelling. Few quantitative analysts are comfortable publicly asking what the underlying assumptions are and are they really well understood. A data scientist would likely hesitate to wonder openly what the assumptions behind an ordinary least square regression are and how well understood they are. I believe it is a sign of intelligence and thoughtful insight to understand the underpinnings and assumptions behind theories and analytical frameworks; especially as simple techniques are now readily available next to more obscure ones on more user-friendly platforms such as Python, R, Excel and cloud services.
The curse of equilibrium, liquidity and rationality: Three words often cited and assumed to be widely understood and accepted. Having been trained in Economics these are some key foundational concepts behind most models both conceptually and in practice when organizations use them to justify policies and services.
One has to distinguish the utility of the idea of an equilibrium to ground an economic model and help focus attention on key behavior and mechanics. This ‘reversion to the mean’ concept is helpful but always has to be contrasted to the real, complex, world where things tend to get messy (captured by the concept of entropy). Tightly connected to this concept is liquidity which captures how smoothly information, things and prices evolve. This is a word that describe a key but fleeting idea. In financial markets, one definition that I find helpful is that Liquidity captures time and price specificity: whether buying or selling a financial instrument, can one be offered a price at any point in time?
Even a small break in this temporal relationship can have painful consequences when it is assumed away. Most models and machine learning techniques do have the smooth behavior implied in liquidity embedded in their assumptions.
Human nature and behavior needs to be embraced. A lot has been studied about behavioral psychology and only recently, useful frameworks help go beyond the numerous insightful psychological experiments.
It is telling that business and research feel that they strive to help people correct their inherent ‘bias’ in decision making and behavior. An analyst, researcher or product designer strive to frame and understand a need and often benchmark an objective that is part of a model or concept. Assumptions about some ideal, optimal, ‘equilibrium’ are made and help figure out how to steer an individual behavior and decisions away from their bias or ‘irrationality’. While the goal to help people make better decision is very helpful, I believe we are still too exclusively focused on the conceptual objective (function) and might be missing out by not understanding and embracing the behavioral reality. This requires to humbly step back and ask questions about the data, observations and model framework we are using.
We continuously re-discover old original concepts: A lot can be discussed about studying the history of the world. For example I was fascinated to read about the dark middle ages, in the early centuries AD. Seafaring merchants colonies in Northern Europe established payment systems that effectively did what modern commercial banks do and created longevity insurance pension systems to provide for the merchant’s extended family to cover long and deadly sea voyages. One can get a lot of deep insight in current financial product and commercial methods by studying the past.
Machine learning and Artificial Intelligence, beyond the hype, offer opportunities and pitfalls: Statistical models need data and available data needs visualization and statistical models to be understood and used. Up until recently, say 2010 to stake the timeline, large quantity of data was difficult to access and manage. The focus of analytics was primarily on the modeling and econometrics. Ensuring that a model, risk management and optimization framework were well thought out, structured and capture the behavior of the ‘typical’ producer, consumer, investor, etc... In the past decade, data is generated and stored by organizations and individual alike and vast storage and computing power are now accessible by all for a few hundred dollars. The pendulum is swinging away from complex models to techniques that aim at looking at the data to infer its structure and underlying ‘message’. Interestingly, the spotlights has shifted from focusing on the quant\math skills to the data management, visualization and computer science skills. This is the area of Machine Learning and Artificial Intelligence. These names have a marketing hype raison d'être. Machine Learning and Artificial Intelligence are terms that cover many optimization techniques such as many types of classifications, regressions, (neural) networks. All are geared to find and gauge patterns and similarity between observations. These data points range from structured metrics (returns, prices, heart rates, etc…) to unstructured information (eg text, images, sound, …). Most, if not all, have been used in the math, statistical and science community for a long time, for example: gradient descent optimization in operations research, statistic classification in statistics, topology in math. One area I am intrigued by and want to learn more about is data sonification by which very high volume of streaming data can be transformed to sounds to help researchers listen to it and detect abnormal patterns (NASA\JPL among others have used it).
That leads me to an exciting opportunity that might be unsettling to some in the Analyst, Computer Science and Data Management communities: silos are being broken.
Anybody needs to be exposed to the ‘hard’ and ‘soft’ sciences, aware of the technology solutions available and learn how to communicate better. Of course, one might focus more on one area but can no longer blissfully ignore these other disciplines.
For instance, a Quantitative Analyst can no longer craft her econometrics models with her own data infrastructure on the computer under her desk. The flexibility offered by new computing ‘cloud’ platforms require her to learn solid concepts of efficient data management and ask questions and collaborate with the database architect and programmer. Alternatively, database experts and computer programmers need to learn about the concepts and ideas behind econometric and mathematical models that connect and interact with the data. Finally and critically, all need to be open minded to creative ways to display information, upstream to look at the raw input data into a model, and downstream to communicate effectively the information and action items resulting from the models.
Cheers
Eric Penanhoat