Professor Rob Hyndman
FORECASTING - SOLVING PROBLEMS LINKED TO FUTURE UNCERTAINTIES
By Ms Nancy Van Nieuwenhove | 5th May 2019 (update* 12/02/2020)
Professor Rob Hyndman
Head of the Department, Econometrics and Business Statistics, Monash University
Research interests: time series forecasting, R programming, exploratory data analysis, machine learning
Professor Rob Hyndman is an accredited statistician with the Statistical Society of Australia. His journey into statistical consulting started at the end of his 1st year undergrad (BSc, Honours) when he started using FORTRAN, the first widely-used high-performance programming language invented in the nineteen-fifties, at a time when computer bugs were cleaned using a toothbrush. “I saw advertised on the noticeboard a job to survey people at a beach. I met the person to contact, the Director of the Statistical Consulting Centre at Melbourne University. He asked my marks and if I could program in FORTRAN, language I wrote 2 functions for, so I said I could. He told me I should work for him instead of doing the survey. So, I read a book about FORTRAN over the weekend and started working for the Statistical Consulting Centre at Melbourne Uni on the Monday. Turns out I was pretty good at it, so all through my undergrads, I worked as a statistical consultant at the same time as studying statistics. It was heaps of fun as I love using mathematics to solve real problems”.
Rob became interested in forecasting while being a PhD student at the University of Melbourne. For him, forecasting has direct relevance for planning, especially around future uncertainties. It brings together lots of things he was/is good in: mathematics, data analysis and coding. “During my PhD, the head of my department asked if I could help to run a forecasting workshop for business people in Melbourne. We ran the workshop and I learnt a lot about forecasting in the process. Then, I read that the authors of a famous textbook on forecasting were looking for a 3rd author for the 3rd edition. I was young, had nothing to lose, so I contacted them and said I’d be happy to be their 3rd author. They contacted me 6 months later asking for some of my work, I sent them some samples, and 6 months later I was in. I was just finishing my PhD and I did what you are told not to do: I wrote a textbook early in my career. And because the book was already famous, suddenly I became well-known and it opened lots of doors”.
Rob has many projects including energy analytics, data visualization, hierarchical forecasting, anomaly detection and time series forecasting. Rob is interested in forecasting and building models generally applicable across multiple industries. He started thinking about energy forecasting around 2006 when he was contacted by someone from the South Australian market operator. They needed help with long-term probabilistic forecasting of peak electricity demand. “South Australia has the most volatile electricity domain in the world, in terms of the peaks and the troughs. I developed a model that worked for South Australia, and we also tried it in Victoria. Around that time AEMO was formed, so I started working with them".
The ingredients for a good forecasting problem are: plenty of good data and clear mechanisms driving the changes. "Forecasting electricity demand is relatively easy because there are lots of meteorological data available. But as we move to renewables and to local storage, it makes forecasting difficult because you base your forecast on historical data and there are sufficiently large changes in the industry that the past is no longer a good guide to the future. We have good models for wind and solar energy generation, but they need to be adapted to local circumstances. For example, if you know that tomorrow will have 30 km/h SE wind, 25 degrees Celsius and 40% cloud cover, what does that mean for the network in terms of how much energy has been generated on the network compared to what’s going to come through wind, solar, and where you’re going to need it. What we need is an integrated model combining the various generation sources as well as a good model for total demand".
Rob is the Head of the Department of Econometrics and Business Statistics at Monash University. “I arrived at Monash in 1995. I was in the Maths department for 3.5 years and switched to the Business School in 1998 and just became the head of department 4 months ago. I’ve had research fellows, postdocs and PhD students working with me on energy-related topics, but it’s a small part of my research portfolio. Working in the Business School has been rewarding. They support high impact research that can be immediately implemented in practice, good publications in top journals, and it is well-funded. What I like doing is to work with an organisation, understand their problem, come up with solutions that they can implement, and then continue to work and write articles inspired by the ideas generated in the consulting project.”
Rob has learned a few languages over the years, but these days he mainly works in R. “I was using R long before it became popular, which means I got to write a lot of the tools around forecasting that are now widely used in R. I do not want my work to be locked in a commercial product, so I tend to use open source products, for my research to be openly accessible anywhere in the world”.
To go further:
- Feature-based forecasting algorithms for large collections of time series https://www.youtube.com/watch?v=IRCfeG2_SKs
- Hyndman, RJ, XA Lin, and P Pinson (2018). Visualizing Big Energy Data: Solutions for This Crucial Component of Data Analysis. IEEE Power and Energy Magazine 16(3), 18–25. https://robjhyndman.com/publications/visualizing-big-energy-data/
- * Rob J Hyndman, Shu Fan (2010) Density forecasting for long-term peak electricity demand. IEEE Transactions on Power Systems 25(2), 1142-1153. https://robjhyndman.com/publications/peak-electricity-demand/
- * Hyndman, RJ and G Athanasopoulos (2020). Forecasting: principles and practice. 3nd edition. Melbourne, Australia: OTexts. OTexts.org/fpp3
- * Souhaib Ben Taieb, James W Taylor, Rob J Hyndman (2020) Hierarchical Probabilistic Forecasting of Electricity Demand with Smart Meter Data. J American Statistical Association. https://robjhyndman.com/publications/hpf-electricity/
- * Twitter: @robjhyndman