Dr Ed Wakelam

PhD Dissertation: The Application of Data Mining Techniques to Learning Analytics and its Implications for Interventions with Small Class Sizes
University of Hertfordshire
May 12, 2020

There has been significant progress in the development of techniques to deliver effective technology enhanced learning systems in education, with substantial progress in the field of learning analytics. These analyses are able to support academics in the identification of students at risk of failure or withdrawal. The early identification of students at risk is critical to giving academic staff and institutions the opportunity to make timely interventions.
This thesis considers established machine learning techniques, as well as a novel method, for the prediction of student outcomes and the support of interventions, including the presentation of a variety of predictive analyses and of a live experiment. It reviews the status of technology enhanced learning systems and the associated institutional obstacles to their implementation and deployment.
Many courses are comprised of relatively small student cohorts, with institutional privacy protocols limiting the data readily available for analysis. It appears that very little research attention has been devoted to this area of analysis and prediction. I present an experiment conducted on a final year university module, with a student cohort of 23, where the data available for prediction is limited to lecture/tutorial attendance, virtual learning environment accesses and intermediate assessments. I apply and compare a variety of machine learning analyses to assess and predict student performance, applied at appropriate points during module delivery. Despite some mixed results, I found potential for predicting student performance in small student cohorts with very limited student attributes, with accuracies comparing favourably with published results using large cohorts and significantly more attributes. I propose that the analyses will be useful to support module leaders in identifying opportunities to make timely academic interventions.
Student data may include a combination of nominal and numeric data. A large variety of techniques are available to analyse numeric data, however there are fewer techniques applicable to nominal data. I summarise the results of what I believe to be a novel technique to analyse nominal data by making a systematic comparison of data pairs.
In this thesis I have surveyed existing intelligent learning/training systems and explored the contemporary AI techniques which appear to offer the most promising contributions to the prediction of student attainment. I have researched and catalogued the organisational and non-technological challenges to be addressed for successful system development and implementation and proposed a set of critical success criteria to apply.

"The Mining and Analysis of Data with Mixed Attribute Types"
IMMM 2016: Sixth International Conference on Advances in Information Mining and Management
May 22, 2016

Abstract:
Mining and analysis of large data sets has become a major contributor to the exploitation of Artificial Intelligence in a wide range of real life challenges, including education, business intelligence and research. In the field of education, the mining, extraction and exploitation of useful information and patterns from student data provides lecturers, trainers and organisations with the potential to tailor learning paths and materials to maximize teaching efficiency and to predict and influence student success rates. Progress in this important area of student data analytics can provide useful techniques for exploitation in the development of adaptive learning systems. Student data often includes a combination of nominal and numeric data. A large variety of techniques are available to analyse numeric data, however there are fewer techniques applicable to nominal data. In this paper, we summarise our progress in applying a combination of what we believe to be a novel technique to analyse nominal data by making a systematic comparison of data pairs, followed by numeric data analysis, providing the opportunity to focus on promising correlations for deeper analysis.

"The Potential for Using Artificial Intelligence Techniques to Improve e-Learning Systems"
European Conference on e-Learning (ECEL)
October 01, 2015

Abstract: There has been significant progress in the development of techniques to deliver more effective e-Learning systems in both education and commerce but our research has identified very few examples of comprehensive learning systems that exploit contemporary artificial intelligence (AI) techniques. We have surveyed existing intelligent learning/training systems and explored the contemporary AI techniques which appear to offer the most promising contributions to e-Learning. We have considered the non-technological challenges to be addressed and considered those factors which will allow step change progress. With the convergence of several of the required components for success increasingly in place we believe that the opportunity to make this progress is now much stronger. We present a description of the fundamental components of an adaptive learning system designed to fulfil the objectives of the teacher and to develop a close relationship with the learner, monitoring and adjusting the teaching based upon a wide variety of analyses of their knowledge and performance. This is an important area for future research with the opportunity to deliver significant value to both education and commerce. The development of improved learning systems in conjunction with trainers, teachers and subject matter experts will provide benefits to educational institutions and help commercial organisations to face critical challenges in the training, development and retention of the key skills required to address new, emerging technologies and business models.

The collection, analysis and exploitation of footballer attributes: A systematic review
Journal of Sports Analytics
March 11, 2022

There is growing on-going research into how footballer attributes, collected prior to, during and post-match, may address the demands of clubs, media pundits and gaming developers. Focusing upon individual player performance analysis and prediction, we examined the body of research which considers different player attributes. This resulted in the selection of 132 relevant papers published between 1999 and 2020. From these we have compiled a comprehensive list of player attributes, categorising them as static, such as age and height, or dynamic, such as pass completions and shots on target. To indicate their accuracy, we classified each attribute as objectively or subjectively derived, and finally by their implied accessibility and their likely personal and club sensitivity. We assigned these attributes to 25 logical groups such as passing, tackling and player demographics. We analysed the relative research focus on each group and noted the analytical methods deployed, identifying which statistical or machine learning techniques were used. We reviewed and considered the use of character trait attributes in the selected papers and discuss more formal approaches to their use. Based upon this we have made recommendations on how this work may be developed to support elite clubs in the consideration of transfer targets.

The Collection, Analysis and Exploitation of Footballer Attributes: A Systematic Review
Journal of Sports Analytics
January 20, 2022

There is growing on-going research into how footballer attributes, collected prior to, during and post-match, may address the demands of clubs, media pundits and gaming developers. Focusing upon individual player performance analysis and prediction, we examined the body of research which considers different player attributes. This resulted in the selection of 132 relevant papers published between 1999 and 2020. From these we have compiled a comprehensive list of player attributes, categorising them as static, such as age and height, or dynamic, such as pass completions and shots on target. To indicate their accuracy, we classified each attribute as objectively or subjectively derived, and finally by their implied accessibility and their likely personal and club sensitivity. We assigned these attributes to 25 logical groups such as passing, tackling and player demographics. We analysed the relative research focus on each group and noted the analytical methods deployed, identifying which statistical or machine learning techniques were used. We reviewed and considered the use of character trait attributes in the selected papers and discuss more formal approaches to their use. Based upon this we have made recommendations on how this work may be developed to support elite clubs in the consideration of transfer targets.

The potential for student performance prediction in small cohorts with minimal available attributes
British Journal of Educational Technology
June 25, 2019

The measurement of student performance during their progress through university
study provides academic leadership with critical information on each student’s
likelihood of success. Academics have traditionally used their interactions with
individual students through class activities and interim assessments to identify those “at
risk” of failure/withdrawal. However, modern university environments, offering easy
on-line availability of course material, may see reduced lecture/tutorial attendance,
making such identification more challenging. Modern data mining and machine
learning techniques provide increasingly accurate predictions of student examination
assessment marks, although these approaches have focussed upon large student
populations and wide ranges of data attributes per student. However, many university
modules comprise relatively small student cohorts, with institutional protocols limiting
the student attributes available for analysis. It appears that very little research attention
has been devoted to this area of analysis and prediction. We describe an experiment
conducted on a final-year university module student cohort of 23, where individual
student data are limited to lecture/tutorial attendance, virtual learning environment
accesses and intermediate assessments. We found potential for predicting individual
student interim and final assessment marks in small student cohorts with very limited
attributes and that these predictions could be useful to support module leaders in
identifying students potentially “at risk”.

"Developing an Agent-Based Simulation Model of Software Evolution"
Journal of Information and Software Technology
April 08, 2018

Abstract
Context: In attempt to simulate the factors that affect the software evolution behaviour and possibly predict it, several simulation models have been developed recently. The current system dynamic (SD) simulation model of software evolution process was built based on actor-network theory (ANT) of software evolution by using system dynamic environment, which is not a suitable environment to reflect the complexity of ANT theory. In addition the SD model has not been investigated for its ability to represent the real-world process of software evolution.
Objectives: This paper aims to re-implements the current SD model to an agent-based simulation environment ‘Repast’ and checks the behaviour of the new model compared to the existing SD model. It also aims to investigate the ability of the new Repast model to represent the real-world process of software evolution.
Methods: a new agent-based simulation model is developed based on the current SD model's specifications and then tests similar to the previous model tests are conducted in order to perform a comparative evaluation between of these two results. In addition an investigation is carried out through an interview with an expert in software development area to investigate the model's ability to represent real-world process of software evolution.
Results: The Repast model shows more stable behaviour compared with the SD model. Results also found that the evolution health of the software can be calibrated quantitatively and that the new Repast model does have the ability to represent real-world processes of software evolution.
Conclusion: It is concluded that by applying a more suitable simulation environment (agent-based) to represent ANT theory of software evolution, that this new simulation model will show more stable behaviour compared with the previous SD model; And it will also shows the ability to represent (at least quantitively) the real-world aspect of software evolution.

Footballer analytics – the elephant in the room
Thinkers360
April 26, 2022

When we reviewed the past 20+ years of research on footballer analytics (check out The collection, analysis and exploitation of footballer attributes: A systematic review) we identified a startling 1500 different player attributes being used in analyses. Unsurprisingly, they included all the usual dynamic suspects such as pass completion, interceptions, shots, tackles and the usual demographic attributes such as age, height, sprint speed, power. But where are the character attributes that the commercial world has tried to incorporate into their selection of people for over 100 years?

These are often the ones that fans and pundits speak about the most – determination, performance under pressure, motivation, tenacity, persistence, guile, relentlessness…? To be fair, there were some, but a very small proportion – less than 5%. 

AS Monaco’s Technical Director, Laurence Stewart, stepped up and made the point effectively in his 2021 video that identifies that Scouting character is the next frontier. As Laurence points out, the player recruitment process doesn’t provide the interviewing opportunities that the commercial world enjoys and yet the fit of a player to club culture and principles is a key risk factor for many clubs.

It's clearly a scary topic for established footballer recruitment teams. The nature of transfer discussions and negotiations is already a tough and stressful process without adding this dimension, but over the past two years of research and discussions with those involved I’m convinced that getting a handle on such attributes is the future. The key is how to do it with some formality and how to practically incorporate this dimension into the comprehensive analyses that club recruitment teams already have in place. 

We’re working on this now, developing our research into how such attributes can be usefully included in footballer analyses to formalise what we believe the very best coaches and club recruitment teams do naturally, maybe even subconsciously. 

Perhaps by providing club recruitment teams with some reliable tools to support their inclusion we can turn the elephant in the room into that extra competitive edge that will inevitably result from exploiting these attributes before their rivals do. Crucially, the real elephant in the room will be the integration of the highest ethical standards and this is something that university ethics departments excel at. 

Footballer analytics – what does the past 20 plus years of research tell us?
Thinkers360
March 24, 2022

But what’s it all for? Reflect upon last October’s prediction for 2022 and the background to why elite clubs have no choice but to become more serious about their analytics (Thinkers360 Predictions Series – 2022 Predictions for Artificial Intelligence). 

Despite the good intentions, clubs may still be limited by long established scouting and analyses structures and methods, in some cases with resistance in their organisations to even discussing, never mind adopting modern methods and it will take executive leadership and drive to change this.

Lawlor, Rookwood and Wright’s excellent 2021 paper “Player scouting and recruitment in English men’s professional football opportunities for research” discusses this mixed progress in clubs and calls for more research. A positive development shown by my own research, performing a systematic review of all the relevant literature, shows a steep increase in the number of studies involving footballer analytics research in the past twenty-seven years.

In addition, there is certainly no shortage of footballer attributes to be measured, with over 1,500 different attributes identified, although we must be careful to identify which are objective and measurable and which are based upon people’s opinions and therefore subjective. This is not because we shouldn’t value subject matter expert opinion, indeed this wealth of experienced judgement is essential, but we need to understand the blend of objective and subjective, to successfully apply our analyses. The research also shows that there appears to be scope for increasing and intensifying the application of modern machine learning analyses given that 65% of the papers identified solely applied statistical techniques and only 21% applied Machine Learning techniques with the remaining 6% applying a mixture of both. 

The future is bright – but only for the club executive teams who grasp the nettle. If you’d like to learn more, check out The collection, analysis and exploitation of footballer attributes: A systematic review.

Did Covid accelerate learning analytics in our universities?
Thinkers360
March 26, 2021

Image by Pexels from Pixabay So, did universities grasp the opportunity to mitigate student drop-outs by increased adoption of learning analytics? It’s not yet clear if institution’s responses to the pandemic included increased use of learning analytics to monitor their students’ progress given the increase in remote learning. Those that did are much more likely to identify those at risk, and consequently to make positive interventions and potentially reduce drop-outs. The UK’s Higher Education Statistics Agency reports a total of 6.7% of full-time first year degree students dropping out in the academic year 2018/19. In Australia and the US drop-out rates are typically higher, for 2018, Australia Higher Education Department reported 18% and the US National Center for Education Statistics reporting 24% for 2018. Consequential impacts on the students and their families can be significant and these must be our first concern, but the financial impacts on the universities are substantial. In the case of the UK, if a student drops out early in their first year, at worst, the university loses three years of around £9,000 fee revenue. With almost 327,000 first year students, the national drop-out level of 6.7% translates into an almost £200M hit to total university budgets. We won’t know the aggregate national statistics for university student drop-out rates for academic years 2019/20 or perhaps more interestingly those for 2020/21 for months, but it may be that administrations should be preparing for the worst if they haven’t yet taken learning analytics initiatives. Why? Because a glance at the established research shows that historically the very factors having the most significant effects on drop-outs are those that are most likely to be the ones exacerbated by Covid. Atif et al.’s 2015 paper “Student Preferences and Attitudes to the use of Early Alerts” described research showing that in the schools of Computing, Physics and Mathematics at Macquarie University, Sydney, Australia, the highest three factors affecting student performance were Emotional Health (48%) and Family Responsibility / Commitments (28%), followed by Financial Issues (21%). These would appear to be three of the highest components of people’s lives likely to be directly affected by the pandemic. This research, supported by similar research focussing upon Open University student performance (Castles, 2004, “Persistence and the adult learner: Factors affecting persistence in Open University students”) indicates that academic issues such as subject knowledge or difficulties of understanding topics are much less important factors in poor performance and hence drop-outs. Castles’ research corroborates Atif’s 2015 survey of the factors affecting student performance, albeit with a different categorisation of factors. Worryingly, the OU’s research is of course directly relevant to on-line/remote learning – and it is on-line/remote learning that has very significantly increased by Covid restrictions on universities. The Open University has world class learning analytics, identification of students at risk and positive intervention systems and processes, so they have the tried and tested capabilities to take care of their students - as do the other institutions who already had learning analytics in place before the pandemic. It could be that the a proportion of the increase in on-line/remote learning driven by the pandemic sticks when it’s over and institutions go back to normal, whatever “normal” is. The trend had already been for increasing on-line availability of teaching material, providing flexibility for students to match the demands of their lives, and with the consequent benefits to academic staff and institutional infrastructure. This shift makes the implementation of learning analytics a necessity, not a nice to have. Those institutions that post the effects of the pandemic have recognised the value of implementing learning analytics systems deserve the benefits that the implementations will give their students and themselves. For those yet to take the first steps, now is the time. Remember – “The best time to plant a tree was 20 years ago. The second best time is now”. I did try to point the way. (If you’d like to learn more, check out The Potential for Student Performance Prediction in Small Cohorts with Minimal Available Attributes, if you’d like to learn a lot more, and you are somewhere where falling asleep would not be hazardous to your health, then The Application of Data Mining Techniques to Learning Analytics and Its Implications for Interventions with Small Class Sizes).

How’s your recruitment going? In these tough times, it’s even more important to get the best people on board.
Thinkers360
October 29, 2020

[Image by Gerd Altmann from Pixabay] It’s a fallacy to assume that all businesses aren’t recruiting at the moment. The old saying of “when the going gets tough, the tough get going” has merit and the time to focus on what’s important to your business is always now. Whenever I had to downsize a business, my imperative to identify top class people to improve the business and make it more efficient was unchanged and remained a parallel activity. But, I have a concern that otherwise excellent organisations are drifting down a damaging path. I’m seeing a disturbing and in my opinion self-destructive trend in some corporations of reducing at least the early stages of their recruitment processes to mechanistic and technical evaluations of candidates in the name of “filtering”. Worse, some executive teams are delegating the process down to their well-intentioned junior levels and sometimes to their HR functions alone. Don’t get me wrong, strong HR functions are a critically important component of successful organisations, and in every one of my three corporations, I was very lucky to have great support, advice and guidance. But, HR functions need reciprocal support and guidance to stay aligned in the way that the business needs. In my experience it’s the corporations that are most careful to measure and select new potential recruits, at every level, to the culture of their organisations that succeed and thrive. If the candidate isn’t at least a close fit then don’t take the risk. Modern methods of psychometric testing are at best an aid, but no substitute for experienced interviewers who understand the requirements of the job and the culture of the organisation. When I’m advising candidates on job applications and interviews, I emphasise that the technical requirements of the job are only one of several criteria they’re being measured on. With a few exceptions, technical skills can be honed or trained. The best organisations are looking for evidence of the really important qualities such as being able to work as part of a team, communication ability, judgement, leadership, problem solving and motivation. The best interviewers will look for evidence of these – whether in previous work experience or through their personal lives. And whether you are a candidate or a recruiter, bad experiences and how they were turned around or worked through are just as important as the positive ones. For example, whenever I was trying to recruit a project manager for a key programme and I had two candidates of very similar qualities and skills, but one had experience of managing and finally delivering a project that had gone badly wrong (significantly over-running on timescale and budget) that’s the one I’d potentially be most interested in. Either, that candidate had the skills and experience of turning around an inherited disaster project, or perhaps it was their own lack of experience had led to the project they’d managed from the start going wrong. In which case, what had they learnt that would stop them falling into the same traps? If someone has only ever dealt with success, then that might be a great indicator of their future performance, but would they be able to recognise when a programme was going in the wrong direction? And, if they did, would they know what to do? On the other hand, if after some serious interrogation it was clear that the candidate had been through a disaster project and could explain the story, the resolution and the lessons they had learnt that could mean that they were the best choice. This is a terrific example of why businesses must have strong experienced interviewers who know the role they are trying to fill inside out. For my part, given a straight choice between the candidate who had never had to deal with difficulties and one that had, provided I was certain that they weren’t an on-going part of the programme problem, I’d always go for the latter. If you’re a candidate, be aware of this and don’t be afraid to discuss issues you have faced and what you learnt from them. Prepare yourself to describe the issues, obstacles and difficulties you have had to face and how you overcame them successfully. They are as valuable to your future employer as the positive ones and the best organisations will be asking questions to elicit and explore examples. My message to corporate executives is that the wrong recruits, at whatever level, are often a drag on your organisation for years. Take note – YEARS. Audit your processes, top to bottom, it’ll only take a week or two at most and do whatever root and branch changes you need to – and do it now. The best candidates should and will think very hard before accepting a job with organisations who don’t demonstrate an interest in establishing whether they have these qualities. And it’s the best candidates you are after. ###

Covid-19 and the opportunity to mitigate university student drop-outs through the adoption of learning analytics
Thinkers360
September 24, 2020

It has always seemed to me that the meaning of the proverb “The best time to plant a tree was 20 years ago. The second best time is now” is too deep for a lot of people. But, for those that get it, it can be a life changer. For too many years, understandable legal, ethical and moral fears by university leadership have tended to paralyse progress in the deployment of learning analytics to support their students. There are laudable exceptions, the UK Open University is a world leader in the successful deployment of effective learning analytics OU Analyse and the UK’s Jisc organisation continues to deliver first class leadership and support to universities in this field. These fears coupled with the inevitable institutional challenges and barriers to change and the understandable concerns of teaching staff that these analytics might be used to make judgements of their own performance, are legitimate, but can all be addressed. In fact, the widespread use of AI based learning analytics is inevitable and not too far away. Why? As always, it will be money that will drive change. A UK student dropping out of their degree in the first term can represent as high as a £30,000 irreplaceable hit to the university budget. On average, over 6% of UK first year students drop out during their first year, representing a staggering UK wide total impact to university budgets of between £600M and £1Bn. In Australia and the US the statistics are far worse, with over 20% of students leave their studies by the end of the first year. Not the noble reason we’d all like, but nevertheless the driver needed to achieve the noble goals. In response to the dreadful effects of Covid, Universities have responded magnificently to support students in their return to study, with a major component of that response focused upon remote learning wherever appropriate and feasible. The risk of course is that academic staff’s ability to recognise students at risk is diminished by the reduced face to face time and we may expect drop-out rates to increase. This risk had already been increasing as student attendance at lectures and tutorials has steadily reduced as on-line availability of teaching material has increased. Universities can expect to see the actual impacts in 2021, but past evidence of student drop-out rates suggests that those institutions who exploit learning analytics to predict potentially at risk students may reduce the potential damage to their budgets. It’s worth noting that the evidence shows that the vast majority of reasons for a student struggling have nothing to do with their academic ability, and everything to do with outside factors, ranging from emotional health issues and financial commitments to lack of support. The later that academic staff identify that an issue exists the more likely it becomes that a student may be lost. Key however is that if academic staff are able to detect issues there can be ways to provide timely support to the student and help them to continue. Now is the time for universities to implement AI led learning analytics – 20 years ago would have been ideal, but like planting a tree …. (If you’d like to learn more, check out The Potential for Student Performance Prediction in Small Cohorts with Minimal Available Attributes), if you’d like to learn a lot more, and you are somewhere where falling asleep would not be hazardous to your health, then The Application of Data Mining Techniques to Learning Analytics and Its Implications for Interventions with Small Class Sizes).

"Understanding How Corporations Align Their Strategies and Objectives Ensures Career Development Ahe

Location: Anywhere    Date Available: March 23rd, 2018     Fees: For discussion

Submission Date: March 23rd, 2018     Service Type: Service Offered

How to succeed in your career. ("Understanding How Corporations Align Their Strategies and Objectives Ensures Career Development Ahead of People That Don't")

"Strategy the Business and Me (well actually, you)"

Location: Anywhere    Date Available: March 23rd, 2018     Fees: For discussion

Submission Date: March 23rd, 2018     Service Type: Service Offered

Understanding how businesses and organisations work, top to bottom and how to maximise your own contribution and development.

Creating business value and profitability with Artificial Intelligence and Data Mining techniques

Location: London, UK    Date Available: March 23rd, 2018     Fees: For discussion

Submission Date: March 23rd, 2018     Service Type: Service Offered

Creating business value and profitability with Artificial Intelligence and Data Mining techniques