What tracking GenAI skills can teach us about the future of work

Without effective tracking of learning progress and outcomes, businesses risk falling short of realising AI’s full potential, writes Dr Gleb Tsipursky.

Generative AI (GenAI) is reshaping the workplace, offering powerful tools for creativity, productivity, and efficiency. However, unlocking its potential hinges on more than just adoption; employees must develop a nuanced understanding of how to use this technology effectively. Organisations must go beyond traditional training approaches and embrace rigorous tracking of learning progress and outcomes specific to GenAI skills. By measuring key performance indicators (KPIs) such as skill application rates, engagement metrics, and real-world results, leaders can ensure that their teams stay competitive in this rapidly advancing field.

Why tracking GenAI skills progress is crucial

GenAI tools, from text generators to image creation platforms, require a blend of technical expertise and creative application. Without a clear system to measure how employees are learning and applying these tools, organisations risk misaligned training efforts and underwhelming outcomes. Tracking provides actionable insights that guide improvements in learning programs, ensuring employees acquire not only knowledge but also the confidence to leverage GenAI effectively.

1. Skill application rates: It’s not enough for employees to complete a training module on GenAI; organisations must evaluate how well they apply those skills in their roles. For instance, are content teams using GenAI-generated suggestions to improve efficiency, or are they ignoring its inputs, preferring to generate and edit their own content?
2. Engagement metrics: Measuring time spent on training modules, participation in GenAI simulations, and frequency of interaction with learning tools can reveal whether employees are actively engaged with the content or merely going through the motions.
3. Post-training results: The ultimate test of GenAI learning is its real-world impact. Metrics such as increased productivity, error reduction, and enhanced innovation reflect how effectively employees are utilising GenAI to meet organisational goals.

Clinical case study: Scaling GenAI skills adoption at a regional retailer

A regional retailer illustrates the transformative power of tracking GenAI learning progress. Facing mounting competition, the company sought to use AI-driven tools to improve marketing personalisation and streamline supply chain operations. However, initial adoption efforts fell short. Employees struggled to integrate GenAI applications into their workflows, and training programs yielded inconsistent results.

To address these challenges, the company partnered with me as a consultant specialising in GenAI adoption strategies. We implemented a robust tracking system with the following components:

• Baseline assessments: We tested employees on their familiarity with GenAI tools and core AI concepts before training began.
• Tailored learning modules: We customised training to address specific gaps, such as using GenAI for customer segmentation or predictive analytics.
• Real-time progress monitoring: Dashboards provided managers with insights into module completion rates, engagement levels, and assessment scores in real time.
• Outcome tracking: We also measured post-training KPIs, such as increased marketing campaign ROI and reduced inventory mismanagement.

Within three months, 87 per cent of employees reported confidence in using GenAI tools, up from just 40 per cent before training. More importantly, the retailer achieved a 15 per cent reduction in inventory errors and a 20 per cent increase in marketing campaign performance, demonstrating the tangible value of targeted, data-driven learning programs.

Identifying GenAI skills gaps

Tracking learning progress is particularly valuable in identifying skills gaps, which are often amplified when adopting complex technologies like GenAI. Many employees may struggle with specific aspects of GenAI, such as prompt engineering, interpreting AI outputs, or understanding ethical considerations. By analysing pre- and post-training assessments, organisations can pinpoint these challenges and refine their programs.

For instance, if data shows that employees consistently perform poorly on tasks related to evaluating AI-generated insights, it could indicate a need for more focused training on critical thinking and contextual judgement. Similarly, if team members excel in basic operations but struggle with advanced applications, leaders can design supplemental modules to close these gaps.

Generative AI is not a one-size-fits-all tool, and we should not approach its training in that way. Tracking learning outcomes enables organisations to personalise the learning journey for each employee, tailoring it to their specific strengths, weaknesses, and roles. Personalised learning fosters higher engagement and better retention, ensuring employees are not overwhelmed or under-challenged.

For example, a marketing analyst may need intensive training on creating compelling AI-generated copy, while a data scientist may focus more on configuring AI models for predictive analytics. Tracking data such as individual progress rates and feedback allows organisations to offer customised learning paths that adapt in real-time to employees’ needs.

Leveraging AI tools to track AI learning

Ironically, one of the best ways to track learning progress in GenAI programs is by using AI itself. Advanced learning management systems (LMS) with built-in AI capabilities can analyse employee interactions, generate insights on performance trends, and even recommend personalised training modules. These tools simplify the process of collecting, interpreting, and acting on learning data, allowing leaders to focus on strategic improvements.

For instance, AI-powered LMS platforms can flag employees who may need additional support, such as those repeatedly scoring below average on AI ethics modules. They can also identify top performers who might be ready for leadership roles in AI adoption initiatives.

Best practices for tracking GenAI learning

To maximise the impact of tracking, organisations should follow these best practices:

1. Define clear objectives: Align training goals with strategic business priorities. For GenAI, this could mean improving innovation rates, reducing repetitive manual tasks, or enhancing customer experiences.
2. Integrate real-world scenarios: Ensure training programs simulate practical challenges employees are likely to face when using GenAI tools. This bridges the gap between theory and application.
3. Foster a culture of feedback: Use both quantitative data and employee feedback to refine training programs. Understanding learners’ experiences helps fine-tune content and delivery methods.
4. Continuously review and adapt: GenAI technologies evolve rapidly, so training programs must keep pace. Regularly updating learning content and tracking mechanisms ensures long-term relevance, while managing risks.

Conclusion: Data-driven learning for the GenAI era

The rise of GenAI presents organisations with incredible opportunities – but also challenges. Without effective tracking of learning progress and outcomes, businesses risk falling short of realising AI’s full potential. By implementing robust systems to monitor skill acquisition, identify gaps, and personalise learning, leaders can ensure their teams are equipped to thrive in the AI-driven future. Tracking learning outcomes isn’t just about measurement; it’s about creating a culture of continuous growth and innovation where employees and AI work together to achieve extraordinary results.

Dr Gleb Tsipursky, called the “Office Whisperer” by The New York Times, helps leaders transform AI hype into real-world results. He serves as the CEO of the future-of-work consultancy Disaster Avoidance Experts and wrote seven bestselling books, including “The Psychology of Generative AI Adoption”.