The demand for data-based approaches in education steadily grows, as more institutions aim to leverage data to identify gaps, personalize learning and allocate resources. However, while methods such as A/B testing has become an essential part of the development of digital products, from e-commerce to advertising, their use in physical services, especially education, remains rare.

The project implemented by Kseniia Lukina provides a ground-breaking example of bridging the gap between education and technology. As a talented data analyst and ML engineer with diverse experience of implementing cutting-edge digital solutions in edtech, fintech, and e-commerce, currently working as CTO of CAM Edu and Senior Analyst at Tabby, she led the ambitious adaptation of the Growthbook open-source platform into an offline educational environment, developing an innovative and highly efficient hybrid Edtech model.

The result is not only higher engagement and improved learning experience for students at CAM Edu but a blueprint for any physical business that wants to put AI and analytics at the core of its growth. Today, the way CAM Edu operates makes it a  technology-driven business where proprietary digital tools and data experimentation systems lie at the core of customer-facing services and its product-centric strategic decisions. In this conversation, we explore how she made digital experimentation work in the world of in-person classes, providing the students of CAM Edu with a unique experience. 

Kseniia, your extensive work spans across analytics and MIL engineering. What influenced your idea to implement Growthbook at CAM Edu?

Regardless of the domain, I always focus on using data to make better decisions. At CAM Edu, we were running after-school and summer programs in offline format. Estimating real-world outcomes, such as student engagement or class fill rates, can provide a difficult task in a physical environment, and we need a more efficient way to gather data to base our decisions upon. Eventually, it played a crucial role not only for internal improvements but also for building a digital infrastructure capable of scaling as a core value-generating product. 

Although Growthbook is built for digital products, its architecture is flexible, so I saw the potential to repurpose it. At that time, I found the idea of bringing that same experimental rigor to a physical classroom challenging but very inspiring at the same time, and it motivated me throughout the many months that we have been working on Growthbook implementation.

How did you implement the technical aspect of the project, adapting Growthbook, which is designed for digital SDKs, to the real-world learning space with no screens or real-time digital inputs?

Since we couldn’t rely on SDKs in our scenario, we had to find an alternative approach. To resolve this issue, we constructed a hybrid architecture linking offline triggers to digital data pipelines. The system was built around feature flag configuration in Growthbook. Instead of attaching feature flags to software elements as is usually the case, we attached them to student group profiles in our CRM. As a result, flags get mapped to physical aspects of the process, such as instructor schedules.

The task of identifying and exposure tracking was solved in a similar fashion, by using the parent’s phone/email or the student ID as a consistent identifier. Using QR check-ins, instructor forms, and survey links, we had been capturing “exposure” data, mimicking digital events such as views, clicks, or conversions. As a result, this approach allowed us to turn our internal experimentation into a modular, scalable system that could serve as the basis for a fully fledged digital product. Moreover, this system has evolved way beyond its original internal use: it became a digital engine that supports decision-making, provides the necessary customization and can be used as a foundation for monetizable services, while being integrated into operations, yet is also valuable as a standalone revenue-generating asset.

Implementing an experimental framework in an offline setting sounds like a challenge in itself. What were some of the biggest hurdles you encountered, and what helped you to overcome them?

There were some challenges related both to the technical and cultural aspects of the project. Firstly, unlike a website, physical business does not generate clean, timestamped events. Consequently, we need to approach the issue creatively and develop a hybrid architecture that will bridge the gap through QR codes or CRM hooks.

Secondly, to implement the project successfully, we had to overcome some skepticism coming from the instructors. Educators, especially those specializing in working with offline classes, often have a negative attitude towards data-driven approaches, considering them too “cold” or “formal”. We needed to demonstrate that it is not true, and, moreover, a data-driven approach can provide a steady foundation for all the creative ideas and suggestions coming from the instructors.

Combining this approach with the methods that I’ve already mentioned, we demonstrated that offline environments could emulate the logic of online platforms through data infrastructure, careful segmentation, and flexible flag-based experimentation.

What impact did this project have on CAM Edu’s operations and strategy? What were the most significant practical outcomes that you got out of these experiments?

The most significant changes affected our decision-making processes at CAM Edu. Before implementing the experimentation, many decisions were taken based on instincts or precedents. While there is nothing inherently incorrect with this approach, to thrive in a highly competitive and constantly evolving environment it is already not enough. Experiments provided us with more objective data to base our decision-making upon. Moreover, it positively affected the culture of the company, transforming it into what can be called a data-driven culture, where curiosity and experimentation are backed by evidence, and team members are eager to suggest new creative solutions and innovative ideas rooted in objective data.

Here is one practical example. One of our tests was between two session formats: a traditional 90-minute session and a 45-minute module with a take-home kit. We discovered that the shorter format boosts attention and retention, with the satisfaction score increasing by 23,5%. Another experiment clearly demonstrated that emotional, story-driven messaging in our emails to parents more than doubled the email open rate in comparison to factual messages. These insights help us to improve the educational experience for more participants and continuously improve our program, building upon the system we have established and turning it into a scalable product with different possible applications. 

Furthermore, in addition to offline program organization, CAM Edu also offers a subscription-based online after-school service. All A/B tests, whether they are related to content delivery, communication or session structure, are conducted and evaluated online, forming a cohesive layer of digital experimentation that behaves like a standalone product in itself. The dual model creates a solid foundation for the company to explore hybrid formats, as the saturation of the online formats post-pandemic makes it clear that a fully virtual model may not suit all learners. This further highlights the need for adaptable digital tools that complement and enhance offline formats rather than replace them entirely, similar to what was achieved at CAM Edu. 

In conclusion, what advice would you give to other companies, especially those working offline, if they want to follow in the steps of CAM Edu and bring digital tools and data-driven decisions into offline spaces?

It is important to remember that innovations start not with the technical challenges but with the mindset shift, as implementing novel approaches often requires reimagining and rethinking existing processes. Moreover, you don’t need to jump into massive changes right away or implement cutting-edge AI technology to be data-driven. Even some basic or small-scale experiments, when tracked consistently, can provide powerful insights that establish a foundation for data-driven decision-making.

Be prepared to develop and improve the solution you want to implement continuously. Start with the tools you already have and build from there, implementing changes iteratively. This was the approach that allowed us not only to implement Growthbook in a physical environment but also to develop the project further, increasing testing velocity and increasing the number of experiments from 3-4 to 20 per month. 

While starting with smaller-scale experiments, this approach represents the mindset shift that can ultimately serve as a foundation for a larger-scale business transformation. For businesses that were previously focused on traditional offline operation, it unlocks the potential of a hybrid format, with the benefits of data analytics, automation, and other advanced technologies, enhancing both user experience and commercial outcomes. In doing so, businesses can evolve into tech companies: staying firmly rooted in the physical world, but powered by digital products that drive strategy, growth and revenue. CAM Edu  provides a vivid example of how it can be implemented, while transitioning from a traditional offline provider into a digital-first business that integrates experimentation and automation into different aspects of its operation. This is why I believe that this hybrid approach is becoming the new norm, allowing businesses to build an experimentation layer to the real world, thus creating a strong base for more efficient and precise decision-making.