Data-driven decisions are possible more than ever. Still, data science projects fail to be fully integrated into the organisation. Learn how to use the Proof of Concept approach to boost innovation regardless of your role.
Hi, I’m Nazly, connect with me on LinkedIn to share ideas and keep the conversation going!
Starting a data science project (or AI project) is part of an innovation process. It is complex and daunting. There are many reasons these types of projects fail, such as problems with data, lack of skills in the team, or, more importantly, failure to deliver value.
PoC (Proof of Concept) are a great way to start projects and test early for those pitfalls. However, PoCs also fail when they fall short of the expected outcomes.
I can imagine you have been reading and seeing AI’s possibilities and use cases, particularly of Generative AI, with ChatGPT or other technologies flooding your LinkedIn feed. You may also have taken one of our Data modules and are excited about using all your learnings and putting them into practice.
Do you know how you can start?
Build your use case.
What can you do differently daily, and where can data help you? For me, working at Growth Tribe, I am interested in:
Metrics: How many people attend my events? How many learners are currently active in a course? What is the ratio of people enrolled vs people finishing the modules?
Variations in time: How many people were enrolled in the first quarter of the year, and how many in the last quarter? Predictions: How many people will most likely finish the courses? How many people will drop out?
Filtering and slicing: Can I see the information and compare it by year? How many people from countries outside the Netherlands joined the webinars in March, June, and August? Which modules have better satisfaction scores and completion rates?
Unstructured data: Text from reviews, feedback forms, and customer service. What can we conclude about the satisfaction or improvements of our platform?
Predictions: How can we anticipate requests and recommendations for our learners?
Think about what brings the most value. Projects that help reduce costs, increase revenue or generate new lines of business provide the most value. For me, this could be:
Metrics from assistance to webinars to conversion and completion rates of courses. Which could provide a clear view of cost (time invested) and return on the investment.
Profiling the learner, understanding how they interact with the content, where there is engagement, and the most valuable topics we should focus on. Generate content that matters and increase revenue.
To get an idea of what could be interesting to pursue, it is essential to have conversations with colleagues, other departments or managers who can give their views on the needs and impact.
Gathering notes and points of view is an excellent start to collecting assumptions and business understanding testing and validation.
Consider what is technically possible. Data collection and storage could determine how complex the project will be. Here is where the team’s knowledge and skills need to be tested. For my case, I would approach the Tech department and cover some of these questions:
Where is the data? Do we have access to raw data, or are metrics already computed in systems? This can be HubSpot with conversion flows, Excel files with exports of satisfaction reviews or Website metrics.
Do we need to train the team? Have we done similar projects inside the company? Do we need to outsource talent or tools?
Inventory and mapping of the systems and capabilities will be the starting point to constraint and scope the project.
Now that we have built the case and you gathered assumptions, it is time to look at the solution.
The examples I mentioned can be huge projects, depending on the number of departments and systems involved.
The PoC is that nice spot between what is technically feasible and what gives value right away.
The key is to start small. Focusing on testing the previous assumptions.
Now, here is the twist. While working on this narrow scope for maybe one or three months...
Keep the bigger vision in mind. What a first version of this solution can look like.
To keep alignment and avoid pitfalls, think about an agile approach.
Iterations of fully functional outputs that give value and can be used.
The image below shows how a PoC is a project with different steps: Plan, Design, Develop, Test, Deploy, Review and Launch. We need to launch this test so people can use it.
In my example, a simple dashboard with the metrics of the attendants to my webinars could be a POC. When I use it to make decisions, I can test if those metrics are enough and if that information already impacts how I do my job.
After validations are tested, we can move to a Pilot or a Minimum Viable Product (MVP)—a simple, functional version of the solution. With the dashboard example, the MVP can be a web app with a simple user interface where I can access these metrics weekly. If this version works, it solves my problem and gives me value.
Read this old but interesting article on how Dropbox started as an MVP back in 2010
The first version of the solution will be the next step. A more robust system, probably a better web application, with more security and login access so other colleagues can also have access to metrics that matter to them.
The goal is to monitor the usage and see if the results are helpful. The iterative approach continues, and the solution can probably change, evolve, or integrate into other systems.
Data availability: Not only for the testing phase but also for keeping the bigger solution in mind. Validate with data engineers and check how the data architecture could be designed to be functional and scalable.
Data security: Discover where personal and sensitive data could be handled and establish policies and mechanisms to comply with regulations early.
Systems readiness: How resilient to changes and stable are the applications, databases and integrations? Again, having in mind how to scale beyond the PoC.
Usability of the solution: Monitor who and how users understand the solution and which barriers they encounter.
Team structure: Check the current team capabilities and dynamics when building the solution. Also, if it is necessary to have new hires or upskill.
Stakeholder alignment: Alignment of priorities and points of view on the problem and solution. Check for change resistance and metrics definition.
We have covered how Proof of Concepts are a great way to start a data science project, building your case while having the business value and technical possibilities on top of your mind.
This is just the start of implementing data science projects in your organisation. Remember the experimental nature of these processes. After a solution is built, monitoring and validation are still key steps, leading to iterations and changes.
Finally, we recommend you read Picnic’s article on how they see data science as a product.
And if you want to go a step further, this article by Klippa covers how to start a PoC with more technical depth.