Introduction
Welcome back to the success story interview series with a successful data scientist and our DataHour Speaker, Vidhya Chandrasekaran! In today’s data-driven world, data scientists play a crucial role in helping businesses make informed decisions by analyzing and interpreting data. With their expertise in statistics, machine learning, AI, and programming, they are able to extract meaningful insights from complex datasets.
This article features an interview with Vidhya, a successful data scientist who currently leads the Machine Learning and AI products at PayPal. With over 18+ years of experience, Vidhya shares insights into her career journey, the challenges she faces as a leader in applied data science, and her strategies for fostering innovation in her team.
She also discusses the importance of proper planning in AI projects. Also, shares advice from her mentors on trying new things and failing fast. So, let’s embark on this thrilling interview session with Vidhya.
Interview Excerpts with Vidhya Chandrasekaran
AV: Hello Vidhya! Please introduce yourself and give us an insight into your professional and educational background.
Vidhya: Hi, I work as a senior manager at PayPal and lead ML and AI product management. I have about 18 years of experience in data and 8 years in leadership. With Bachelor’s in Mathematics and a Masters in Computer applications, I also did a 1 year PG program in AI with Great Lakes. Currently, I am doing my Doctoral research program on Personalization with Data.
At PayPal, over the past 5 years, I have had the opportunity to lead and build BigData, ML Engineering, AI product, and ML Science teams and initiatives. In my current role, I show the ML for Merchant products and Marketing, building Product recommendations and personalization solutions.
AV: That sounds spectacular and insightful. You started as a Software Engineer; how did you get into the field of Data Science?
Vidhya: I was always connected with Data in my career one way or the other, starting as a Database application engineer to being an architect to leading ML initiatives now; data has been the connecting thread. However, my career journey has been dynamic due to my personal circumstances. I was a homemaker for many years after graduation and had to strive harder than most to get an entry in tech owing to a lack of experience. But once I started, I found opportunities in each role that prepared me for the next role. I have also been blessed with great mentors and managers in all the companies I worked for, who gave me opportunities to indulge in my aspirations.
AV: I agree. A great mentor can help you climb the corporate ladder easily, which shows in your career trajectory. How do you foster a culture of innovation as a leader in Applied Data Science?
Vidhya: Unlike research teams, Applied ML teams operate under tight guidelines and strict timelines. Due to these constraints, they sometimes do not have the same luxury to explore new technologies, algorithms and implement new papers. However, I have remained conscious of the importance of innovation in ML teams, where every single day, a copious amount of new things going on.
Here are some of the strategies that have been instrumental in my management of an applied ML team without compromising on long-term innovation.
- Providing 10 to 15% of the time for working on stretch assignments or research/Proof of concepts.
- Encourage the culture of intra and inter-team collaboration with an emphasis on feedback loops – Innovation happens more in groups than in isolation.
- Allow safe space to fail. Machine learning is experimentation compared to software engineering and has many possibilities of failing. If failing safely is not allowed, the attempt to innovate is curtailed.
- Make innovation a part of the goals.
AV: Those are some pretty interesting strategies you follow. That’s great! Although, managing a team can be difficult. What do you consider a top challenge in leading a Machine learning and Artificial Intelligence team?
Vidhya: One of the challenges is striking a fine balance between timely business deliveries and innovation which is a time-intensive process. Apart from the fact that innovation is non-negotiable in any Tech Industry, also as a leader, we are responsible for our team’s careers by providing them opportunities for continuous learning and keeping them motivated, especially in AI and Data Science where the related tech is constantly moving at a faster pace.
AV: What do you consider is key to succeeding in the Applied AI team?
Vidhya: Most projects fail due to poor planning. AI Product managers play a key role in this phase. The key first step is understanding the business problem and where AI can solve it. As part of the planning, estimating opportunity sizing and agreeing upon clear, well-defined KPIs is of tantamount importance. A key next step is working backward from the expected outcome to arrive at and crystalize the appropriate metric or KPI. For example, the model could bring more customers to the website or drive new customer acquisition. Model metrics like Precision, Recall, or F1 are often misunderstood as KPI; the business would not worry about the model metrics but would be very much interested in the business metrics.
Developing capabilities and processes to bring an idea to fruition is another crucial aspect of success. Incorporating capabilities such as data catalog search functionalities, retraining automation, monitoring capabilities, continuous integration, etc., can significantly shorten the time required to test and learn from your ideas. This approach also guarantees that the valuable resources of our teams are not diverted towards monotonous and repetitive tasks but instead utilized to create engaging solutions.
AV: Going back to your mentors, what is one piece of advice that you got from your mentors, and how did you implement it?
Vidhya: There are two pieces of advice I got from 2 of my mentors, which is:
- Try At Least One New Thing in Every Model Development. The ‘newness’ can be anything like a new algorithm, a new type of data that is experimented or different feature engineering techniques.
- Fail Fast: Try to get the model deployment as soon as possible. Going for a perfect model, excellent results, and a new in-the-market algorithm could be enticing but oftentimes comes with an opportunity cost. Try a simpler, lean model to measure success or fail fast as soon as possible. We can always go for improvisation later. Shooting for a faster time to market and incrementally improving it to a cutting-edge model is critical.
AV: According to your profile, you have teaching experience as a mentor; how do you think that your experience as a mentor has influenced your career growth and success in machine learning?
Vidhya: Teaching is a great way to learn. I signed up to teach AI/ML on the weekend when I was doing my PG program in AI. When you teach a topic to someone else, you have to organize your thoughts, break down complex concepts into simpler ones, and explain them clearly and concisely.
I have also learned from mentoring e-commerce business leaders on Data driving marketing. As part of the Chennai Entrepreneurial chapter’s mentorship program. This gave me a perspective on strategic/ structural thinking, decision-making, planning and also made me come out of my comfort zone.
AV: I must say, your journey is truly inspiring. Working in a constantly evolving industry, how do you stay up-to-date with the latest developments and trends in machine learning? Can you give an example of a recent development that you find particularly exciting or promising?
Vidhya: I followed a disciplined learning approach for many years, spending either regular hours in the week or on the weekends learning from courses, blogs, and books. These days, though I continue my individual learning, I spend lesser time than before as I learn from my team directly when they try novel things; that is a perk of working with a team who are way smarter than you.
The latest development that I am very excited about, like several others, is what is happening with Generative AI, which seems to have all ingredients to disrupt everything from the creative industry to personalization. This is set to revolutionize the way businesses are done. As more concerns are raised regarding ethics and the possibility of obfuscated narratives, I am curious to see how Governments, organizations, and policymakers create processes to fortify the social fabric against potential threats.
AV: So, wrapping up, I would like to ask you, what is your advice for people who are planning to transition to Data Science?
Vidhya: Although there is no one-size-fits-all approach, I believe that the strategies and plans for transitioning to a data science or ML role are dependent on an individual’s career stage. Transitioning from a junior role is significantly simpler than a mid-management or a senior-level role. Those that are individual contributors can start their learning from basics, learning statistics, probability, Mathematics, and basic ML concepts. There are several free courses and YouTube resources. Following a bottom-up approach is important when learning, as it is easy to get a model done in a few lines of library code. This knowledge is not sustainable. There are several hackathons that one can compete to learn or just follow along the code to understand different feature engineering and model strategies. Once you have gained sufficient expertise on the concepts, plan to switch over to a team internally that has exposure to ML projects.
While it is crucial for individuals in senior and middle management positions to comprehend the capabilities of machine learning, it is even more critical to acquire expertise in the broader context of artificial intelligence strategy, engineering, and integration requirements. They can start to identify opportunities to innovate in their own area/project or domain and then communicate the success to their leadership and stakeholders. They can use their small initial successes to build ML capabilities and teams internally.
Some Resources for learning ML ground up – ML and Deep learning specializations by Andrew NG in coursera:
- https://www.coursera.org/collections/machine-learning
- https://www.coursera.org/collections/machine-learning?query=deep+learning
- https://www.w3schools.com/sql/
- https://sparkbyexamples.com/pyspark-tutorial/
- Medium and Analytics Vidhya blogs
- Kaggle Kernels
Conclusion
In this interview, Vidhya Chandrasekaran’s journey showcases the transformative impact of data science in the technology industry. Her insights, challenges, and strategies for fostering innovation offer valuable lessons for aspiring data scientists and leaders in applied data science.
She has shown that effective planning, understanding business problems, and defining clear KPIs are critical to succeeding in the applied AI team. Her mentors’ advice to try new things in every model development and fail fast has also contributed to her success. Vidhya’s expertise and leadership continue to shape the field of data science. It serves as an inspiration to future data scientists and leaders in applied data science.
If you wish to read more about our interview on the journey of a person in tech, click here.
