Building Machine Learning Products with Pegasystems PM

This week our #AskMeAnything session welcomed Sai Charan Tej Kommuri, Product Manager at Pegasystems. Sai Charan expanded on his experiences working in Machine Learning and AI Product Management. He also discusses his background and path to becoming the experienced Product Manager he is today.

sai charan tej banner

Meet Sai Charan Tej Kommuri

Sai Charan Tej Kommuri is a seasoned Product Manager, specialized in building Machine Learning products from scratch. Today, as a Product Manager at Pegasystems, he identifies business problems, leverages design-thinking principles to deeply understand the problem and later to apply his own technical expertise to implement the most suitable solution.

Sai has worked as a Product Owner for Bank of America for almost 6 years. During that time, he led product development by incorporating feedback from end-users. Prior to his current position at Pegasystems, a cloud-architected software for customer engagement and digital process automation, he worked as a Product Manager at CallidusCloud. An avid speaker, Sai enjoys giving talks about tech, AI and his path to becoming a Product Manager.

When did you want to follow the path of Product Management and why?

It all started during my MBA stint. I worked for 5.5 years at Bank of America before deciding to pursue a full-time MBA at the Indian School of Business. During my time at ISB, I spent most of my time participating in B-plan competitions. I was so committed to them that I was sure that I was going to start something on my own immediately post-MBA (MBA experience does this to people).

When I was unable to land any decent funding, I realized that a ‘Product Manager’ role is the next best thing to a ‘Founder’. That is when I decided on the Product Management path.

What’s a good suggested way to boost AI product adoption in a legacy company (eg financial services) that fears AI stealing their jobs?

I personally experienced a situation during my early career where my entire team was laid off due to the automation project completed by my team a year ago. Everybody was scared by Automation alone, forget about AI.  In the past 5 years, I realized that it is not the automation that led to the trimming down of the team.

It is the fact that the team has not moved onto doing higher-impact work that ultimately led to the churn. I believe if you can identify the right AI use cases with long-term value generation, supported by a path-breaking vision, you can get successful AI adoption in any industry.

Setting the stage for data entry is big for Machine Learning products. What examples do you have from your experience regarding inputting data upstream which leads to full-fledged products downstream?

This is so true! There was a product where our AI module was giving coaching recommendations about Sales Reps to their managers. When we first built the model, we were all excited by the fact that the AI module was coming up with quantitative recommendations that wowed people during demos.

But when we actually interviewed the end-users, we realized that most of the managers simply, blatantly I should say, ignored our model’s recommendations. When we dug deep, we realized that we were not really giving an option for the Manager to modify a coaching recommendation and most of the time Managers wanted to add some personalized advice.

So, we gave them an option to add content to a recommendation and save it as a template. After six months, we tracked all the templates that were successful by matching the improvement in a Sales Rep’s performance with the assignment of a coaching plan.

This way, we built another module that suggested complete ‘Coaching Plans’ that worked best across the organization for a customer. This is one such example I can think of where our focus on collecting user-inputted data led to a better and more robust feature.

How do you deal with people who are your seniors and are set on the path your product should take even though you know they are wrong?

In my experience, I have been into positions where it quickly became a ‘Me vs Them’ kind of battle. One thing I learned from these experiences is that it is so important to justify your findings using multiple dimensions. For example, if I was proposing an AI feature that helped create a ‘Briefing’ email for Sales Reps, I would not just present it saying that ‘Gartner has asked for it’.

I will present the idea from the target persona’s point of view and understand where the difference in understanding lies with the management. Then do more research, gather more facts from the field and present again, till the time all of us are on the same page. Gathering more data and empathizing with the management go a long way in building the right thing – this was my biggest learning.

What are some good learning resources for ML & AI?

If you are a programmer starting to learn Machine Learning, I would suggest:

  • 100-page Machine Learning Book by Andriy Burkov
  • Elements of Statistical Learning by Trevor Hastie.

If you are a statistician trying to learn AI programming, I would suggest learning python via:

  • Learn Python the Hard Way
  • Python for Data Analysis by Wes Mckinney.

Machine Learning/Data Science type of projects often have a very different cadence from other product teams/dev team (i.e., we are never sure if the model is going to work plus it takes a long time to fine tune the models). In your experience, what is the best way to create smooth collaboration between the two worlds?

When I first started recruiting Data Scientists in my team, there was confusion among the team members, particularly the Scrum masters, because they were not sure as to how the research work done by a Data Scientist fits into agile methodology.

After all, not many of the devs had even heard about Machine Learning until then. After trying and failing miserably with multiple approaches, we figured out that getting the Data Scientist to work in a silo was not working out. So, we started tagging the Data Scientist with the feature owner and started creating user stories/epics that have tasks to be handled by both the parties.  

We have also relaxed the timelines for these stories and were OK with the sprint metrics taking a hit. Over the course of time, both the devs, QAs and the Data Scientist got a better understanding of the team dynamics. We reaped the benefits of this association in later sprints where the team rattled off user stories pertaining to AI features at a much faster pace.

sai charan quotes

What factors do you consider in making the outputs of a Machine Learning algorithm simple and easy to understand for the end-user?

“Explainability” is the name of the game when it comes to ML world. In my experience, I have learned that customers are OK with 80% accurate models as long as they understand how the model is making its predictions. For any complex problem that the customer is trying to solve with AI, we always first try to build a stupid model that uses simple easy-to-explain Machine Learning algorithms like Logistic Regression, Naive Bayes, KNN.

I call this stupid because this is the first model we would be building for the end-user and tell him what is happening in the background. Once he kind of gets a sense of the working of these basic models, we crank it up a notch by introducing more advanced models like XGboost, RandomForest, etc. By this time, since the user already has a sense of how the basic ones work, he is more welcoming to adopting the more complex models.

How would you interact with your end users by having complex tech products that use Machine Learning? Do you find it hard to communicate and to align the roadmaps and features because of that? What are the main difficulties about that also with your tech team?

This is a huge challenge! It becomes really difficult when you are forcing an AI solution for the sake of it. This is where the ‘Design Thinking‘ approach has helped me immensely. We always invited representatives from the customers to be part of the ‘Design Thinking Workshop’ conducted with my teams at the start of the release.

In this workshop, we focus completely on the end-user problem and forget about AI. We then let brainstorming take over the room and list out possible solutions that solve the end-user’s problem. Only when we see an AI solution to be the highest-ranked solution, we move forward. Since, we are getting the buy-in from the customers early on in the development process and making the tech team also a part of the solution, making them adopt AI was considerably easy.

futuristic city photo

What are your suggestions for a new PM who moved from a long career in engineering?

As a PM, you would be in the best possible position to leverage your engineering knowledge. I would suggest focusing on getting the business angle to the discussions with your tech team – like the financial impact of the feature, market research, prioritization logic. This way, you will be able to complement your technical skills.

How do you explain the difference between Program Management and Product Management?

In my opinion, Product Management is responsible for coming up with a vision that is going to add value (dollars) to the company’s bottom-line whereas Program Management is responsible for realizing the vision by optimizing the existing resources (without burning a hole in the company’s resources).

When you have a very technical stake holder team, how do you think one should pitch the non technical aspects of the product?
Essentially, how to avoid Senior Managers from micromanaging your engineering team’s effort?

I would say try to understand the motivation/incentive for the technical stakeholder. Try to speak in terms that he is going to relate to and tie them together to the company’s product vision.

Rather than avoiding him from product discussions, make him part of the decision-making process by leveraging his technical knowhow. The more he is involved in the product decisions, the better he is aligned with your vision. This is one strategy that worked for me!

In your opinion, what is the ideal organizational design for a product team within a legacy organization? (Not just the roles, but where should they sit?)

The product team should have clear ownership of the features/product they are going to deliver. The team should be self-sufficient with developers, designers, testers, scrum masters, release management, technical documentation, engineering manager, architect, and the Product Manager. The Product Manager will represent the team while discussing with the management and will represent the management while discussing it with the team.

When starting a data-informed initiative internally working across teams, what did you focus on first – the top business questions from each team (and trying to find the data to answer those questions) or focus on data quality/ stewardship/ aggregation first?

It has always been the ‘Question’ that kicks things off. In my experience, what worked during cross-team projects is that you need a mission-critical use-case that aligns all the teams together. It should be a high-value use-case that shows quick wins for every stakeholder involved. Once you identify this use-case and get the buy-in from stakeholders, data quality and retrieval will be taken care of by the corresponding teams themselves.

Any advice for aspiring Product Managers?

One parting thought for people aspiring to transition into a Product Manager role – build a product and quantify its impact in your resume. They say “A Product Manager is only as good as the products he has built”.

Did you miss this AMA? Make sure to check out our events page to attend the next one!

#Productcon banner

Enjoyed the article? You may like this too: