Product Development

It's essential to regularly evaluate and address tech debt as it can significantly impact a mature product's overall health and sustainability. If left unaddressed, it can build up and require a massive focus from the team that can disrupt existing customers or prevent you from adding new customers to the product until addressed (this is something I've run into before, and it's no fun!). Here's how I approach it: 1. Regular technical reviews: I schedule regular technical reviews with my development team to identify any technical debt that has accumulated and determine the best way to address it. We typically have a staff-level engineer or dedicated architect who can provide regular guidance and reviews as we build to ensure we consider the larger technical vision and the scale we need to deliver. 2. Budget for tech debt: I allocate a budget specifically for addressing tech debt and ensure that it is included in our development plan. I use three buckets at the team level to align what percentage of our overall capacity to devote to each: Innovation (new features, new products), Iteration (incremental enhancements), and Operation (tech debt, bug fixes, etc.). 3. Incorporate tech debt into prioritization: Once you have the target capacity allocation across the big three buckets, you can incorporate tech debt into the product backlog and prioritize it alongside new features and enhancements. This helps ensure that you take a proactive approach to reduce tech debt and improve the product's codebase. In conclusion, considering tech debt is an ongoing process when working on a mature product. By regularly evaluating and addressing it, you can ensure that your product remains healthy, scalable, and sustainable over time.

I've found it helpful to craft a ~3-year product strategy that articulates the market and trends, the challenges/opportunities, and the product's path forward with coherent actions to address those challenges. This frames the product's direction and how it will impact the company and customers you serve.  From there, determine the % allocation of your teams' capacity across investment buckets. Agree with your team on how you'll spend your time across these to achieve your product strategy: 1. Innovation: bold changes to make leaps and bounds towards the customer journey vision. E.g., new features, the overhaul of existing features, and integrations with partners. 2. Iteration: incremental changes to the existing product to deliver additional customer & business value. E.g., conversion funnel optimizations, A/B testing, and minor fixes that provide an incremental lift of a KPI. 3. Operation: The cost of managing a modern SaaS product. E.g., Security/data privacy, performance/uptime, tech debt/upgrades, and bug fixes. Next, you can consider the opportunities/ideas that may fall into each of these buckets to create a stack rank for each based on the Impact it can have. Again, a simple scoring framework can help, like RICE (detailed below). While prioritizing within each bucket, consider the following: 1. Align with the product vision and goals: Ensure that the features being considered align with the product's long-term vision and goals and that they support growth initiatives. This is often the first filter when prioritizing items. You can take additional steps to determine Impact and Effort if an opportunity is aligned. 2. Evaluate customer needs and feedback: Regularly gather customer feedback and analyze data to understand their needs and priorities. Is this opportunity impactful to these customers by solving a significant problem? How many customers might this opportunity affect? 3. Consider the cost-benefit of each feature: Evaluate the resources and time required to develop and implement each feature and weigh that against the potential benefits. A helpful framework for this is RICE, where R is the Reach (# of customers), I is Impact (degree of Impact on these customers), C is confidence (how much evidence have we gathered to support our assumptions), E is the Effort (how much effort and time might it take to develop this solution?). You take R*I*C divided by E to provide a numerical score. 4. Balance short-term and long-term benefits: Consider the Impact of each feature on both immediate and future product success. 5. Collaborate with cross-functional teams: Work closely with design, engineering, and marketing teams to ensure that all relevant perspectives are taken into account when making decisions.

Business objectives help the broader team and business understand if they've achieved the intended impact but are difficult for a product team to use sprint to sprint for a few reasons: * Revenue isn't super actionable for the team: if it goes down or up, it's not immediately obvious what actions the team should take in response. * Business outcomes are often lagging metrics: length of sales cycles, churn, and expansion all impact revenue, and this takes time. It doesn't enable the team to take new actions in response to these sprint by sprint. * A complete focus on financial metrics might cause unintended behavior: We've all heard about the Wells Fargo example where they had a KR of selling eight products to every customer, which led to teams signing up customers for products without their knowledge. With business outcomes or financial metrics as the only objectives given to product teams, they could be pulled into short-term thinking to hit revenue growth instead of focusing on the customer behaviors they need to change. With so much revenue not controlled by the product team itself (sales cycle, pricing, discounting, renewals, expansions), teams should be looking at sub-metrics that are more leading and actionable.  To translate business objectives to product objectives, I'm a big fan of Josh Seiden's book, Outcomes over Output as a helpful framing. In this book, he defines an outcome as a change in customer behavior that drives business results.  Using this framing, teams can ask these questions to find product objectives and associated metrics: * What are the user and customer behaviors that drive business results? * How can we get people to do more of those behaviors? * How do we know we're right?

The best PMs define success before building a feature. To define success, teams should think through: 1. The customer problem they're solving, what does it look like when solved (for the customer and the business)?  2. How impactful is it if solved (time-saving, cost-saving, revenue-generating)? 3. How does solving this problem help the company achieve its longer-term vision/strategy? Is there a KPI or measure for the feature that helps the team know they're moving toward that strategy? Once the feature is launched, there are a few different ways to determine whether a new feature or update has been successful: * Track usage metrics: such as the number of users using the feature, the frequency with which they are using it, and any changes in retention or engagement. * Gather customer feedback: This can be done through surveys, user interviews, or tracking social media mentions of the product. * Monitor for changes in key performance indicators (KPIs): Depending on the goals of the feature or update, you can also monitor for changes in relevant KPIs, such as revenue, conversion rate, or customer satisfaction. * A/B test: If you're unsure how a new feature or update will be received, you can run an A/B test to compare the performance of the new feature or update against a control group. Lastly, a helpful question that Sean Ellis has popularized is the PMF test: ask users, “how would you feel if you could no longer use the product?” and measure the percentage who answer “very disappointed.” After benchmarking nearly a hundred startups with his customer development survey, Ellis found that the magic number was 40%. This can be a helpful tool to identify users to interview further and segment your customer base around those customers that find a lot of value in your product that you should be targeting.

This is hard! For me, it's a mix of having a good understanding and confidence that you have  (1) a clear hypothesis that you can test with a minimally viable product that is shaped by data and customer/market research, (2) confidence that you have a potential solution that can prove the hypothesis correct, and (3) an understanding of the risk and opportunity for building that solution, including the time it'll take to build, the availability of users willing to try your solution. When in doubt, it's always a helpful rule of thumb (in my opinion) to simplify the scope of your solution as much as possible, to both reduce engineering time/complexity AND to not squander the opportunity cost of shipping too late (see the Reid Hoffman quote question posted as well!). The hardest one in my opinion is 1 - making sure that you get down to the real essence of an unmet customer need and being incredibly clear and specific on how you think your product can solve it, and how you will know.

Before investing in engineering resources you want to build conviction around the following: 1. Is there a market need? Are you fulfilling a true gap in the market? 2. Do you have a differentiated vision to deliver on this need? 3. Is there willingness to pay?  4. Does the business model make sense such that you see a path to ROI for the business? 5. Is there a clear route to market (you know how to sell / acquire customers)? 6. Does your business have (or plan to have) the capabilities to deliver on this product (ex operational, technical or other expertise) such that it is strategic to expand in this direction? Overall you want to be able to articulate what the investment unlocks for the company and how.

For a new product in a new market, I don’t think you can ever validate the problem space enough. Once you have reasonable confidence that the users (and buyers) exist for the problem space your product tackles, and there are viable ways to create a business out of that solution, the best next step is to involve engineering to build a prototype of the solution. Then you can test that prototype with a representative sample of the expected user base to get early feedback and iterate on building out the product and grow the business. For 0-1 products, involving engineering early as a partner to the overall product definition and development process is best, so you can iterate together on building the right product for the right audience.

"0-1 product development" is the idea of building something from nothing. That is, you have an abstract customer or business problem you need to solve and no solution for it (0) and, as a PM, you need to figure out the first attempt at a solution (1) to address the problem. An example from my own career is Notebooks, a product I helped ship at Abstract - we had a meaningful number of customers abandoning our initial offering due to changes in the product design tooling landscape, and we needed to figure out what problems they still had that we could build a solution to address. After spending a good amount of time interviewing and researching the product landscape, we identified an unmet need (in that case, a way to centralize design decisions made in the product design process outside of the tool used for design, to easily communicate and ratify decisions among stakeholders). We built an MVP, validated it with customers in a controlled setting, and then iterated on that product until we felt it was ready for a public launch. My first step in developing a 0-1 product actually isn't developing at all - it's understanding or framing the customer problem. Often the problem is given to a PM in some other way that isn't actually the problem at its core. In the case of Abstract Notebooks, we actually had a business problem (customers were leaving our service because they no longer had an unmet need) but we didn't have a good sense of what other unsolved problems they had. In that case - and I'm a proponent of this - we went out and talked to users to help get an understanding of what problems they still had in their product design processes. We talked to a number of different kinds of users - former Abstract customers, curent ones, those who had never heard of us before - and started to find common challenges amongst them. That allowed us to shift away from thinking about the internal business problem to something very focused on the user. It certainly took some time to wrap our heads around a clear, focused problem we could solve, and there are a number of tactics one can use to do this (design sprints are one I like) - but that first step of beginning to frame the user problem is what I always find a useful starting point.

Part of the fun of building something 0-1 is that you have a green field in front of you — you can build anything! (Or that's what we wish as PMS....) As much fun as it would be to the world is your product oyster, constraints help provide focus and a direction. I see the first step is making sure that you and stakeholders are aligned on what a goal or success looks like. And usually, that comes in the form of a metric that you're trying to move.  If you work for an ecommerce company, you'll make very different decisions around what to build if your metric is increasing ASP versus unlocking a new vertical or segment. If your metric was the former, you might start doing user research on customer willingness to pay, or what they're thinking about when they're in the check-out flow. And if it's the latter, you'd probably start with user research into people who aren't yet your customers.  Without that first constraint — what metric do we want to move, what does success look like — there's no way to focus your research and problem defintion phase. Once you have that north star to work toward, you can enter user research with your specific goal in mind. 

Start by forming assumptions and making hypotheses around the desirability, feasibility, and viability of your new product. Then validate those assumptions, learn and iterate. Desirability assumptions could be * Is the product addressing a problem that the customer is facing? Always look at how big the problem is in customers' lives and how much time, money, and headache it saves. What is at stake if the problem goes unsolved? * How frequent is the problem? Building a product for a one-time problem is risky. It's much better to evaluate the possible stickiness of the product and retention, listing what can get the product there. * Do customers actually need this problem to be solved? Will this solution be treated as a "vitamin" or a "painkiller," a "nice-to-have" or "must-have?" Some products address problems that are more pronounced than others, which naturally increases their desirability. Feasibility assumptions could be * Technology. What kind of technology and professionals are required? Should the business hire a full-stack engineering team, or is it possible to start with a minimum viable product built with no code? Physical resources, licenses, or cloud services — are those critical for the product's success? * Budget and timing. How much funding and time will it take to bring the product to the market? * Partnerships and resources. What are the pieces that businesses need to align to make the product a success? Those are partnerships to leverage, GTM resources, and support from the compliance or legal sides. Viability assumptions * What would be the core assumptions to have a profitable business model? Unscalable business models with unreasonable monetization will quickly hit a dead-end for the new product. * Market opportunity. How big is the market for the planned product? What is the value of the problem that the business is planning to solve and how much will potential customers be willing to pay?  * Revenue streams and channels. What are all the marketing channels to reach customers and what will be the monetization model?  * Cost structure and unit economics. Is the pricing attractive compared to direct and indirect competitors? Is the cost vs profit ratio indicating a profitable business? What could be acceptable CAC? Will the product have a free trial or freemium? Based on my article at Entrepreneur.

0 -1 product development refers to building a new product or service line from scratch (0) to bringing its first iteration into the hands of customers and users (1) The first step to develop a 0-1 product is to deeply understand the market need. I look at this from the buyer perspective, the end user perspective, and the competitive landscape perspective. Unless you understand, what's needed, what exists, what's missing, and what will differentiate your solution and validate your need to exist, you cannot begin the next phase, which is product definition. 

I will share first two steps that I follow. Step 1: Is this problem worth solving? 1.1 Problem definition and user segmentation * B2B product: A business customer must have a genuine painpoint that they are willing to pay for. Some problems are not big enough problems, hence not high priorirty for the business users, these are not worth pursuing. Fine tune the problems till they hit   * B>B>C: The business user/stakeholder may have rightly identified a problem but may not have the best ideas in terms of what solutions can work. Validate the problem is real with user research. * B2C: Similar to B>B>C, some segment of users have a need. Identify who they are and what the real problems are. 1.2 Is the Problem TAM big enough Step 2: Why us? Why now? 2.1 Do competitive studies  2.2 SWOT analysis  2.3 Are you best positioned to build this and build now?

At a high level, a 0 --1 product offers a completely new solution or functionality to a user problem, often creating a new market category or sub-category within an existing large market.   The top thing in developing a 0--1 product is validating that there is a real user need in the market that will be served/solved by this new product, and, that the need is big enough to build a viable business around it. * Identifying/defining the top user persona for the product – their motivations, goals, tooling needs, and pain points with existing solutions, if any * Defining the buyer persona (if different from the user) – motivations, incentives, goals for themselves and their teams, and pain points with existing solutions, if any * Understanding the market – what existing market segment or category would the new product fit in, the competitors, incumbents, market sizing and basic TAM (total addressable market) analysis to validate there is a big enough market (or potential market) to build a business * If creating a new market category with the product, it is still important to understand what is the closest market segment to the new category you want to create. It helps gauge the TAM for business development purposes as well as the closest competitors to the new product.  

So, in my experience of building 0-to-1, I've never had to do this before exploring a potential new product 😅 and candidly, I really don't like doing it because any projections are in my experience educated guesses based on inherently flawed source data - historical data that may not apply anymore, all sorts of biases, differences between other products and your new product, etc. What I try to do instead of offering revenue projections is work with my leadership/stakeholders/et al to understand that the primary initial goal of shipping the MVP is to learn and validate. We can use other metrics to understand whether the product is viable (eg. repeat usage, engagement, etc. depending on the product's value proposition) and, in parallel, start to gauge what potential users are willing to pay for said product. Those two things together will help inform possible revenue projections.

Start with a Total addressable market (TAM) * share of TAM you can get in next 3-5 years *confidence level *Revenue per user per year * 3-5 years. In the RICE framework, you would divide TAM * Share of TAM (influence) * Confience/ Effort to then help prioritize. (Desc) Calculating Share of TAM you can get in next 3-5 years is a art more than science. Do a SWOT analysis for yourself, incumbents in the market, if any and emerging players. Also keep in mind that market/users may not always be ready to adopt. So factor in the barrier to adoption/switching costs. Confidence level is based on your ablity to execute and deliver results- ship great products, great customer support, sales channels, marketing (even if lo-budget). Do you have the right team to get this done? Is the technology there yet? Are there high risk dependencies?

When setting up any new line of business, revenue projection is an important step to understand the expected outcomes and viability of the business. For new a new product, you should have: * The basic market analysis including market sizing and TAM (total addressable market) * Competitor growth rates and revenue acquisition at comparable stages of growth * The market segment you are positioning the product in, and its current and projected growth rate * Revenue and monetization strategy for your new product, including pricing, expected paid user acquisition and retention rates (based on initial product/market testing) You can use these inputs to create an initial revenue projection model. You can do a best-guess estimate for rates where there is little data available. You can always tweak these models as data starts coming in post-launch. I recommend creating three projections – expected or target growth, above target growth, and below target. This will help you get a basic floor and ceiling or bounding box of your revenue in the first year.

It's never going to be "as ready" as you want it to be. Because we don't work in a vacuum, the product is never built to 100% (I've never worked on anything where we had P2 requirements built into the product on launch day - it doesn't happen). There are three components that come together to ID when a product is ready to be shipped:  1) Business needs: If you run an ecommerce product, then your product needs to be out for Black Friday and the holiday rush. Similarly if you work in fitness, January 1 is going to be a big day. Aligning on these key dates and deadlines with stakeholders up front is important. At Quizlet, we focus on the Back to School season, which we define as August 1 through mid September. As a result, we'll sometimes know that something needs to be live on a given date — and then we'll work around that business need to scope the project and/or add more resources as needed.  2) User Jobs to be Done: I wrote in one of my other responses about being obsessed with problems. Once you're sure what the problem is, you can reframe it as a job to be done to focus. For Quizlet and the teacher products I worked on, this was "When I'm teaching my students, I want to keep them engaged and check for understanding, so that I can differentiate my teaching as needed." When you can take your MVP back to users and test it out, and verify that it's serving the job you've defined, you're ready to ship.  3) Eng readiness: Especially with launching a new product, ensuring that your engineering partners are ready is key. This includes asking questions like "What happens if we're wildly successfully and this product is used by XX people overnight" and "Can we test our logging to make sure these key actions are being logged" etc. Moving from MVP/beta to a GA launch means knowng that your product can scale up as it gains traction and that you'll be able to measure and evaluate its success!

It depends on the lifecycle of the product and goals. 1.  0>1 product: Goal - to find product-market fit. Here its very important to think through your hypotheses, possible outcomes (Prove, Disprove, insufficient signals) and what you would do next - next set of features, V2 of the MVP and so on. Charting this out helps you then answer what you must absolutely answer to get to the next step. Once your product enables you to achieve testing of your hypothesis, ship it! This assumes that product/solution is usable and stable (not overtly buggy etc). Also in general, start with a smaller userbase, tweak and polish before going full bright. * Consumer 0>1: IMO, consumers have high expectations and they WILL compare your product with way more mature products. So its more important to pay attention to usability and product excellence (latency, bugginess, low connectivity coverage etc.) for consumer products. Its easier though to experiment relative to consumer products.   * Enterprise 0>1: Here offline/hacky support solutions can work initially to help you support operationally without building all the bells and whistles. So focus even more heavily on what problems are worth solving and are the solution ideas resonating. Remember though that for B>B>C, the enterprise might think the solution works, but their users may not think the same. So user research is still no substitute to actual launch data.  2. 1> 100 product. Goal- growth in engagement userbase, retention, monetization etc. I think its good enough to ship when the solution fixes a known issue or gap even incrementally. Iterate depending on the effort vs impact equation. This fix could help retain some existing users, so it still matters. If its a novel feature, then follow the 0>1 principles in #1.#1.

For early stage products, a feature or solution is usually ready to ship when it meets the functionality for the main user journey(s) i.e. "golden paths" defined in your PRD or user journey maps, and passes the predefined usability bar from a QA perspective – user can complete the common tasks within expected reliability and performance metrics, all expectation cases may not have been hardened yet.  In basic terms, a user should be able to use the solution to complete the predefined task in a reasonable manner, and be able to provide feedback that can be used to enhance the product. For example, if the product keeps crashing in the middle of the user task journey, then that leads to a bad user experience such that the user can’t really provide effective functionality feedback for you to evolve the product, and they may never adopt the product at all. 

One way I like to prioritize problems is based on the level of risk these will pose to the final solution. Which are the riskiest assumptions or riskiest bets that will affect the success of your product? (Risk can be defined crudely in terms of Low, Medium, High or in some cases you might have a model with some sensitivity analysis built in). Regardless, if you can quantify the risk (and thus impact) of the problem to the final solution, you have a clear blueprint of where to begin. A related method is to consider one-way vs two-way decisions. One way decisions are challenging or impossible to reverse - these have multiple downstream effects on the solution. Two way decisions can be reversed easily or adjusted over time once you have more data. I prefer to focus my time and energy on one way decisions first, as these will build the pillars of the product. If there is considerable time or effort spent by your team on a two way decision, you can make the argument to come back to this once you have more information or once all the one way decisions have been made.

Usually, there is a pre-determined prioritization framework that your product org should use to prioritize which problems to work on. There are multiple inputs with different weightage that are used in the prioritization framework, depending on the phase of the product, business and company. The most common inputs are: * business and strategic product priorities * degree of user friction (eg: adoption blocker vs nice to have) * market and sales priorities (eg: is delivery of a feature important to win against a competitor in this cycle) * internal priorities (eg: technical debt removal, infrastructure upgrades etc.)

Not at all - it just changes how I think about product delivery and the tools my teams and I use. In the office, it was common to rally a bunch of teammates together in a "war room"-like setting, heads behind laptops and quickly bouncing status updates or ideas or urgent issues around to move a product forward. Now, we do that over Slack and/or Zoom.  The main opportunity remote work has brought is a critical need for documentation - that is, an easily interpretable and navigable paper trail for decisions made along the product development lifecycle. In the office, decisions could be made in a meeting or a side conversation; if you forgot about a decision, you could always go ask someone in person. In remote work, you could do that (again, via Slack), but the expectation of a response has fundamentally changed. Slack is best used as an asynchronous communication channel, meaning that colleagues can respond when they choose. If you need information quickly, what better way to get what you need from the doc, Confluence page, or Slack thread where that decision was made? Straight from the source. Organizing all that information when building products is hard, and I have yet to find a single tool that is *amazing* at it (Abstract Notebooks was a product I worked on that did a pretty good job at addressing this!), but I do consider it a critical role as a PM to ensure that your partner team has all the context they need to build great things, and in a remote context that involves ensuring easy access to context and decisions that matter.

After 2.5 years into working remotely, the two areas that have been the most challenging for me are 1) cross functional exploration and ideation and 2) user research outside of video interviews.  First, I've found it really challenging to replicate is the ability to get in a room with design, engineering and PMM partners and think through problems and solutions to those problems. Whiteboarding and brainstorming are frequently thought of as startup cliches, but there's something really powerful about getting smart folks in a room to deeply grapple with a challenging problem.  Tools like Figma and Miro can help (I've been so impressed with Figjam and how important it's become to my workflow!) but it's still so challenging to get folks to commit, distraction-free, to online collaborative project time. Slack, emails, pets, kids — it's all a distraction. This challenge most frequently arises during the ideation phase of "how might we address this [agreed upon] job to be done?" Secondly, and much more tactically for my role at Quizlet in particular, is limitations on observing and interviewing users in the field. My customer is one who works in a particularly challenging environment in COVID world — teachers! Prior to COVID, we used to go into the classroom and see how teachers used Quizlet with their students. I'd make a point to do this activity outside of the Bay Area and visited classrooms in other parts of the U.S., and even internationally. This was an incredibly useful user research activity and I'd regularly learn new things about the problems teachers faced, and where Quizlet could be useful. Due to COVID, we haven't been able to spend time in the classroom and so user research has become much more clinical, with video interviews one-on-one (i.e. no visibility into the student side).  While Quizlet is certainly used directly by students, my focus area has been teachers. For a product where the "buyer" (teachers) may not be the end user (students), solving for only the buyer needs without understanding more about how the buyer and end user interact has been challenging.  Abstracting that out more generally, I'd imagine that for many product managers, not being able to interview and observe users in the field is a big impediment. 

Working remotely has impacted each individual and their team differently. For some it has boosted productivity and flexibility. For others, it has impacted morale and connection to the mission. I encourage each and every one of us to reflect deeply on what kind of work set up brings out our best selves: is it remote, hybrid, in-person? How do you want to collaborate and what gives you energy vs. what drains your energy: team offsites, async vs sync communication, time zone alignment?  There are now ample options for each of us to find the work set up to support how we work best. Ultimately our ability to deliver products will depend on our ability to thrive in the environment.

In my experience, real human interactions help in building relationships and trust. There may be other effective ways of achieving this too. Once you establish those, working remotely does not adversely impact delivering best products. In fact, I have seen hyper productivity working remotely in some instances.

In my experience working even pre-pandemic with hybrid, remote and globally distributed teams, the most important things for product development teams are: * clear and open communication channels * written objectives, priorities, and any decisions for all stakeholders so all parties are clear on overall product direction, priorities, tasks and metrics to optimize for. If everyone involved is on the same page, product delivery can be smooth regardless of remote work.