Can Imperfect Models Still Add Value?

Mike Kreiger (CPO of Anthropic) & Kevin Weil (CPO of OpenAI)

Credit and Thanks: 
Based on insights from 20VC with Lenny's Podcast.

Key Learnings

  • Embrace the unpredictability of AI advancements to stay ahead in product development.

  • Understand the unique dynamics of enterprise/consumer environments to tailor your approach effectively.

  • Prioritize user feedback in the iteration cycle to refine products and enhance user satisfaction.

  • Recognize that even imperfect AI models can deliver significant value if designed thoughtfully.

  • Foster a culture of continuous learning and experimentation within your team to drive innovation.

Today’s Podcast Host: Sarah Gou (Conviction)

Title

A conversation with OpenAI's CPO Kevin Weil, Anthropic's CPO Mike Krieger, and Sarah Guo

Guests

Mike Kreiger and Kevin Weil

Guest Credentials

Mike Krieger is the Chief Product Officer at Anthropic, joining the AI company in May 2024 after co-founding and serving as CTO of Instagram. His career highlights include growing Instagram's engineering team to over 450 people and scaling the platform to more than a billion users before departing in 2018. Krieger's estimated net worth is around $350 million as of 2024, primarily accumulated through Instagram's success and subsequent ventures in the tech industry.

Kevin Weil is the Chief Product Officer of OpenAI, where he leads teams focused on applying AI research to consumer products and services. His impressive career includes roles as President of Product and Business at Planet Labs, VP of Product at Instagram, and SVP of Product at Twitter, where he played a key role in the company's growth and IPO. Weil's estimated net worth is at least $22 million as of February 2025, based on his stock holdings in Twitter and Planet Labs.

Podcast Duration

40:58

Read Time

Approx. 5 mins

Deep Dive

When Mike Kreiger and Kevin Weil transitioned into their new roles, the reactions from their peers and friends varied widely, reflecting a mix of excitement and skepticism. Mike noted that those who knew him well were supportive, recognizing that he would thrive in a challenging environment. In contrast, others questioned his decision to return to work after a brief semi-retirement, expressing disbelief that he would choose to take on such a demanding role. This spectrum of reactions serves as a reminder for founders to cultivate a network of supporters who understand their vision and can provide encouragement during pivotal career moves. Founders should be prepared for mixed feedback and use it as an opportunity to clarify their motivations and goals.

Mike's biggest surprise in his new role has been the pace and complexity of enterprise operations. He found himself grappling with the slower timelines typical of enterprise environments, where decisions can take months to materialize. This contrasts sharply with the rapid iteration cycles often found in consumer tech. For founders, this highlights the importance of patience and adaptability when navigating different market segments. Understanding the unique dynamics of enterprise sales and product development can help founders tailor their strategies accordingly, ensuring they are prepared for the longer sales cycles and the need for extensive stakeholder engagement.

The iteration cycle discussed by Mike and Kevin emphasizes the necessity of continuous feedback and adaptation in product development. They shared that successful iterations often arise from real-world usage rather than theoretical models. This insight is particularly valuable for startup founders, who should prioritize user feedback to inform their product iterations. By actively engaging with users and incorporating their insights, founders can create products that better meet market needs and enhance user satisfaction.

Mike and Kevin also touched on the value of tasks, noting that even a model performing at 60% effectiveness can still provide significant value if designed thoughtfully. This perspective encourages founders to focus on creating products that deliver meaningful outcomes, even if they are not perfect. Founders should embrace the idea that a product can evolve over time, and they should not be deterred by initial shortcomings. Instead, they should view these challenges as opportunities for growth and improvement.

Developing intuition around AI and its applications is another key theme from their discussion. Mike and Kevin stressed the importance of product managers deepening their understanding of AI technologies to enhance decision-making. Founders can take this to heart by encouraging their teams to engage with AI tools actively, fostering a culture of experimentation and learning. This proactive approach can lead to innovative solutions and a more agile response to market changes.

Educating end users, particularly in enterprise contexts, emerged as a crucial point. The duo emphasized the need for tailored educational materials to help users adapt to new technologies. Founders should prioritize user education as a core component of their product strategy, ensuring that customers understand how to leverage their offerings effectively. This can lead to higher user satisfaction and retention rates.

In discussing how to use AI, Mike and Kevin shared their experiences with OpenAI/Anthropic, highlighting the potential for AI to automate repetitive tasks and enhance productivity. They recounted a humorous anecdote about a beta test where AI successfully ordered pizza for their team, showcasing the technology's capabilities in a lighthearted manner. Founders should explore similar use cases for AI within their organizations, identifying areas where automation can alleviate burdens and allow teams to focus on more strategic initiatives.

Looking to the future, both experts expressed excitement about the potential for AI to become more proactive and asynchronous in its interactions. They envision a world where AI can anticipate user needs and provide timely assistance, fundamentally changing how people interact with technology. Founders should prepare for this shift by considering how their products can leverage AI to enhance user experiences and streamline workflows, ultimately positioning themselves at the forefront of technological innovation.

By synthesizing these insights, founders can better navigate the complexities of product development in an AI-driven landscape, fostering a culture of adaptability, user engagement, and continuous learning within their organizations.

Actionable Insights

  • Build a supportive network that encourages open dialogue about career transitions and new ventures.

  • Actively engage with users to gather feedback and inform product iterations.

  • Design products that can adapt and evolve based on real-world usage and user insights.

  • Invest in training your team on AI technologies to enhance their decision-making capabilities.

  • Create educational resources to help users understand and effectively utilize your AI-driven products.

Mind Map

Key Quote

"Your output for experiment should be learning, not necessarily perfect products you're going to ship every time."

As AI technologies continue to advance, we can expect a shift towards more proactive and personalized user experiences. The integration of AI into everyday tasks will likely lead to increased efficiency and productivity, particularly in enterprise environments. Founders should anticipate these changes and consider how their products can adapt to meet evolving user expectations, ultimately fostering a more intuitive and responsive interaction with technology.

Check out the podcast here:

Latest in AI

1. Mira Murati, former CTO of OpenAI, has launched her own AI startup called Thinking Machines Lab, following her departure from OpenAI in September 2024. This move comes amid significant leadership changes at OpenAI, including the exits of other key executives, and coincides with CEO Sam Altman's efforts to restructure the company into a for-profit entity. Meanwhile, OpenAI's board unanimously rejected Elon Musk's $97.4 billion takeover bid, with chairman Bret Taylor calling it an "attempt to disrupt his competition.".

2. Perplexity AI has open-sourced R1 1776, a modified version of DeepSeek's R1 language model designed to eliminate Chinese Communist Party-aligned censorship while maintaining strong reasoning capabilities. The model, trained on 40,000 prompts across 300 censored topics, aims to provide unbiased and factual information on sensitive issues. Available on Hugging Face, R1 1776 represents Perplexity AI's commitment to enhancing freedom of expression in AI technology, though its name has sparked debates over potential nationalist undertones.

3. OpenAI has unveiled SWE-Lancer, a new benchmark comprising over 1,400 real-world freelance software engineering tasks sourced from Upwork and the Expensify repository, with a total value of $1 million in actual payouts. The benchmark evaluates AI models on both individual contributor tasks, ranging from minor bug fixes to major feature implementations, and managerial decision-making, using end-to-end tests verified by professional software engineers. In the initial evaluations, Claude 3.5 Sonnet outperformed other models, achieving a 26.2% pass rate on individual contributor tasks, while the best model reached a 44.9% pass rate on managerial tasks, highlighting the significant room for improvement in AI's ability to handle complex software engineering challenges.

Startup World

1. Hightouch, an AI-powered data activation platform for sales and marketing teams, has secured $80 million in a Series C funding round led by Sapphire Ventures, reaching a valuation of $1.2 billion and achieving unicorn status. The San Francisco-based startup, founded in 2018, plans to use the funds to accelerate the adoption of its AI Decisioning product, which uses machine learning to optimize marketing campaigns and improve customer experiences. Hightouch's platform, which integrates with over 250 tools and operates on top of a company's existing data infrastructure, has attracted notable clients such as Spotify, PetSmart, and Tripadvisor, while more than doubling its revenue last year.

2. Luminance, a legal AI startup founded in Cambridge, UK in 2015, has raised $75 million in a funding round led by billionaire Steve Cohen's Point72 Private Investments. The company, which uses AI to draft and review legal documents, has developed a system called the "Panel of Experts" that allows businesses to integrate AI across contract generation, negotiation, and analysis. Luminance's clients include major organizations like AMD and Hitachi, as well as prominent law firms such as White & Case and Clifford Chance, with the new funding bringing its total raised over the past year to $115 million.

3. Humane, which became notorious for its ill-fated $500 AI Pin, has been acquired by HP for $116M — while the Pin itself will be discontinued.

Analogy

A founder’s career journey is like switching from sprinting on a track to navigating a marathon through rugged terrain. Mike Kreiger’s transition from fast-moving consumer tech to the slower cycles of enterprise work was a shift in pace, much like a sprinter adjusting to endurance running. Skepticism from others is inevitable, but founders must focus on those who understand their vision. Just as AI evolves through iteration, so must leaders—embracing challenges, adapting strategies, and continuously learning. In an AI-driven world, the winners will be those who anticipate the terrain ahead and adjust their stride accordingly.

What did you think of today's email?

Your feedback helps me create better emails for you!

Loved it

It was ok

Terrible

Thanks for reading, have a lovely day!

Jiten-One Cerebral

All summaries are based on publicly available content from podcasts. One Cerebral provides complementary insights and encourages readers to support the original creators by engaging directly with their work; by listening, liking, commenting or subscribing.

Reply

or to participate.