At Microsoft’s 50th anniversary celebration, Mustafa Suleyman, the company’s CEO of AI, shared his unique approach to evaluating the performance of Microsoft’s AI assistant, Copilot. While the company tracks common adoption metrics such as daily and weekly active users, distribution, and usage intensity, Suleyman emphasized a more specific focus: SSR, or the rate of successful sessions.
In an era when consumer feedback was less immediate, metrics like time spent with a product or the problems it helped solve were used as proxies for quality. Now, Suleyman explained, Microsoft uses anonymized logs to extract real-time feedback and sentiment from users.
Suleyman, who joined Microsoft after leading Inflection AI, said that Microsoft has implemented an AI model to assess this sentiment and calculate Copilot’s SSR. The model has shown significant improvement over the last four months, and the company continues to optimize for this metric.
While he did not reveal the absolute SSR or other specific metrics for Copilot, Suleyman shared new features the company is rolling out. These include a more personable voice for Copilot, the ability to analyze web pages in real time, personalized podcasts, a research tool for complex queries, and eventually customizable Copilot appearances for individual users and conversations.
Suleyman also hinted at future plans, expressing interest in a “cutesy” Copilot design, comparing it to a “little Furby-type thing.”
