Cortana with Sandeep Paruchuri (Big Bets 8)

This is a newsletter about big bets, explored through conversations with the product leaders who worked on them. For this one, I chatted to Sandeep Paruchuri, my colleague from WhatsApp and a former PM at Microsoft, about Microsoft’s AI assistant, Cortana. A competitor to Siri and Google Now, Cortana was a bottom-up bet that became a major part of Microsoft’s strategy to make their Windows Phone succeed and to prove their prowess in the AI arena.

I always start by asking people where the idea for their big bet came from, but the question seems redundant in the case of Cortana, given that they named the product after their inspiration. Cortana is a fictional AI character within the Halo video games, who serves as an advisor and assistant to the player character, Master Chief. However, Sandeep tells me there was a bit more to the idea than that.

Inventing Cortana

He takes me back to 2011, when Microsoft committed to a full system rewrite of Windows Phone, and therefore did not expect to add many features into their next version of the OS. This created an opportunity of about 18 months in which engineering was busy, and product management had space to come up with new ideas. Sandeep was working on the browser, search and maps team. Despite recent improvements to search, which meant that Bing results had super high relevance, users still weren’t choosing the inbuilt search on Windows Phones, but opening up the browser to go to Google. His team realised they could never win by just making search better.

Siri had recently launched, and it felt like the beginning of something exciting. But Sandeep’s team felt Siri was too reactive; you still had to ask it to do something. So they started to wonder: how can you do what Siri does, but get ahead of the questions people are going to have before they ask them? This was also the same time that the new Halo was coming out and they felt Cortana was the perfect metaphor, as she was always looking out for the chief to see what trouble was coming. Everyone in the team wished they had that, the perfect personal assistant who could see around corners for you.

The Windows Phone PMs - about 10 of them in total - started workshopping this idea. One of them in particular, Robert Howard, wrote a concept note exploring what they could do. But all of engineering, and everyone working on Bing, were too busy to talk to them at the time.

That all changed in October 2012 when Windows Phone 8 was finally released, and everyone had cycles again. So the PMs started working on the idea in earnest, whiteboarding it with eng and design. They even shadowed some admins to learn about how they did their jobs. One of the admins kept a notebook with all the key information and interests of the person she looked after - this inspired the team to give Cortana a digital ‘notebook’ which stored any personal information that was available for Cortana to see and use. Users could review the notebook and remove anything they didn’t want Cortana to have access to - this was a transparent privacy model that put users in control and I’m sorry it didn’t become an industry standard for personal AI.

The output of all this work was a science fair-esque poster show that they put on for their partner teams, showing how Cortana would work for a day in the life of a persona called Doug, to try to get their buy-in. And it worked. ‘The beauty of Microsoft’s culture was that people always got excited about hard challenges,’ Sandeep tells me.

Nevertheless, they weren’t given carte blanche to spend all their time on this bet. Sandeep tells me that when they showed the posters to leadership, the reply was: that’s cute, but you have a job to do on search and maps, and that’s your focus - but if you can find spare cycles, go for it. After Windows Phone 8, their product group had a whole heat map of areas where they were losing out to the competition, and they were expected to fill those gaps. In practice the team dedicated 60% of their time to the core work, and 40% to the newly-created Cortana.

Making the milestones

A bit of background on how the Microsoft process worked for each release cycle of Windows Phone: each OS release had 3 big milestones, and every spec had to be fully fleshed out before going into execution. There’s a lot less room for error with a device OS, because you have no flexibility on the hardware timelines. This waterfall process was a problem for the team working on Cortana, because work commenced on the first milestone (M1) before they even knew what they were building. So that they could get moving, but also create future flexibility, they started by building a kind of React into the OS - a framework for web-driven experiences that would allow them to keep innovating outside of client code releases. They also invested in building location tracking, since it seemed like it would have multiple applications. ‘We had to guess what building blocks we’d need, even though we didn’t know what we were actually building,’ says Sandeep.

Before they knew it, M2 rolled around, which meant they only had 18 weeks left of development to build the entire product (M1 and M2 were 12-week cycles; M3 a 6-week cycle). Thankfully, by this point they’d worked out what they wanted to build, and they suddenly received an influx of talent. Smart Search on Windows 8.1 was starting to launch and not doing that well - it didn’t have the adoption or usage that they needed. That team realised the thing they’d been backing wasn’t going anywhere fast, so they tracked down the Cortana team and asked ‘you know that crazy thing you were working on, are you still working on it?’. They decided that Cortana was going to be their strategic bet, and lots of PMs and engineers hungry for impact showed up to work on it.

Sandeep and his fellow PMs were now able to pull in other people from across the business to start working on scenarios with them (‘Microsoft was going through a scenario-driven product moment, not a feature-driven moment’), such as people from calendars and audio. Having then created a list of scenarios much longer than what they could possibly build, they found themselves with a prioritisation exercise. A principal PM joined and became the ‘editor’ of the product, working out the narrative for the whole programme and deciding what to cut. 

Around this time in Spring 2013, marketing also became involved, and insisted that Cortana be given a personality and a sense of humour. Their plan for a public name was Alyx, because it was so pronounceable - Cortana had always been intended as a code name, not a public-facing name. But the name Cortana leaked (someone discovered the string in a beta product) and the bloggersphere went nuts. More than a million people signed a petition demanding that they ship it with the name Cortana. The marketers decided it wasn’t sensible to burn bridges with their nascent influencer group, so (after a minor negotiation with Halo) the name stuck.

Sandeep’s story makes the whole bet sound very bottom-up, so I ask him about the leadership layer for this project. He tells me there were 3 real parents of Cortana: Marcus Ash, who was the head of the product group, and the person who initially created, and then defended, the space for the team to work on it; Robert Howard, who wrote the initial memo; and Sogol Malekzadeh, who was the head of design, content and UXR for the area, and who led the charge on making Cortana feel like a real person. It was so important that Cortana feel like someone you could trust, so that you wouldn’t feel annoyed by her interruptions, and it took very careful design to make that work without feeling like big brother. He also calls out the intrepid eng director, who enabled the PMs to move away from waterfall, and excelled at finding new people to work on the project, persuading entire teams to come over from other orgs.

The trough of sorrow

Going into M3, morale was at its lowest. The team had a bunch of features that were working at 80%, but they worried they weren’t going to make it in time for the release. And once they’d dropped the things they wouldn’t be able to finish, they weren’t sure whether there would be a product left. They had already cut corners: for example, they only had a product for US English, not a global product, which was an issue given that the Windows Phone strategy had shifted away from the original high end US focus to an emerging markets focus. Sandeep tells me: ‘We all stayed up at night wondering if we’d have anything to show for this journey. We worried: was it the right product at the wrong time, for the wrong audience?’.

They started aggressively cutting scenarios, such as Cortana proactive transit alerts, (‘leave now to make it to the metro by 11.15am or you’ll miss your first meeting’). It was just too ambitious to combine the transit data and have a really deep understanding of your calendar to be able to interpret it correctly - so they scoped it back to just driving alerts. Even more painful than losing use cases, they had to cut some of the things that gave Cortana her personality, like her custom voice commissioned from the voice actor of Halo’s Cortana, Jen Taylor. Luckily, their early investment in the react-type framework bailed them out of a lot of their problems by allowing them to develop features server-side.

By now leadership had fully bought into Cortana, and were making it one of the few marketing stories for this release. That was helpful, because the team needed support from above to start flexing some of the release rules. ‘We had to play games to ensure that we could ship in Windows Phone 8.1,’ Sandeep tells me. ‘For example, we’d sneak in new features as bug fixes, or we’d hijack the commercialisation releases for further features or polish. We did outrageous things, like turning up to release committee dressed in onesies so that they’d laugh and let our feature in.’ This stunt allowed them to get Cortana’s voice back into the release.

The one last hurdle to clear was approval from the CEO, a position which was in the process of being handed over from Steve Ballmer to Satya Nadella. ‘Ballmer had poor product taste,’ says Sandeep with a slight smile. ‘He wanted the whole thing to be Microsoft branded. And then his parting gift was to try to name it Bingo. But we waited it out.’ Luckily, Satya had different ideas, and was a huge supporter of the project because of his belief in the power of AI. Under his leadership, Cortana got to keep her name and was declared ready to ship.

Release

Cortana was successfully released as part of Windows Phone 8.1. The user reaction was great - it seemed like the scenarios really landed, and people loved the scenarios, including package tracking and smart reminders, which allowed you to e.g. create a reminder to buy flowers next time you passed a flower shop. The investments in making Cortana have a strong personality and great look and feel were also noticed. Between when they started working on Cortana and when it launched, Google had released Google Now, which was based on the same insight about being proactive, but had the opposite approach to persona and approachability.. Reviews described Cortana as ‘smart and witty’ or ‘like Google Now, but with cohesion and polish’.

There were some negative responses too - mostly connected to the fact that Cortana was US English only - and there were various problems with the OS update in general, including that the flagship device got cancelled. There was also a lingering concern that, although Cortana was good, it wasn’t good enough to make people buy a Windows Phone if they wouldn’t have done otherwise. But overall, the team were delighted at the response. And it was quite a team at this point - almost half of Bing in 2015 was working on Cortana in one form or another, it was a multi-thousand person effort. They geared up to start working on the next release.

The decline

We all know that Cortana didn’t have a bright future. Windows Phone itself was discontinued in 2017, and the Cortana apps were turned off for iOS and Android last year. I ask Sandeep what went wrong, after this promising start. He says the decline began immediately. ‘Everything that was right about the first release went wrong for the second release,’ he tells me. No longer did they have a small team (‘us against the world’), working in an iterative way, investing in a great toolkit. For v2, there were hundreds of PMs trying to get in on the action, and getting anything done required dozens of cross-company meetings. All of the new people diluted the original ethos they’d built and diluted the focus. They’d worked out how to scale their product but not their culture.

They started to ship their org chart: the Windows Phone team had merged with the Windows team, so it felt natural to put Cortana on Windows and XBox. But why? Your computer has a lot less contextual data than your phone, and therefore is in a much worse position to be your handy assistant. Desktops at the time didn’t even have a microphone, or access to the calendar. Across the company, anything that was AI or ML started being branded Cortana, so the name ceased to stand for anything at all. It was the beginning of the end. With the rapid dilution, the brand faltered before it even had a chance to truly reach the masses.

Top takeaways:

• PMs often hate rewrites, or time spent paying down tech debt. Instead, see it as an opportunity to spend more time on product strategy.

• If you have a bet to go after, don’t take no for an answer. Write a product vision and spec so compelling that even the sceptics will reluctantly agree on the opportunity. Get creative and ask for forgiveness not permission.

• You can do incredible things if you learn to tell a great story and win people over to it. Think of the PM as the editor of that story, cutting what’s not essential and making sure it all holds together.

• To ensure long-term success of your bet, learn how to scale your culture as well as your product.