A pharmaceutical company decided to discontinue using Microsoft’s Copilot AI tool, with an executive pointing to its high cost and limited value. After deploying Office 365 Copilot for 500 employees, the company found it underwhelming, likening its performance to that of middle-school presentations. In response, Microsoft is considering restructuring its 365 licenses to enhance the monetization of AI features, particularly as spending in this area reaches unprecedented levels.

AI models and tools often appear impressive when evaluated using specialized benchmarks by researchers. However, their performance can fall short in practical, real-world applications, as illustrated by a recent case involving a pharmaceutical company’s IT executive. 

According to a research note from Morgan Stanley, the company’s chief information officer invested in Microsoft’s Office 365 Copilot, an AI-enhanced upgrade to the productivity suite, for 500 employees during the final quarter of 2023 and the first quarter of 2024. 

After six months, the executive decided to cancel the upgrade, finding the AI tools insufficiently effective to justify the additional cost. The executive likened the slide-generation capability of Microsoft’s AI tools to presentations typical of middle school, as noted in a transcript from a call with Morgan Stanley analysts. Identified only as Greg, he expressed disappointment in the value offered by the tools, especially given the significant cost increase. 

The E3 version of Microsoft’s 365 software suite costs around $34 per user per month, with the addition of Copilot AI features raising the price by another $30 per user per month. For 500 employees, this would result in an approximate annual increase of $180,000. 

Despite being regarded as leading examples of advanced artificial intelligence in practical use, these new Microsoft tools have fallen short of expectations for this pharmaceutical company, even as investors continue to drive up the stock prices of Microsoft and other major tech firms, anticipating widespread adoption by paying customers.

If a major pharmaceutical company finds it difficult to justify $180,000 in value from these tools, it raises concerns that could resonate across the entire tech industry.

The 365 Copilot AI feature that the IT executive found more appealing was its capability to archive and summarize video meetings on Microsoft’s Teams app. However, the company’s legal team expressed concerns about retaining meeting transcripts, leading them to decide against using this feature.

The executive expressed skepticism about the utility of the tools, stating that they seemed more focused on building something akin to ChatGPT rather than enhancing core applications like Word. He described the generative AI slide capability as being at the level of middle school presentations and noted that the AI features in Excel were not something most spreadsheet users would find practical. He also mentioned that the company might consider adopting Office 365 Copilot for a limited group of employees if the price were significantly lower.

Microsoft Faces Internal and External Pressure to Justify AI Investments Amid Concerns Over Value and Pricing Strategies

These comments come amid internal concerns at Microsoft about whether its AI services will generate enough value to convince corporate customers to pay more. Wall Street is also questioning how Microsoft will recover its substantial AI investments. 

The company is acquiring 1.8 million graphics processing units to develop and operate AI models and related products, alongside plans to triple its data-center capacity, primarily to support AI workloads. In the most recent quarter, Microsoft’s capital expenditure reached a record $14 billion. 

A company spokesperson declined to comment. Microsoft is reportedly considering changes to how it packages 365 software licenses, like E3, to increase revenue from AI. Possible strategies include integrating AI features into existing licenses or creating a more expensive bundle that includes Copilot AI capabilities, according to a source familiar with the plan.