🔍 Read the full analysis: Amazon’s AI Investments Put A Spotlight On What Engineers Measure on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Amazon published engineering guidance on October 27 telling software teams to judge AI by customer, business and team outcomes rather than token counts or tool adoption. The memo says usage metrics can still help identify patterns, but using them as targets risks higher costs and complexity. Amazon’s Nova and Anthropic investments provide a broader, still-developing test of how the company measures AI returns.
Amazon published new guidance for its software engineers on October 27, saying teams should assess artificial-intelligence work by its impact on customers, the business and colleagues—not by token counts or how often workers use AI tools. The memo matters beyond internal performance reviews: it sets out a measure of success for a company making large AI investments, while its own results from those bets remain mixed and incomplete.
The memo, published on Amazon’s news site, is attributed to Terk Terkowitz, a vice president and distinguished engineer in the company’s stores business. It introduces a revised role guideline for software engineers, written by senior engineers. Rather than prescribe specific AI tools, the guideline focuses on outcomes that can outlast changes in technology. The memo says a definition tied to today’s AI tools could look outdated within a year.
Amazon acknowledges that long-term outcomes can take time to measure. Some teams have consequently used token usage, AI-tool adoption, code changes and reviews as stand-ins. The memo calls these “deeply flawed metrics” when disconnected from meaningful results, warning that turning them into targets can raise costs and add complexity. It specifically says tokenmaxxing conflicts with frugality and using lines of code as a target conflicts with simplifying systems.
The guidance does not say AI use is unhelpful. It says these input measures currently “correlate well” with important outputs, and that teams adopting AI more extensively are delivering more, while teams that do not adopt it are “falling behind.” Its distinction is between tracking usage to understand work and scoring employees for usage. Leaders are told to examine both low and high adoption, use AI on their own production work, and record where the tools help or fail to help.
Six months at Amazon
From token leaderboards to an outcomes memo
Amazon spent 2026 learning the same lesson twice: once with how its engineers use AI, once with its own AI models.
- May 2026Token leaderboards backfireFinancial Times (via The Decoder): staff give an internal agent tool pointless tasks to inflate token counts. Reported, anonymous sources.
- Late May 2026KiroRank “deprecated”Amazon spokesperson to Business Insider. SVP Dave Treadwell: “Please don’t use AI just for the sake of using AI.”
- 24 Jul 2026San Francisco AGI Lab site closesConfirmed by Amazon to GeekWire. Frontier model research continues under Pieter Abbeel.
- 28 Jul 2026Most flagship Nova models wound downBusiness Insider, people familiar: Premier, Omni, Reel, Canvas to “keep the lights on”. Amazon: models “remain one of the most important things we’re working on”.
- 14 / 30 Sep 2026Nova Sonic v1, Nova Reel, Nova Canvas reach end of lifeAWS Bedrock model lifecycle page. Requests to them now fail.
- 26 Oct 2026Anthropic files its S-1Amazon notes: $6.2 billion principal, 12% ownership cap that Amazon can waive. Anthropic commits at least $110 billion to Amazon through April 2036.
- 27 Oct 2026The memo: outcomes, not tokenmaxxingUsage and throughput metrics are “deeply flawed” as targets. “Tokenmaxxing is at odds with frugality.”
- 29 Oct 2026Amazon’s third-quarter resultsAnnounced by Amazon. Not yet published when this article went live.
Sources: Amazon engineering memo (27 Oct 2026); Business Insider (29 May and 28 Jul 2026); GeekWire (24 Jul 2026); The Decoder (12 May 2026, summarising the Financial Times); AWS Bedrock model lifecycle (read 28 Oct 2026); Anthropic S-1 (26 Oct 2026).
Powered by Thorsten Meyer AI
From AI Activity to Engineering Results
The memo challenges a common management shortcut: counting activity that is easy to track instead of testing whether it improves a product or service. Tokens consumed and tools opened can show that AI is being used, but they do not establish that a customer received a better experience, that software became more reliable, or that a team finished useful work faster.
That distinction affects both costs and employees. If token consumption becomes a target, workers may generate activity that uses computing resources without improving an outcome. If code volume is rewarded, teams may add complexity rather than simplify systems. Amazon’s guidance instead treats adoption data as a diagnostic: managers should investigate why usage is low or high and whether the work benefits.
The advice also gives engineers a way to report limits without treating them as poor performance. The memo says that when current AI tools do not help a particular task, that is a finding to document and share. That approach could help companies learn where tools are useful, though the memo does not set out a universal formula for measuring productivity or customer impact.
Products Worth Considering
Samsung 27-Inch Odyssey QD-OLED G8 4K 240Hz Gaming Monitor
LG 34GX90SA-W 34-inch Ultragear WQHD OLED Curved Gaming Monitor
LG UltraGear 32GX850A-B 32-Inch 4K UHD Glossy OLED Gaming Monitor
As an affiliate, we earn on qualifying purchases.
Amazon’s Models and Anthropic Stake
The memo follows earlier reports about internal AI-use tracking. The Financial Times reported in May that some Amazon employees gave an internal agent unnecessary tasks to increase token counts. Those accounts were anonymous and are not independently confirmed by the memo. Amazon did confirm to Business Insider at the end of May that an informal employee-built leaderboard called KiroRank had been “deprecated,” saying it was never intended to encourage AI use for its own sake. Senior vice president Dave Treadwell also told staff, “Please don’t use AI just for the sake of using AI.”
Amazon’s broader AI portfolio offers a separate test of outcomes. Business Insider reported in July that most flagship Nova models had been moved to a “keep the lights on” status. Amazon’s lifecycle information lists Nova Canvas, Nova Reel and the first Nova Sonic as retired. These developments suggest changes in product support, but do not by themselves establish the financial performance of Amazon’s entire Nova effort.
Amazon also has a major investment in Anthropic, the maker of Claude models. The source report says Amazon invested about $18 billion and that a June 30 filing valued its holdings at about $190 billion. Those figures describe a reported investment and a paper valuation, not cash proceeds or a realized return. Amazon is scheduled to report third-quarter results on October 29, one day after the memo was published; those results may offer further financial context.
““Tokenmaxxing is at odds with frugality.””
— Amazon engineering memo
The Returns Still Need Evidence
The memo does not provide a company-wide measure of how much AI has improved engineering output, nor does it quantify cost savings, product quality changes or customer impact. It says usage measures correlate with important outputs at present, but does not publish the underlying data or define how teams should compare different kinds of work.
The reported $190 billion valuation of Amazon’s Anthropic holdings is a paper figure tied to a filing, not a realized gain. The source material also does not establish the full financial outcome of Amazon’s Nova effort or explain the reasons behind each model’s status. Reports about employees inflating usage are based on anonymous accounts; Amazon’s confirmation that KiroRank was discontinued does not independently verify every reported example.
Earnings Put the Bets in View
Amazon’s next stated milestone is its third-quarter earnings report on October 29. The results may shed light on the company’s financial performance and AI spending, but they will not necessarily resolve how much value its engineering teams get from AI tools. The memo itself sets no date for a follow-up evaluation or release of adoption and outcome data.
For now, Amazon’s stated direction is to use adoption figures as evidence to investigate, not as a score to optimize. The practical test will be whether managers can connect AI use—or the decision not to use it—to documented changes in customer outcomes, business results and team effectiveness.
Key Questions
What did Amazon tell engineers to measure?
The memo says to focus on long-term outcomes for customers, the business and the team, rather than treating token use or AI-tool adoption as performance goals.
Is Amazon telling employees to stop using AI?
No. The memo says AI adoption can correlate with important outputs and warns that teams not adopting AI may fall behind. Its objection is to rewarding usage for its own sake.
What is tokenmaxxing?
It refers to increasing AI token consumption, including by giving tools unnecessary work. The memo says token counts are a flawed target because they can encourage extra cost without demonstrating useful results.
What do Amazon’s Nova and Anthropic moves show?
They provide different examples of Amazon’s AI portfolio: several Nova products have been retired or placed in maintenance, while Amazon holds a large reported investment in Anthropic. The available figures do not establish the realized financial return on either effort.
When will Amazon report its next results?
Amazon is scheduled to report third-quarter results on October 29, 2026, the day after it published the engineering memo.
Source: ThorstenMeyerAI.com
