Discover · fb.com

Meta Andromeda: Powering the Next Generation of Ads Retrieval Through Model-System Co-Design

Engineering at Meta fb.com

This post describes the next generation of Meta Andromeda, Meta’s personalized ads retrieval engine, and how co-designing the model, system, and GPU hardware unlocks a new efficiency frontier for ads retrieval.

In production, the system scores every eligible ad with near ranking-level model performance at retrieval scale and surfaces the top candidates at over one billion ads per second, a new throughput benchmark for ads retrieval-stage models (i.e., a 100x lift compared to the 2024 Meta Andromeda system).

More ways to use this
Open on another device
Scan this QR code to open the same public page

Scan to open this page. Your saved notes and account access are not included.

Download QR code

Open the page link

Names and references

  1. Generation — person
  2. Design — person
  3. System — person

In the piece

Challenges and Opportunities in Scaling RetrievalThe 2024 Andromeda deployment reimplemented retrieval with an end-to-end optimized retrieval implementation from custom AI hardware with hierarchical indexing and GPU-native operators to a purpose-built ML model, growing model capacity by four orders of…Facebook EngineeringHigher Model Performance Requires a New ArchitectureAds retrieval has an outsized ROI on incremental quality: Small improvements in retrieval model performance can drive large gains in advertiser value. The rapid expansion of our ads candidate pool makes optimizing for hardware capacity ROI an increasingly…Facebook EngineeringNew System Bottlenecks at Billion ScaleThe 2024 Meta Andromeda system evaluates at tens of millions of ads per second per GPU card. Meeting the new business demand requires a throughput at multi-billion ads evaluated per second per card in the system. At billions of ads per second throughput, the…Facebook EngineeringPushing Towards the Hardware LimitsWith retrieval running on GPU, the important optimization question is “How close are we to the hardware ceiling, and how much room remains for future model growth?” Answering this requires pushing each component of the retrieval pipeline to the limits of…Facebook EngineeringUnlocking Retrieval Performance Through Model-System Co-DesignFacebook EngineeringNext-Gen Retrieval Model Architecture — Retrieval Mixture-of-ExpertsFacebook EngineeringCo-Designing the Model and System Onto a Single GPU PipelineFacebook EngineeringFused KernelFacebook Engineering

Keep going

Each of these is read from what this estate already kept: the feed a piece came through, the publisher behind it, the credits and topics the publisher printed. None of it is a recommendation, an identity or a record kept anywhere else.

More like this

Also from Engineering at Meta

  1. Cuttlefish on Mac: Running Android’s Virtual Device Framework Natively on Apple Siliconfb.com
  2. Intelligent Delivery: Making Every Unit of Compute Serve Better Adsfb.com
  3. Your Algorithm: Giving People More Ways to Personalize Facebook Recommendationsfb.com
  4. How We Built TLX: A Hardware-Native GPU Compiler for Production ML at Scalefb.com
  5. Extending LLDB to Debug Applications That Span CPUs and GPUsfb.com
  6. F3 Lightning: How Meta Accelerates Ads ML Features to Production at Scalefb.com