On-Device Models for PMs: Memory Arithmetic, Hybrid Routing and What Runs on a Phone
A model on the device answers offline, keeps data local and costs the provider nothing per request, but it must fit in memory, run within battery and heat limits and accept a quality gap. Learn the memory arithmetic, how Apple fits a roughly 3-billion-parameter model on phones, and how to design the split between device and cloud.
CONCEPT / MODELS & PLATFORMS
FREE ACCOUNT / CONCEPT
Read this concept with a free account
Models & Platforms
Work out what fits on a device, what it costs the user, and when to hand a request to the cloud.
01
Understand the mechanism
Read the full explanation behind this product decision.
02
Use it in an interview
Follow the related practice questions and put the idea into an answer.
Your free account includes this concept. No card needed.