AI · Flashcard
A developer is told the model processes text as tokens rather than whole words. What is a token most accurately?
- AA chunk of text such as a word piece or subword, the unit the model reads and generates
- BExactly one English word, since the model always maps a single word to a single token cleanly
- CA single character, since the model reads text strictly one letter at a time as it generates
- DA full sentence, since the model consumes and emits complete sentences as its atomic unit
Why this is the answer
A token is a subword chunk — the unit the model reads and produces. It is not exactly one word (2), not a single character (3), and not a whole sentence (4); tokenization sits between characters and words.
Official docsMore Claude Certified Developer — Foundations flashcards
- A team builds a chat feature and assumes the API remembers previous messages by conversation ID, but Claude never recalls earlier turns. Why?
- A team must pick a model for a new feature. What is the fundamental trade-off across model tiers?
- A team debates whether their feature is an agent or a workflow. What most clearly distinguishes an agent?