Palantir AI Engineer – Associate Study Guide: An Exam About Proving That AI Works
You might expect an AI engineering exam to be about prompts. The Palantir AI Engineer – Associate exam is mostly about evidence: how you know a Logic function, an agent or a retrieval pipeline works, how you find out why it stopped working, and which controls hold even when the model misbehaves. In the 475 PLTR-AIE questions in this question bank, the tool that appears most often is not AIP Logic or AIP Chatbot Studio (formerly AIP Agent Studio) but AIP Evals, tagged on 139 questions (29%). This guide is organized around that fact: first how to measure, then how to diagnose, then what to enforce outside the prompt, and only then the design choices.
Study guide · Palantir AI Engineer – Associate (PLTR-AIE)

The exam in brief
95 multiple-choice questions in 3 hours, open-book, pass mark 70% correct, across eight domains from AIP Logic Development to Security, Governance and Responsible AI. Certificates are valid for two years, with a 14-day wait before a retake after a failed attempt. Palantir does not publish domain weights, so our practice sets treat the eight domains equally. Check the official exam page for current details. All our questions have a single correct answer; more than a quarter are troubleshooting scenarios and about a fifth are trade-offs.
Part 1: what a good evaluation looks like
An evaluation suite has test cases, a target function (an AIP Logic function, a chatbot published as a function, or a code-authored function), evaluators and metrics. Each case has an input, an expected result confirmed independently (never copied from the function under test) and an evaluator that compares them. Coverage matters more than size: a few hundred well-chosen cases, including deliberately hard ones, beat thousands of clean samples.
- Deterministic evaluators first. Exact match for IDs, dates and amounts; set comparison for unordered lists; length limits; citations that must match retrieved chunks. Structured outputs are scored field by field.
- Model graders for free text, with a per-criterion rubric (required facts, no fabrication, tone, length), calibrated against human ratings. Watch for bias towards longer answers. A new grader means a new score scale and a new baseline.
- Groundedness is not correctness. An answer can sound right without being supported by the retrieved passages; check claims against them.
- Agents need tool-selection checks: compare the tools called, and their arguments, with the expected calls, including cases where no tool should be used.
- Report metrics separately (correctness, groundedness, retrieval hit rate, refusal behavior, latency, cost), with a release gate per component, so one strong number cannot hide a regression.
Refusal is measured in both directions: cases where the agent must refuse or escalate, and cases where refusing would be wrong. Unanswerable cases are built from verified gaps in the corpus and expect an explicit "not found".
Going deeper: regression or run-to-run variance?
Model output varies between runs, so a two-point drop may be noise. Before calling a regression, run both versions several times on the same unchanged suite with the same model settings and evaluators, estimate the spread, and look at which cases flip and why. Set thresholds relative to that measured variance. Never change the suite and the function in the same comparison: if you add new cases while testing a new prompt, you cannot tell distribution shift from a real drop. A comparison is reproducible only when suite version, grader, model configuration and parameters are all the same. And because averages hide local damage, segment results by category and consequence.
Beginner tip: before you study the other domains, write a ten-row evaluation suite by hand in a spreadsheet for an imaginary support-ticket classifier. Give each row an input, the expected output and the evaluator type (exact match, set match or rubric). Include at least one ticket the classifier should refuse, one with no answer in the knowledge base, and one with an instruction hidden in its text. Once you have filled in those rows yourself, questions about suite design read like checklists.
Part 2: diagnose from evidence
When something goes wrong, the exam rewards the option that reads the evidence before changing anything.
- Logic debugging and traces. A test run opens the Logic Debugger, which shows the model's step-by-step reasoning for each block and the tools it used; AIP observability adds execution history, distributed tracing and logs. Walk the evidence in order: the prompt the model received, each tool call and its result, the model's response, the parsed output. A literal placeholder left in the prompt means a variable binding failed, often after a rename.
- Retrieval versus generation. Measure them separately. A labelled retrieval set gives hit rate at k and precision. Good retrieval with bad answers points to context assembly or the prompt. If recall rose and answers got worse, the context became noisier: work on precision with metadata filters, a relevance threshold or reranking, not on retrieving more.
- Production drift. If suite scores stay stable while production metrics fall, the inputs have changed. Refresh the suite with recent production cases.
- Latency and cost. Look at p95 and p99, broken down by step. Split cost into volume and size per request; spikes when users are idle point to automations, backfills or retry loops.
Part 3: controls the prompt cannot provide
Prompts are a layer that does not enforce anything. When a requirement says "must never", the answer is a platform control.
| Requirement | Enforced by |
|---|---|
| The agent must never place orders | Not giving it any tool or Action that can |
| Users see only documents they may read | Retrieval running as the user, with permission and approval filters applied before ranking |
| AI output must not corrupt data | Action validation, kept even when the function already checks values |
| High-consequence changes | Human approval, tiered by consequence |
| Draft documents must not be cited | An approval-status filter enforced by the pipeline, not the model |
| Data must stay in a region | Model enablement and georestrictions |
| Resist injected instructions | Read-only, narrowly scoped tools; allow-listed sources; approval before consequential actions |
Going deeper: why it works for the author but fails in an automation
Automate evaluates conditions and runs Action and Logic effects with the permissions of the automation's owner, while notification effects use each recipient's permissions. A function that works when its author runs it can therefore fail inside an automation owned by someone else, because a different principal lacks access to the objects or tools. The fix is to identify that principal and grant exactly what it lacks, or to change the execution identity deliberately, for example to a least-privilege service identity for a narrow elevated task. Widening access for everyone is never the right answer. The same reasoning explains why user-facing retrieval runs as the user: the Ontology's permissions then decide what each person sees, and an unauthorized user simply gets an empty result.
Part 4: the design choices
- Logic or code. AIP Logic for language understanding; code for exact arithmetic, fixed rules and large scheduled batches. Mix them: rules first, the model for the residue.
- Typed outputs. Enumerations, numbers, booleans and structs, with "not found" as an explicit field, instead of free text that a consumer must parse.
- Agents. A business assistant is an AIP Chatbot built in AIP Chatbot Studio (formerly AIP Agents and AIP Agent Studio); AIP Assist answers questions about using the platform. Embed the chatbot in Workshop when it needs the selected object (passed in through application variables); use it elsewhere, such as AIP Threads or through the Ontology SDK and platform APIs, when it does not.
- Retrieval. Chunk by section with metadata (document, version, effective date); re-embed the corpus when the embedding model changes. Structured facts such as a price list stay as objects with exact lookup.
- South or north. Reusable, row-wise results go into the pipeline as properties; answers that depend on the current user and moment are generated at interaction time.
- Models. Filter by hard constraints (capability, residency, latency), then take the smallest model that passes the suite. Set up a tested fallback from another provider before you need it.
- Cost. Measure first, remove waste, add caching, route easy cases to a cheaper model, and change the main model last.
Distractors to recognize
Wrong options tend to look alike: a bigger model, a longer prompt, more retrieved context, a higher temperature, broad access, silent filtering, or publishing a fix to every consumer at once. Another family pushes the answer outside the AI engineer's role, such as training a custom foundation model when retrieval would solve the problem. When two options both sound reasonable, choose the one that finds the cause, keeps a guard in place and can be verified with an evaluation or a trace.
A study order
- Evaluation, testing and quality assurance first; every other domain refers back to it.
- AIP Logic, then agents.
- RAG and semantic search, then AI-powered data processing.
- Automation and orchestration; model selection, observability and cost.
- Security, governance and responsible AI, then timed mixed sets.
Practise
Practise in the question bank: every question explains each option, right and wrong. Start with the domain you just studied, then take a timed mock exam.
- PLTR-AIE exam page · All 535 PLTR-AIE questions · Timed mock exam · 17-day study plan
- By domain: AIP Logic Development · AIP Agent Design and Deployment · RAG Architecture and Semantic Search · AI-Powered Data Processing · Evaluation, Testing, and Quality Assurance · Automation, Orchestration, and Integration · Model Selection, Observability, and Performance · Security, Governance, and Responsible AI
Exam facts as described in Palantir's exam guide for the AI Engineer – Associate certification. Domain weights are not published; check the official exam page for format and policy changes.