AI Agent Engineer

COURSE CATALOG10 knowledge domains · 57 course entries · 4 role tracks

ROLE TRACKS

Agent application development32

AI Foundations Learning PathCore
Python FundamentalsRecommended
JSON Learning IndexRecommended
Data Structures FundamentalsRecommended
API Learning PathRecommended
MarkdownRecommended
Git Learning PathRecommended
Linux CommandsOptional
Regular ExpressionsOptional
Probability and StatisticsRecommended
Machine LearningRecommended
Modern LLM Capabilities and Model SelectionCore
Prompt EngineeringCore
Context Engineering Learning PathCore
LLM API Integration Learning PathCore
Tool Calling (including Function Calling)Core
Agent CoreCore
MCPOptional
Environment-based AgentsOptional
Agent Skills Learning PathOptional
Agentic Design PatternsOptional
LangChainOptional
CrewAI Learning IndexOptional
Workflow Automation Learning PathCore
Data Annotation Learning PathRecommended
Data VisualizationRecommended
Evaluation Framework Learning PathRecommended
LLMOps Learning PathRecommended
AI Safety Learning PathCore
Privacy-Enhancing Technologies Learning PathRecommended
Multi-Agent Collaboration Learning PathOptional
A2A ProtocolOptional

RAG and knowledge bases35

AI Foundations Learning PathCore
Python FundamentalsRecommended
JSON Learning IndexCore
Data Structures FundamentalsOptional
API Learning PathRecommended
MarkdownRecommended
Git Learning PathRecommended
Linux CommandsOptional
Regular ExpressionsRecommended
Probability and StatisticsRecommended
Linear AlgebraRecommended
Machine LearningRecommended
Modern LLM Capabilities and Model SelectionCore
Prompt EngineeringCore
Context Engineering Learning PathCore
LLM API Integration Learning PathCore
Multimodal AI Learning PathOptional
Data CleaningCore
OCROptional
Speech RecognitionOptional
Document ParsingCore
Knowledge Base ConstructionCore
Chunking StrategiesCore
Vector FundamentalsRecommended
EmbeddingsCore
Vector DatabasesCore
Semantic Search Learning PathCore
Data Annotation Learning PathRecommended
RerankingCore
RAG Learning PathCore
Data VisualizationRecommended
LangChainOptional
Evaluation Framework Learning PathCore
AI Safety Learning PathCore
Privacy-Enhancing Technologies Learning PathRecommended

Agent platform and reliability38

AI Foundations Learning PathCore
Python FundamentalsRecommended
JSON Learning IndexRecommended
Data Structures FundamentalsRecommended
API Learning PathRecommended
MarkdownRecommended
Git Learning PathRecommended
Linux CommandsRecommended
Regular ExpressionsOptional
Probability and StatisticsRecommended
Machine LearningRecommended
Modern LLM Capabilities and Model SelectionCore
Prompt EngineeringCore
Context Engineering Learning PathCore
LLM API Integration Learning PathCore
Tool Calling (including Function Calling)Core
Agent CoreCore
MCPOptional
Environment-based AgentsOptional
Agent Skills Learning PathOptional
Agentic Design PatternsOptional
LangChainOptional
CrewAI Learning IndexOptional
Workflow Automation Learning PathCore
Data CleaningRecommended
Data Annotation Learning PathRecommended
Data VisualizationRecommended
Evaluation Framework Learning PathCore
Synthetic DataRecommended
Benchmark Design Learning PathCore
MLOps Learning PathRecommended
LLMOps Learning PathCore
Runtime Monitoring Learning PathCore
AI Safety Learning PathCore
Privacy-Enhancing Technologies Learning PathRecommended
AI Governance Learning PathCore
Multi-Agent Collaboration Learning PathRecommended
A2A ProtocolOptional

Multimodal and real-time interaction29

AI Foundations Learning PathCore
Python FundamentalsRecommended
JSON Learning IndexRecommended
Data Structures FundamentalsOptional
API Learning PathRecommended
MarkdownRecommended
Git Learning PathRecommended
Linux CommandsOptional
Regular ExpressionsOptional
Probability and StatisticsRecommended
Linear AlgebraRecommended
Machine LearningRecommended
Deep LearningRecommended
Modern LLM Capabilities and Model SelectionCore
Multimodal AI Learning PathCore
OCRRecommended
Speech RecognitionCore
Text to SpeechCore
Tool Calling (including Function Calling)Core
Agent CoreCore
Data Annotation Learning PathRecommended
Data VisualizationRecommended
Real-Time Multimodal InteractionCore
Evaluation Framework Learning PathCore
AI Safety Learning PathCore
Privacy-Enhancing Technologies Learning PathRecommended
Image Generation Learning PathOptional
Video Generation Learning PathOptional
Multi-Agent Collaboration Learning PathOptional

KNOWLEDGE DOMAINS

01Engineering and mathematical foundations14

01AI Foundations Learning Path
Course overview
1. Concept Map6
  AI Map and Core Terms
  Data, Models, Training, and Inference
  Generalization, Data Splits, and Leakage
  Evaluation Evidence and Feedback Loops
  How LLMs Generate Answers
  How Agents Complete Tasks
2. Engineering Decisions4
  Capability Boundaries and Failure Modes
  When to Use AI and Select a System Shape
  From Prototype to Launch and Exit
  Responsible Use and Risk Controls
3. Project And Self Assessment2
  Integrated Project: Meeting Action-Item Assistant
  Course-Wide Self-Check and Mastery
02Python Fundamentals
Course overview
1. Minimum Python Practice for Agent Engineering
2. Engineering Route12
  Foundations And Boundaries4
  Interpreter, Virtual Environments, and Dependencies
  Type Hints and Data Models
  Functions, Modules, and Dependency Injection
  Files, JSON, and Input Validation
  Reliability And Testing3
  Exceptions, Timeouts, Retries, and Resource Management
  Configuration, Logging, and Sensitive Information
  Unit Testing, Mocks, and Regression
  Concurrency And Delivery3
  Async Concurrency, Cancellation, and Rate Limits
  Project Layout, CLI, and Reproducible Runs
  Agent Tool Function Design
  Project And Self Assessment2
  Project: Reliable Task Queue
  Course-Wide Self-Check and Mastery
03Data Structures Fundamentals
Course overview
1. Foundation Models2
  Data Structures, Abstract Data Types, and Invariants
  Complexity, Resource Costs, and Measurement
2. Linear Structures3
  Contiguous, Linked, and Binary Search
  Mutability, Copying, and Iteration
  Stacks, Queues, and Deques
3. Indexing And Priority2
  Hash Tables, Dictionaries, Sets, and Counting
  Heaps and Stable Priority Queues
4. Relational Structures2
  Trees and Hierarchical Traversal
  Graphs, Dependencies, and Topological Order
5. Engineering Selection1
  Structure Composition, Capacity Boundaries, and Performance Diagnosis
6. Project And Self Check2
  Project: Dependency Task Scheduler
  Course Self-Check and Mastery Review
04JSON Learning Index
Course overview
05API Learning Path
Course overview
06Markdown
Course overview
07Git Learning Path
Course overview
08Linux Commands
Course overview
09Regular Expressions
Course overview
10Probability and Statistics
Course overview
11Linear Algebra
Course overview
12Calculus Fundamentals
Course overview
13Machine Learning
Course overview
14Deep Learning
Course overview
1. Deep Learning: Engineering Practice and a Modern Workflow
2. Dive into Deep Learning: Sources and Full Index

02Models and context4

18.5Modern LLM Capabilities and Model Selection
Course overview
19Prompt Engineering
Course overview
20Context Engineering Learning Path
Course overview
21LLM API Integration Learning Path
Course overview

03Retrieval and data11

15Vector Fundamentals
Course overview
16Data Cleaning
Course overview
17Data Annotation Learning Path
Course overview
22Document Parsing
Course overview
23Knowledge Base Construction
Course overview
24Chunking Strategies
Course overview
25Embeddings
Course overview
26Vector Databases
Course overview
27Semantic Search Learning Path
Course overview
28Reranking
Course overview
29RAG Learning Path
Course overview

04Multimodal systems7

48Multimodal AI Learning Path
Course overview
1. Foundations And Architecture3
  Modalities, Representations, and Alignment
  Image, Text, Audio, and Video Input Contracts
  Fusion Architecture Intuition
2. Engineering And Quality4
  Preprocessing and Spatiotemporal Evidence
  Multimodal Prompting and Structured Output
  Multimodal Quality Evaluation
  Latency, Cost, Privacy, and Safety
3. Project And Self Assessment1
  Offline Multimodal Evidence Routing Project
49OCR
Course overview
1. Engineering And Quality4
  Layout, Tables, and Reading Order
  Language Post-processing and Confidence
  Evaluation, Error Analysis, and Human Review
  Privacy, Deployment, and Operational Troubleshooting
2. Foundations And Data3
  The OCR Pipeline and Input Contract
  Imaging Principles and Preprocessing
  Text Detection, Recognition, and Data Annotation
3. Project And Self Check1
  Project: Structured OCR Result Audit
50Speech Recognition
Course overview
1. Engineering And Quality4
  VAD, Segmentation, Timestamps, and Speakers
  Batch, Streaming, and Post-processing
  WER, CER, and Slice Evaluation
  Fairness, Privacy, Deployment, and Troubleshooting
2. Foundations And Data3
  Audio Digitization and the ASR Pipeline
  Acoustic, Language, and End-to-End Intuition
  Data Annotation and Splitting
3. Project And Self Check1
  Project: Offline Transcript Evaluation
51Text to Speech
Course overview
1. Engineering And Quality4
  SSML and pronunciation control
  Voice selection, batch processing, and streaming
  Quality, intelligibility, latency, and evaluation
  Cloning risk, disclosure, and traceability
2. Foundations And Data3
  The TTS pipeline and text normalization
  Phonemes, prosody, and vocoder intuition
  Data and voice authorization
3. Project And Self Check1
  Project: an offline synthesis plan and SSML
51.5Real-Time Multimodal Interaction
Course overview
52Image Generation Learning Path
Course overview
1. Foundations And Generation Principles3
  Image-Generation Task Types and Image Representations
  Diffusion and Autoregressive Intuition
  Conditioning and Prompt Design for Image Generation
2. Engineering And Quality5
  Composition, Style, Text, and Image Editing
  Image-Generation Workflows and API Boundaries
  Image-Generation Quality Evaluation and Human Review
  Image-Generation Safety, Copyright, and Content Credentials
  Image-Generation Cost, Reproducibility, and Troubleshooting
3. Project And Self Assessment1
  Project: Image Generation Task Audit
53Video Generation Learning Path
Course overview
1. Foundations And Generation Principles3
  Video Fundamentals and Timing
  Video Generation Principles and Temporal Consistency
  Text and Image Conditioning for Video Generation
2. Engineering And Quality5
  Storyboarding and Shot Lists
  Video Generation and Post-Production Assembly Workflow
  Video Audio, Captions, and Accessibility Interfaces
  Video Quality, Motion, and Identity-Consistency Evaluation
  Video Generation Cost, Failure Recovery, and Provenance Governance
3. Project And Self Assessment1
  Project: Video Generation Job Package Validation

05Agent runtime8

30Tool Calling (including Function Calling)
Course overview
31MCP
Course overview
1. Learning Path8
  MCP Architecture and Roles
  MCP Primitives and Tool Contracts
  MCP Lifecycle, Capability Negotiation, and Transports
  MCP Authorization and Security Boundaries
  MCP Inspector, Debugging, and Version Management
  Project: Offline MCP Message Validation
  Frontier Topic: MCP Extensions, Apps, and Version Compatibility
  Project: Loopback Streamable HTTP and OAuth Resource Boundaries
32Agent Core
Course overview
32.5Environment-based Agents
Course overview
33Agent Skills Learning Path
Course overview
1. Examples1
  Text Statistics1
  Text Statistics
2. Learning Route7
  Positioning, Client Differences, and Permission Boundaries for Agent Skills
  Agent Skills Open Format and Directory Structure
  SKILL.md Frontmatter and Progressive Disclosure
  Agent Skills Trigger Descriptions and Scope Boundaries
  Agent Skills Scripts, Resources, and Safety
  Agent Skills Testing, Evaluation, and Iteration
  Project: Create and Validate an Agent Skill
34Agentic Design Patterns
Course overview
1. Beginner Route7
  Pattern Selection and Minimal Architecture
  Routing, Parallelism, and Joining
  Reflection, Planning, and Stopping Conditions
  Tools, Memory, and State Boundaries
  Human Approval and Safety Boundaries
  Failure Recovery, Evaluation, and Observability
  Project: Recoverable Task Workflow
35Workflow Automation Learning Path
Course overview
47Multi-Agent Collaboration Learning Path
Course overview
1. Engineering And Quality5
  Message Protocols and Shared State
  Conflicts, Synchronization, and Failure Recovery
  Budgets, Stopping, and Human Intervention
  Evaluation, Observability, and Security Boundaries
  Identity, Authorization, and Cross-Boundary Trust
2. Fundamentals And Architecture3
  When to Use Multi-Agent Systems
  Roles, Topologies, and Responsibility Boundaries
  Task Decomposition and Delegation Contracts
3. Project And Self Check1
  Offline Collaboration Simulator Project

06Framework practice2

36LangChain
Course overview
1. Beginner Route10
  Component Map, Environment, and a Minimal LCEL Invocation
  Models, Messages, Prompts, and Structured Output
  Tools, Agent Loops, and Middleware
  Retrieval and RAG Components
  Memory, State, and Persistence
  LangGraph Boundaries, Approval, Recovery, and Evaluation
  Project: Offline Tool-Agent Skeleton
  Project: LangGraph Recoverable Approval Flow
  LangChain Testing, Evaluation, and Upgrade Checklist
  Project: Keyless create_agent Runtime Contract
37CrewAI Learning Index
Course overview

07Evaluation and reliability4

18Data Visualization
Course overview
39Evaluation Framework Learning Path
Course overview
1. Foundations And Design2
  Evaluation Objectives and Basic Units
  Cases, Datasets, and Stratification
2. Methods And Quality6
  Deterministic Assertions, Metrics, and Scoring Rules
  Human Review and Model-Based Evaluation
  Layered RAG, Generation, and Agent Evaluation
  Offline, Online, Statistics, and Regression
  Evaluation Reporting, Audit, and Governance
  Offline-to-Online Evidence Handoff and the Regression Loop
3. Project And Self Check1
  Project: Offline Layered Evaluation Pipeline
42Benchmark Design Learning Path
Course overview
1. Foundations And Design3
  From Claim to a Benchmark Contract
  Task Space, Representativeness, and Stratification
  Data Splits, Leakage, and Contamination
2. Methods And Quality4
  Baselines, Metrics, and Comparable Runs
  Reproducibility, Statistics, and Results Reporting
  Leaderboard Mechanics, Anti-Gaming, and Maintenance
  Agent Environments, State, and Repeated Runs
3. Project And Self Check1
  Project: Build a Maintainable Benchmark
43Synthetic Data
Course overview
1. Foundations And Design3
  Purpose and Data Contract
  Template and Programmatic Generation
  Model Generation and Condition Coverage
2. Methods And Quality4
  Filtering, Deduplication, and Group-Level Splits
  Quality, Utility, and Real-Data Calibration
  Privacy, Memorization, Bias, and Copyright
  Versioning, Evaluation, and Release
3. Project And Self Check1
  Project: Offline Synthetic Evaluation Data Pipeline

08Safety and governance3

44AI Safety Learning Path
Course overview
1. Foundations And Risks3
  Assets, Trust Boundaries, and Threat Modeling
  Prompt Injection and Indirect Injection
  Tool Overreach and Data Exfiltration
2. Controls And Governance3
  Identity, Least Privilege, and the Supply Chain
  Guardrails, Sandboxes, and Human Approval
  Red Teaming, Monitoring, and Incident Response
3. Project And Self Assessment1
  Project: Offline Agent Threat Review
45Privacy-Enhancing Technologies Learning Path
Course overview
1. Foundations And Risks3
  Data Minimization and De-identification
  Differential Privacy Intuition and Budget
  Federated Learning and Secure Aggregation
2. Controls And Governance3
  MPC, Homomorphic Encryption, and TEEs
  PET Selection and Composition Boundaries
  Privacy Evaluation and Leak Response
3. Project And Self Assessment1
  Project: Offline Privacy Solution Review
46AI Governance Learning Path
Course overview
1. Foundations And Risks2
  System Inventory and Accountability
  Risk Tiering and Impact Assessment
2. Controls And Governance5
  Data, Model, and Vendor Governance
  Documentation, Transparency, and Traceable Evidence
  Release Approval and Change Management
  Runtime Monitoring, Incidents, and Retirement
  Regulations, Standards, and Regional Review
3. Project And Self Assessment1
  Project: Offline AI Governance Pack

09Production operations3

38MLOps Learning Path
Course overview
1. Foundations And Lifecycle3
  Experiment and Reproducibility Tracking
  Data, Features, and Training Pipelines
  Model Registry and Candidate Promotion
2. Production Engineering5
  CI, CD, and Automated Quality Gates
  Deployment, Canary, and Rollback
  Model Monitoring, Drift, and Feedback
  Incidents, Rollback, and Retraining Decisions
  Platform Security and Governance
3. Project And Self Check1
  Offline Model Promotion Project and Self-Check
40Runtime Monitoring Learning Path
Course overview
1. Foundations3
  Logs, Metrics, and Traces
  Instrumentation, Collectors, and Correlation
  SLIs, SLOs, and Error Budgets
2. Production Monitoring5
  Quality, Safety, and Cost Signals
  Alert Design and On-Call Operations
  Dashboards and Diagnostic Views
  Drift, Feedback, and Label Latency
  Incident Response and Postmortems
3. Project And Self Check1
  Offline Monitoring Audit Project and Self-Check
41LLMOps Learning Path
Course overview
1. Foundations And Lifecycle2
  Release Units and Version Lineage
  Offline Evaluation Gates and Regression Suites
2. Production Engineering6
  Gateways, Quotas, and Resilience
  Caching, Latency, and Cost
  Traces and Privacy Boundaries
  Canary, Rollback, and Change Management
  Incident Response and Continuous Improvement
  Continuous Evaluation, Provider Drift, and Governance
3. Project And Self Check1
  Offline Release Gate Project and Self-Check

10Frontier and reference1

54A2A Protocol
Course overview

AAEAI Agent Engineer

Roadmap Landmark readings Example projects Resources中文

PERSONAL PROGRESS

Account and learning progress

Account service is not configured yet. Public courses remain fully readable.

Sign in to save a role preference and course-level progress across devices.

Signed in. Learning data is private to this account by default.

Preferred role track

Delete account

This permanently deletes the account, role preference, and all course progress.

Type DELETE to confirm

Privacy notice and data controls

Home

❯

benchmark design

Folder: benchmark-design

4 items under this folder.

Aug 31, 2026
methods-and-quality
Aug 31, 2026
project-and-self-check
Aug 31, 2026
foundations-and-design
Aug 31, 2026
Benchmark Design Learning Path

4 items under this folder.

Aug 31, 2026
methods-and-quality
Aug 31, 2026
project-and-self-check
Aug 31, 2026
foundations-and-design
Aug 31, 2026
Benchmark Design Learning Path