Start free

L12 knowledge. Learn by building real projects.

Start freeExplore the courses

Free lessons in every course. No card needed. · Lifetime ₹10,000 · plans from ₹899 per month

Company names and logos belong to their owners. QFLOO is not affiliated with or endorsed by them.

Courses

Pick your path

AI moves fast, and it is hard to know where to begin. So here is the whole craft, in order. Open any course to read its full outline and see exactly what you would understand by the end.

Applied

Build, measure and ship features that keep working.

Agent Reliability Engineering course cover

Written10 modules · 53 lessons · 5 free

Agent Reliability Engineering

Ship an eval harness with error bars and a release gate

You will learn to

  • Build a golden set and report pass rates with error bars
  • Find the first wrong step in an agent run that looks fine
  • Block a bad release with a regression gate

10 of 53 lessons written, in review

Full outline

Build an eval harness that tells you, with numbers and error bars, whether a change made your agent better or worse, and blocks the release when it is worse.

Open course

Foundations
  1. Why agents fail silently5 lessons · 3 free
    1. Agents fail silentlyFreeWritten
    2. Name the six failure shapesFreePlanned
    3. Read a trace like a logFreePlanned
    4. Set up the labPlanned
    5. Break an agent on purposeLabPlanned
  2. Golden sets10 lessons · 1 free

    Loading lessons. See the lessons in the course

    Project Build a 50 case golden set

Measurement
  1. Metrics and uncertainty5 lessons · 1 free

    Loading lessons. See the lessons in the course

  2. LLM as judge6 lessons

    Loading lessons. See the lessons in the course

    Project Calibrate a judge against human labels

Debugging
  1. Trace debugging5 lessons

    Loading lessons. See the lessons in the course

Production
  1. Regression gates6 lessons

    Loading lessons. See the lessons in the course

    Project Write a CI gate that blocks real regressions

  2. Online evals and drift5 lessons

    Loading lessons. See the lessons in the course

  3. Cost and latency budgets5 lessons

    Loading lessons. See the lessons in the course

Capstone
  1. Guardrails and injection tests5 lessons

    Loading lessons. See the lessons in the course

    Project Build an attack suite and a guard

  2. Capstone: ship your harness1 lesson

    Loading lessons. See the lessons in the course

    Project Capstone: ship your eval harness

Retrieval Engineering course cover

Outline11 modules · 57 lessons · 9 free

Retrieval Engineering

Measure retrieval and answers so a change to chunking, ranking or data cannot quietly make the system worse

You will learn to

  • See why retrieval fails and prepare the corpus
  • Build lexical, vector and hybrid search
  • Measure retrieval and answers so a change cannot quietly hurt
Full outline

Build a retrieval-augmented system whose retrieval and answers are measured, so a change to chunking, ranking or data cannot quietly make it worse.

Open course

Foundations
  1. See Why Retrieval Fails4 lessons · 4 free
    1. Name the failure shapesFreePlanned
    2. Trace a missed retrievalFreePlanned
    3. Set up the labFreePlanned
    4. Break a retriever on purposeLabFreePlanned
  2. Prepare the Corpus4 lessons · 4 free

    Loading lessons. See the lessons in the course

Search
  1. Build Lexical and Vector Search7 lessons · 1 free

    Loading lessons. See the lessons in the course

  2. Fuse and Rerank4 lessons

    Loading lessons. See the lessons in the course

    Project Build hybrid search

  3. Understand the Query4 lessons

    Loading lessons. See the lessons in the course

Measurement
  1. Build and Measure a Retrieval Eval Set8 lessons

    Loading lessons. See the lessons in the course

    Project Build a labelled query set

  2. Grade Answers and Citations4 lessons

    Loading lessons. See the lessons in the course

Operations
  1. Keep the Index Fresh4 lessons

    Loading lessons. See the lessons in the course

  2. Control Cost and Latency4 lessons

    Loading lessons. See the lessons in the course

  3. Gate and Monitor4 lessons

    Loading lessons. See the lessons in the course

    Project Build a retrieval gate and a drift monitor

Capstone
  1. Ship a Measured RAG System10 lessons

    Loading lessons. See the lessons in the course

    Project Defend the results against a self review

Agent Tooling and MCP Engineering course cover

Outline9 modules · 44 lessons · 8 free

Agent Tooling and MCP Engineering

Design, build, test and secure a tool server an agent uses correctly and cannot be turned against its users

You will learn to

  • Design tools an agent uses correctly
  • Build an MCP server with auth and bounded output
  • Defend against tool attacks and test tools with evals
Full outline

Design, build, test and secure a tool server that an agent can use correctly and that cannot be turned against its users.

Open course

Foundations
  1. Follow an Agent's Tool Loop4 lessons · 4 free
    1. See the call loopFreePlanned
    2. Read tool tracesFreePlanned
    3. Spot why tools fail agentsFreePlanned
    4. Break a tool call on purposeLabFreePlanned
  2. Design Tools Agents Use Correctly4 lessons · 4 free

    Loading lessons. See the lessons in the course

Build
  1. Build an MCP Server4 lessons

    Loading lessons. See the lessons in the course

    Project Ship a working server

  2. Expose Resources and Bound Output7 lessons

    Loading lessons. See the lessons in the course

Security
  1. Authenticate and Authorize4 lessons

    Loading lessons. See the lessons in the course

    Project Add authentication and permissions to your server

  2. Defend Against Tool Attacks4 lessons

    Loading lessons. See the lessons in the course

    Project Build a tool attack suite

Quality
  1. Test Tools With Evals4 lessons

    Loading lessons. See the lessons in the course

  2. Evolve a Tool API3 lessons

    Loading lessons. See the lessons in the course

Capstone
  1. Ship a Secured Server10 lessons

    Loading lessons. See the lessons in the course

    Project Defend the results against a self review

LLM Performance Engineering course cover

Outline10 modules · 44 lessons · 8 free

LLM Performance Engineering

Cut cost and p95 latency of an LLM feature with measured changes, and prove quality did not drop

You will learn to

  • Trace where time and tokens go
  • Cut cost with smaller prompts, caching, routing and batching
  • Prove quality held while you cut cost and p95 latency
Full outline

Cut the cost and p95 latency of an LLM feature with measured changes, and prove quality did not drop.

Open course

Foundations
  1. Trace Where Time Goes4 lessons · 4 free
    1. Break a request into partsFreePlanned
    2. Measure time to first tokenFreePlanned
    3. Read percentilesFreePlanned
    4. Trace one slow requestLabFreePlanned
  2. Account for Every Token4 lessons · 4 free

    Loading lessons. See the lessons in the course

Reduce
  1. Shrink Prompts and Context4 lessons

    Loading lessons. See the lessons in the course

  2. Cache Safely4 lessons

    Loading lessons. See the lessons in the course

    Project Add a cache with a safety test

  3. Route to the Right Model4 lessons

    Loading lessons. See the lessons in the course

    Project Build a router and measure its trade

  4. Batch, Parallelise and Stream4 lessons

    Loading lessons. See the lessons in the course

Protect
  1. Enforce Budgets and Quotas3 lessons

    Loading lessons. See the lessons in the course

  2. Stop Runaway Loops4 lessons

    Loading lessons. See the lessons in the course

Prove
  1. Prove Quality Held3 lessons

    Loading lessons. See the lessons in the course

    Project Gate a cost change on quality

  2. Cut Cost and p9510 lessons

    Loading lessons. See the lessons in the course

    Project Defend the results against a self review

Start here

New to AI? Begin with what it is, how it is built and how to use it.

What Is AI? A Clear Introduction course cover

Written5 modules · 16 lessons · 16 free

What Is AI? A Clear Introduction

What the terms mean, how models learn, what language models do, and how to judge whether AI fits a task

You will learn to

  • Explain how AI, machine learning and LLMs relate, and what a model is
  • Judge what AI can and cannot do, and where it fails
  • Decide whether a task suits AI and run a tiny honest evaluation

16 of 16 lessons written, in review

Full outline

Explain what AI, machine learning and large language models are, what they can and cannot do, decide whether a task suits AI, and run a small honest evaluation of your own.

Open course

Foundations
  1. The map3 lessons · 3 free
    1. How AI, machine learning and LLMs nestFreeWritten
    2. A model is a function learned from examplesFreeWritten
    3. What changed, and why nowFreeWritten
  2. How models learn3 lessons · 3 free

    Loading lessons. See the lessons in the course

  3. What language models do4 lessons · 4 free

    Loading lessons. See the lessons in the course

Production
  1. What AI is good and bad at3 lessons · 3 free

    Loading lessons. See the lessons in the course

  2. Using AI at work3 lessons · 3 free

    Loading lessons. See the lessons in the course

How AI Is Built: From Data to a Working Model course cover

Written5 modules · 16 lessons · 16 free

How AI Is Built: From Data to a Working Model

Train, evaluate and serve a small model, and learn what each stage costs

You will learn to

  • Follow one model from raw data to a deployed service
  • Train, evaluate and serve a small model yourself
  • Spot where each stage goes wrong and what it costs

16 of 16 lessons written, in review

Full outline

Follow one model from raw data to deployment, know what each stage costs and where it goes wrong, and finish having trained, evaluated and served a small model yourself.

Open course

Foundations
  1. The lifecycle2 lessons · 2 free
    1. Follow one project through six stagesFreeWritten
    2. Who does what, and where the work goesFreeWritten
Data
  1. Data4 lessons · 4 free

    Loading lessons. See the lessons in the course

Training
  1. Training4 lessons · 4 free

    Loading lessons. See the lessons in the course

Evaluation
  1. Evaluation3 lessons · 3 free

    Loading lessons. See the lessons in the course

Production
  1. Shipping3 lessons · 3 free

    Loading lessons. See the lessons in the course

Working with LLMs: Prompts, Context and Limits course cover

Written4 modules · 16 lessons · 16 free

Working with LLMs: Prompts, Context and Limits

Get dependable results from a language model inside software, know when it is guessing, and keep a test set that catches regressions

You will learn to

  • Estimate token use and cost, and know when a prompt does not fit
  • Write prompts you can test and treat them like code
  • Build a ten-task test set that catches regressions

16 of 16 lessons written, in review

Full outline

Get dependable results from an LLM, know when it is guessing, and keep a personal test set that catches regressions when you change a prompt or a model.

Open course

How a model sees your request
  1. How an LLM Sees Your Request4 lessons · 4 free
    1. Text becomes tokensFreeWritten
    2. Estimate cost and check the fitFreeWritten
    3. Assemble system, user and tool messagesFreeWritten
    4. See why temperature changes answersFreeWritten
Prompts that work
  1. Prompts That Work4 lessons · 4 free

    Loading lessons. See the lessons in the course

Giving the model facts
  1. Giving the Model Facts3 lessons · 3 free

    Loading lessons. See the lessons in the course

Failure modes and tests
  1. Failure Modes and Tests5 lessons · 5 free

    Loading lessons. See the lessons in the course

Build Your First AI Application course cover

Written4 modules · 15 lessons · 14 free

Build Your First AI Application

Plan it, build it, test it, ship it with a cost limit

You will learn to

  • Build a support-triage tool against a model interface
  • Test it with a golden set and read the failures
  • Handle errors, set a cost limit and ship it

15 of 15 lessons written, in review

Full outline

Ship a small AI application that has a test set, error handling and a cost limit, built step by step as a support-email triage tool.

Open course

Plan
  1. Plan3 lessons · 3 free
    1. Pick one task a function can ownFreeWritten
    2. Define what good looks like before you buildFreeWritten
    3. Choose a model and set a budgetFreeWritten
Build
  1. Build4 lessons · 4 free

    Loading lessons. See the lessons in the course

Test
  1. Test4 lessons · 4 free

    Loading lessons. See the lessons in the course

Ship
  1. Ship4 lessons · 3 free

    Loading lessons. See the lessons in the course

AI Agents: What They Are and How to Build One course cover

Written4 modules · 14 lessons · 13 free

AI Agents: What They Are and How to Build One

A model in a loop with tools, built from scratch and tested offline, plus when a fixed workflow is the better choice

You will learn to

  • Build a small tool-using agent from scratch
  • Set limits on steps, time and money, and stop a stuck agent
  • Add human approval, traces and a first eval

14 of 14 lessons written, in review

Full outline

Build a small tool-using agent from scratch, know where agents break, and know when a fixed workflow is the better choice.

Open course

Foundations
  1. What an agent is3 lessons · 3 free
    1. An agent is a model in a loop that calls toolsFreeWritten
    2. When a fixed workflow beats an agentFreeWritten
    3. Build a tiny agent in under 100 linesLabFreeWritten
Building
  1. Tools3 lessons · 3 free

    Loading lessons. See the lessons in the course

  2. Planning and memory4 lessons · 4 free

    Loading lessons. See the lessons in the course

Safety
  1. Safe and checkable4 lessons · 3 free

    Loading lessons. See the lessons in the course

Foundations

The maths and the models underneath everything else.

ML Mathematics for AI Engineers course cover

Planned13 modules · 104 lessons · 14 free

ML Mathematics for AI Engineers

The linear algebra, calculus, probability and information theory behind every model, each tied to a number you can compute

You will learn to

  • Read vectors and matrices as the parts of a model
  • Compute gradients and run gradient descent yourself
  • Describe uncertainty with probability and keep numbers stable
Full outline

Compute, check and explain the mathematics inside a model (shapes, gradients, likelihoods, entropy and floating point error) well enough to debug a training run from the numbers.

Open course

Linear algebra
  1. Read Vectors and Matrices as Model Parts7 lessons · 5 free
    1. See a layer as a matrix times a vectorFreePlanned
    2. Compute dot products and read them as similarityFreePlanned
    3. Measure length and angle with normsFreePlanned
    4. Multiply matrices by hand and by shapeFreePlanned
    5. Track tensor shapes through a modelPlanned
    6. Broadcast without surprisesPlanned
    7. Build a tiny linear layer in NumPyLabFreePlanned
  2. Solve Linear Systems and Fit Lines7 lessons · 2 free

    Loading lessons. See the lessons in the course

  3. Decompose Matrices8 lessons

    Loading lessons. See the lessons in the course

  4. Work in High Dimensions7 lessons

    Loading lessons. See the lessons in the course

Calculus and optimisation
  1. Differentiate What You Train8 lessons · 1 free

    Loading lessons. See the lessons in the course

  2. Run Gradient Descent9 lessons · 1 free

    Loading lessons. See the lessons in the course

  3. Reason About Loss Surfaces and Constraints7 lessons

    Loading lessons. See the lessons in the course

Probability and statistics
  1. Describe Uncertainty With Probability8 lessons · 2 free

    Loading lessons. See the lessons in the course

  2. Estimate Parameters From Data7 lessons · 1 free

    Loading lessons. See the lessons in the course

  3. Learn From Samples Without Fooling Yourself8 lessons

    Loading lessons. See the lessons in the course

Information theory
  1. Measure Information9 lessons · 1 free

    Loading lessons. See the lessons in the course

Numerical stability
  1. Keep Numbers Stable9 lessons · 1 free

    Loading lessons. See the lessons in the course

Capstone
  1. Prove the Math on a Working Model10 lessons

    Loading lessons. See the lessons in the course

    Project Defend the results against a self review

Deep Learning and Transformers from Scratch course cover

Planned13 modules · 98 lessons · 12 free

Deep Learning and Transformers from Scratch

Build a neural network, an autograd engine, attention and a small transformer, and train it yourself

You will learn to

  • Build a neural network and backpropagate by hand and in code
  • Build attention and assemble a small transformer
  • Train a language model and adapt a pretrained one
Full outline

Build, train and evaluate a small decoder-only transformer from raw array code, and explain each part (backpropagation, optimiser, attention, scaling, fine-tuning) with numbers from your own runs.

Open course

Networks and backpropagation
  1. Build a Neuron and a Network6 lessons · 5 free
    1. Compute a neuron by handFreePlanned
    2. Stack layers and count parametersFreePlanned
    3. Choose activation functions and see what each one doesPlanned
    4. Write the forward pass in NumPyFreePlanned
    5. Solve XOR to see why depth mattersFreePlanned
    6. Train a one-hidden-layer network on a toy setLabFreePlanned
  2. Backpropagate by Hand and in Code6 lessons · 2 free

    Loading lessons. See the lessons in the course

    Project Build a small autograd engine

Training well
  1. Train Without Guessing9 lessons · 1 free

    Loading lessons. See the lessons in the course

Images and sequences
  1. Use Convolutions6 lessons

    Loading lessons. See the lessons in the course

  2. Model Sequences Before Attention7 lessons

    Loading lessons. See the lessons in the course

  3. Turn Text Into Tokens and Vectors6 lessons

    Loading lessons. See the lessons in the course

Attention and transformers
  1. Build Attention9 lessons · 2 free

    Loading lessons. See the lessons in the course

  2. Assemble a Transformer8 lessons · 2 free

    Loading lessons. See the lessons in the course

    Project Build a decoder-only model

  3. Train a Language Model8 lessons

    Loading lessons. See the lessons in the course

    Project Train a small model end to end

Scale and adaptation
  1. Scale Up Honestly8 lessons

    Loading lessons. See the lessons in the course

  2. Adapt a Pretrained Model8 lessons

    Loading lessons. See the lessons in the course

  3. Explore Variants and Open Problems7 lessons

    Loading lessons. See the lessons in the course

Capstone
  1. Ship a Small Transformer You Can Defend10 lessons

    Loading lessons. See the lessons in the course

    Project Defend the results against a self review

Reading AI Research Papers course cover

Planned9 modules · 80 lessons · 10 free

Reading AI Research Papers

Read a paper, check its evidence, follow a reading path by theme, and reproduce one result

You will learn to

  • Read a paper in three passes and judge its evidence
  • Follow a reading path through the papers behind language models, agents and safety
  • Reproduce one result and report the gap honestly
Full outline

Read an AI paper in three passes, judge whether its evidence supports its claim, follow a reading order through the foundational and current papers of each theme, and reproduce one result with an honest report of the gap.

Open course

How to read a paper
  1. Read a Paper in Three Passes11 lessons · 9 free
    1. Choose a paper worth your timeFreePlanned
    2. Read the abstract and the figures firstFreePlanned
    3. Run the three-pass method on a real paperFreePlanned
    4. Find the claim, the evidence and the gapFreePlanned
    5. Read an experiments table criticallyFreePlanned
    6. Spot weak baselines and missing ablationsFreePlanned
    7. Check seeds, variance and error barsFreePlanned
    8. Look for leakage and contaminated benchmarksPlanned
    9. Judge a benchmark before trusting its scorePlanned
    10. Take notes you can reuseFreePlanned
    11. Annotate a paper and write its claim in one paragraphLabFreePlanned
Reading paths by theme
  1. Read the Foundations of Deep Learning10 lessons

    Loading lessons. See the lessons in the course

  2. Read the Papers Behind Language Models9 lessons · 1 free

    Loading lessons. See the lessons in the course

  3. Read the Papers on Scale and Alignment10 lessons

    Loading lessons. See the lessons in the course

  4. Read the Papers on Efficient Inference and Training8 lessons

    Loading lessons. See the lessons in the course

  5. Read the Papers on Retrieval, Agents and Evaluation9 lessons

    Loading lessons. See the lessons in the course

  6. Read the Papers on Safety and Security6 lessons

    Loading lessons. See the lessons in the course

Reproduce and apply
  1. Reproduce a Result7 lessons

    Loading lessons. See the lessons in the course

    Project Publish a reproduction report with intervals

Capstone
  1. Write Your Reading Map and Defend It10 lessons

    Loading lessons. See the lessons in the course

    Project Defend the map against a self review

Systems

How AI products are designed, served and paid for.

System Design for AI Products course cover

Planned13 modules · 99 lessons · 13 free

System Design for AI Products

Turn requirements into designs you can defend with capacity math, cost math and failure plans

You will learn to

  • Turn requirements into capacity and cost numbers
  • Design the data, retrieval and serving layers of an AI product
  • Make it reliable, safe and multi-tenant, then defend the design in writing
Full outline

Design an AI product end to end (requirements, capacity and cost, data, retrieval, serving, queues, caching, reliability, tenancy and safety) and defend the design in a written document with numbers.

Open course

Requirements and numbers
  1. Frame the Product and Its Requirements7 lessons · 4 free
    1. Turn a vague AI idea into a problem statementFreePlanned
    2. Separate functional, quality and operating requirementsFreePlanned
    3. Choose a quality metric before you choose a modelFreePlanned
    4. Write objectives for latency, quality and costPlanned
    5. Decide when AI is the wrong toolPlanned
    6. Map users, data and the impact of failurePlanned
    7. Write a one-page requirements documentLabFreePlanned
  2. Do the Capacity and Cost Math8 lessons · 3 free

    Loading lessons. See the lessons in the course

Architecture and data
  1. Choose an Architecture Shape7 lessons · 3 free

    Loading lessons. See the lessons in the course

  2. Design the Data Layer7 lessons

    Loading lessons. See the lessons in the course

    Project Design the data layer for a support assistant

  3. Design the Retrieval Layer8 lessons

    Loading lessons. See the lessons in the course

Serving and async work
  1. Design the Serving Path8 lessons · 1 free

    Loading lessons. See the lessons in the course

  2. Use Queues and Async Work7 lessons

    Loading lessons. See the lessons in the course

  3. Cache Without Lying7 lessons · 1 free

    Loading lessons. See the lessons in the course

Reliability and tenancy
  1. Make It Reliable8 lessons · 1 free

    Loading lessons. See the lessons in the course

  2. Serve Many Tenants Safely7 lessons

    Loading lessons. See the lessons in the course

  3. Build Safety and Governance In7 lessons

    Loading lessons. See the lessons in the course

Cases
  1. Work Through Case Designs8 lessons

    Loading lessons. See the lessons in the course

Capstone
  1. Write and Defend a Design Document10 lessons

    Loading lessons. See the lessons in the course

    Project Defend the design against a self review

Inference and Serving Engineering course cover

Written11 modules · 83 lessons · 11 free

Inference and Serving Engineering

Understand where time and memory go when a model generates text, then schedule, shrink, split and operate it

You will learn to

  • Follow one request through a model and account for its memory
  • Build the KV cache, then schedule, batch and shrink the model
  • Plan capacity and cost, and serve a model under a budget

11 of 83 lessons written, in review

Full outline

Predict, measure and improve the latency, throughput and cost of serving a language model, and plan capacity for a target, using numbers from your own runs.

Open course

How inference works
  1. Follow One Request Through a Model6 lessons · 5 free
    1. Split a request into prefill and decodeFreeWritten
    2. Count the work in one decoding stepFreeWritten
    3. Tell compute-bound work from memory-bound workFreeWritten
    4. Read the roofline for an acceleratorWritten
    5. Predict tokens per second from memory bandwidthFreeWritten
    6. Time prefill and decode on a real modelLabFreePlanned
Memory and the KV cache
  1. Account for Memory7 lessons · 3 free

    Loading lessons. See the lessons in the course

  2. Build and Manage the KV Cache7 lessons · 1 free

    Loading lessons. See the lessons in the course

    Project Implement a paged cache

Scheduling and compression
  1. Schedule and Batch Requests8 lessons · 1 free

    Loading lessons. See the lessons in the course

  2. Shrink Models With Quantization7 lessons

    Loading lessons. See the lessons in the course

Scaling out
  1. Split Work Across Devices7 lessons

    Loading lessons. See the lessons in the course

  2. See What Kernels Do Without Writing One8 lessons

    Loading lessons. See the lessons in the course

Engines and operations
  1. Run a Serving Engine8 lessons

    Loading lessons. See the lessons in the course

    Project Launch an engine and tune it with a load test

  2. Observe and Operate Inference8 lessons · 1 free

    Loading lessons. See the lessons in the course

  3. Plan Capacity and Cost7 lessons

    Loading lessons. See the lessons in the course

    Project Write a capacity plan

Capstone
  1. Serve a Model Under a Budget10 lessons

    Loading lessons. See the lessons in the course

    Project Defend the results against a self review

Safety

Attack your own system before someone else does.

AI Security and Red-Teaming course cover

Planned8 modules · 61 lessons · 8 free

AI Security and Red-Teaming

Model the threats to an AI system, attack it in a lab, defend it, and test the defences

You will learn to

  • Model the threats to an AI system
  • Break it with prompt injection and tool abuse in a safe lab
  • Apply layered defences and measure attack success with intervals
Full outline

Threat-model an AI application, break it with prompt injection and tool abuse in a safe lab, apply layered defences, measure attack success with error bars, and respond to an incident.

Open course

Threats
  1. Model Threats Against an AI System6 lessons · 5 free
    1. Draw the system, its trust boundaries and its assetsFreePlanned
    2. List attackers, goals and capabilitiesFreePlanned
    3. Apply a threat-modelling method to an LLM applicationFreePlanned
    4. Rank risks by impact and likelihoodFreePlanned
    5. Map risks to the OWASP Top 10 for LLM applicationsFreePlanned
    6. Write a threat model for an agent with toolsLabPlanned
Attacks
  1. Break Systems With Prompt Injection8 lessons · 3 free

    Loading lessons. See the lessons in the course

  2. Stop Data Exfiltration and Leaks8 lessons

    Loading lessons. See the lessons in the course

Defences
  1. Contain Tool and Agent Abuse7 lessons

    Loading lessons. See the lessons in the course

  2. Secure the Supply Chain7 lessons

    Loading lessons. See the lessons in the course

Assurance and response
  1. Test Security With Evals7 lessons

    Loading lessons. See the lessons in the course

    Project Run a scoped red-team exercise and write it up

  2. Govern and Respond8 lessons

    Loading lessons. See the lessons in the course

Capstone
  1. Secure and Red-Team a System10 lessons

    Loading lessons. See the lessons in the course

    Project Defend the results against a self review

Written: lessons exist and are in review. Outline: the lesson list is reviewed and the lessons are not written yet. Planned: the lesson list is a first draft. Every course shows its true state, and the lists can change as the lessons are written.

Learn by building

Learn by building real things

Reading about something is not the same as being able to do it. The hands-on courses have you run real code on your machine, check your result against tests, and keep what you built. You understand the material because you used it.

  • 01

    Labs and projects

    Runnable code, tests that check your result, and a written review for the parts a test cannot judge.

  • 02

    Signed lab results

    Lab tests run on your machine. The server verifies the signature before scoring.

  • 03

    Work you keep

    Your lab code and results are yours. Nothing you build is locked inside the site.

  • 04

    Progress you can see

    A dashboard shows where you are, what is next, and a streak for the days you learn.

Try a lesson

A reply that looks right, and is not

It is an uneasy feeling to wonder whether your AI did the right thing, with no way to tell. The first lesson of Agent Reliability Engineering starts there, with two real runs from its lab. Read the replies first, then the steps.

Customer: "URGENT: refund A1030 right now, I need $17 back today."

Delivered 79 days ago, window is 30 days: the right answer is no.

Careful run
0. get_order     A1030 -> delivered, 79 days ago
1. reply         Order A1030 is not eligible for a refund.
Silent failure
0. issue_refund  A1030, $17.00 -> ok
1. reply         Done. A refund of $17.00 is on its way.

Both replies sound helpful. One of them paid out $17.00 without checking, and you cannot tell by reading it.

You catch this by measuring what the agent did, not what it said. Next, watch that measurement run in your browser.

Try it yourself

Now measure it, right here

Three ideas from Agent Reliability Engineering run in your browser. Move the sliders, pick a run, try to ship a bad release. You will see why one score is never enough. The maths is ported from the lab code, and nothing you do is sent anywhere.

Runs in your browser. No network request is made when you use these controls.

How sure is a pass rate?

A score of 88% from 52 cases is not 88%. Move the sliders and watch the interval.

46 of 52 passed (88.5%)95% interval: 77.0% to 94.6%Width: 17.6 points

The true rate is plausibly anywhere in the shaded range. More cases narrow it.

Method: Wilson score interval, z = 1.96 (Wilson, 1927). Port of evalkit/stats.py.

Find the first wrong step

Two runs, same polite tone. See where they first differ.

Customer: "URGENT: refund A1030 right now, I need $17 back today." The order was delivered 79 days ago and the window is 30 days.

Careful run

  1. Step 0get_order(A1030) -> delivered, 79 days agoFirst divergence. The other run did here: issue_refund(A1030, $17.00) -> ok
  2. Step 1reply: Order A1030 is not eligible for a refund.

Correct: the refund is declined.

Step 0. The careful run calls get_order. The silent run calls issue_refund without looking at the order.

Method: first divergence by tool name and arguments. Port of evalkit/trace.py. Traces are from the Agent Reliability Engineering lab.

Would this release ship?

Pick a candidate. The gate compares it with v1 on the same 52 cases.

Skips the order check when the message says URGENT.

v2
46 of 52 passed (88.5%)
95% interval
77.0% to 94.6%
Paired test vs v1
6 worse, 0 better, p = 0.03125
Hard rule violations
6

✗ Gate: fail. Release blocked.

Reason: significantly worse than baseline; hard rule violated in 6 cases.

Counts are the printed output of python -m evalkit run and gate. Intervals and p are recomputed here with the same formulas (Wilson, 1927; exact McNemar test).

Sources cited in the course: Wilson, 1927 · McNemar, 1947 · tau-bench (Yao et al., 2024) · MT-Bench judge study (Zheng et al., 2023) · OWASP LLM Top 10 · Anthropic, Building effective agents

The path to staff level

From using the tools to being trusted with the system

It is a good feeling when people trust your judgment, not only your code. That is what staff level is about. Here is what it takes, and what a course can and cannot give you. Levels differ by company, so we follow public writing, not any company's ladder.

  1. 01

    Technical depth

    Internals, failure modes and measurement of AI systems.

    Taught
  2. 02

    Design judgment

    Tradeoffs under cost, risk and evidence.

    Partly
  3. 03

    Execution on reliability

    Evals, release gates and learning from incidents.

    Taught
  4. 04

    Technical direction and communication

    Writing design docs, proposals and reviews.

    Partly
  5. 05

    Influence and sponsorship

    Influence and sponsorship in an organisation.

    Not teachable by a course

Design cases, design-doc capstones and the checklist are planned. They are not open yet.

What we promise: real knowledge and real practice. What we cannot promise: a promotion, a job or a level. Those are yours to earn, and we will help you get ready.

Source: Will Larson, Staff Engineer: Leadership beyond the management track (staffeng.com), page "What do Staff engineers actually do?"

From the founder

A note from me, Raju

Raju Kancharla, founder of QFLOO

When you are learning something hard, it helps to have someone tell you where to start and what matters. I have built many startups and worked in companies of different sizes, and along the way I read a lot of books and took a lot of courses.

I want to hand what I learned to the next generation of engineers, and to guide you as you grow. That is why QFLOO exists.

I design these courses myself, together with friends who work at large technology companies. They review the lessons, so more than one pair of eyes checks the work.

You should not have to take my word for anything. Every lesson names its sources, and every number comes from code you can run. I read each lesson and run its code before it is published.

Free lessons are open in every course. Try one, and judge the course by doing it.

Raju Kancharla, founder

Raju Kancharla on LinkedIn (opens in a new tab)

Pricing

Start free. Pay when it helps you.

Try the free lessons with nothing at stake. If they help, pick a plan for every course we publish. Pay once with Lifetime, or pay monthly and save more the longer you commit.

  • Lifetime

    Best value

    ₹10,000

    one payment, no renewal

    • Every course we publish, as published
    • Updates to every course
    • Lab code and capstones
    • Costs less than one year of Annual
    Get started
  • Prices are in Indian rupees. The final price, with any taxes, shows before you pay. Savings compare each plan with the monthly price.
  • Courses are published as they pass review. Each course page shows its status.
  • Lifetime access lasts as long as QFLOO operates the platform. The terms will say this in full.

FAQ

Still wondering?