◆ AAC&U Conference on AI and Higher Education · Atlanta · Fri Oct 30, 2026 · Innovation Session

Being users is not good enough.

Building the Agent · Jay Margalus & Mark Rush · Washington & Lee University

Mark Drawing one · how a course ran

The loop every course runs on. The grade sits on the artifact.

AssignTwo explanations of the same event. Which holds up, and why?
→
Student worksAlone, with the reading and whatever help they find.
→
The artifactEssay. Exam. Problem set.
→
FeedbackComments in the margin.

↺ feedback carries to the next artifact · the loop is the course

Mark Drawing one · what we graded, and why it worked

The essay was, and is, proof the thinking happened.

What I graded

  • An accurate account of the phenomenon, or the theory.
  • A thoughtful critique.
  • Clarity of expression.

Why that worked

  • The artifact was expensive, for both of us. Nobody wrote a competent con-law argument without doing the thinking, and careful grading took real time.
  • So the artifact stood in for the judgment, and I never had to grade judgment directly.
  • Feedback on the essay improved the next essay, if the student read past the grade.

When AI arrived I made the essay expensive again: in-class exams, blue books, oral exams. Authentic, slow, and the orals were a lottery.

Jay Then the artifact got cheap

Same loop, but the middle box stopped proving anything.

Assign
→
Student worksWith an agent that builds, writes, and sometimes submits.
→
The artifactIt exists, it looks fine, and it says nothing about the thinking.
→
FeedbackOn what, exactly?
  • A whole platform built in four days over break, with a live storefront and AI ads. Ten visitors, four signups, zero conversions.
  • An alum asked a student about security. She "could only recount the prompts."
  • A browser agent wrote a student's reflection and submitted it before she could edit it.
Jay What most campuses did about it

Ban it. Detect it.
Teach them to prompt better.

All three leave the grade on the artifact.
They're trying to make the essay expensive again.

A better user gets a better essay, and you still can't tell what they understood.

Jay The move

If the artifact no longer carries the thinking,
grade the thing that does.

The system the student built to make it.
And the record of what they decided along the way.

Jay Drawing two · how the course runs now

The student builds the system. The system makes the artifact.

Student builds the systemA rules file. Skills they wrote. Fences on what it may touch.
→
The system makes the artifactFast and cheap, and average unless the rules are good.
→
Reality reactsA real person off campus who owns the job and can say no.
→
The decision logWhat I decided, what evidence I had, what I'm still guessing.

↺ feedback becomes a line in the file the agent reads every time it starts · so the fix sticks

Jay Drawing two · what we grade, and why

Three things to grade, and none of them need you to code.

WhatHow you read itWhy it's the grade
The rules fileIt's a paragraph. Read it like a syllabus and ask why each line is there.It's the judgment, written down. "Argue with me" is a decision about what good looks like.
The logWeekly, 500 words. What did you decide, on what evidence, and what are you still guessing?"I tried it and it failed" passes. "Nothing this week" doesn't. The log is where the thinking shows.
RealityDoes it run on real information? Did the owner sign off? Twenty real inputs, five working days live.The owner doesn't care how elegant the reasoning was. They care whether it works.

Satisfactory or Needs Revision against a checklist they saw on day one. No points, because points let you skip the log and make it up with a nice build.

Jay Why the work gets better · week 4, nobody assigned this
"Argue with me. Don't agree just because I said it."

Eight of twenty first-years wrote some version of this into their agent. One of them tested it. She restarted the session, fed it a bad assumption, and checked whether it pushed back. A few sessions later the praise crept back, and she caught it.

In drawing one

That's one good essay, and next week starts from zero.

In drawing two

That's a rule on every assignment for the rest of the term. The artifact got better because the system that makes it got better.

Mark Work the class before you work the exam · POL 1005, Sept 30

The model made the summary. I taught from it.

What I do now

Group work with a twist. Sullivan and Feldman's eight AI futures (Foreign Affairs, Jan 2026). Which made more sense, and why? Groups answer in class and send me their assessments. The model finds the median, the standouts, and the strays. Five minutes instead of eight hours.

QuestionSplit
How far does AI go?9 superintelligence · 7 bounded
Can rivals catch up?13 easy · 4 hard
Is China racing?15 yes · 2 no

Not a grade yet. The class sees itself, and I run the Socratic questions off that picture.

What I want to build

An agent trained on how I read exams. It stops a student mid-essay and asks the follow-up I'd ask if I could sit beside all seventeen at once.

The Socratic method as a file. My judgment, made permanent.

Jay Where the line is · what students may hand to the agent

Always. Sometimes. Never.

Always

The thinking is already done. All that's left is the typing.

  • Scaffolding, formatting, boilerplate
  • First drafts of code
  • Transcribing the interview they ran
  • The tedious job itself, once the rule is written

Sometimes

Someone reads it before it counts.

  • Research and summaries
  • Drafting prose
  • Analysis of their own data
Two gates:
Would I know a good answer if I saw one?
Would I let a new hire do this unsupervised on day one?

Never

Judgment calls, and anything that goes out the door.

  • The decision: what to build, who it's for, whether it's good enough to ship
  • The log. It's their account of their own thinking.
  • Anything that goes out the door without a person signing off
  • Anything they can't explain when an alum asks
Jay Where to look · four checkpoints, cheapest first

Where the evaluation happens.

CheckpointWhat you doCosts you
The fileRead the rules file. Ask why each line is there.Five minutes
Monday demoThey show it running to a classmate, who signs the log as witness.Class time you already had.
The logFriday batch, same structure every week, so a student's trend shows.An evening a week
RealityThe owner's sign-off. Before-and-after numbers. Twenty real inputs.One sign-off per team

A checking agent reads each build before I do and gives the student 48 hours to fix it. It's one opinion. A build that passes the agent with a log that can't explain it still comes back.

Six weeks in: 18 of 20 running their own agent by end of week 1 · 18 of 20 with fences by week 3 · first graded builds Oct 19.

Mark One box my drawing never needed

Who answers for the act of an agent?

Jay's agents act, and they act on boring stuff. Intake forms, spreadsheets, a reflection one of them submitted before the student could edit it. Mine would act on constitutional law: obscenity, profanity, pornography. Microsoft's Tay lasted sixteen hours.

Our honor code has no clause for that, and I'd bet yours doesn't either.

In Jay's class the answer is a fence the student writes. Nothing goes out the door until a person signs off. My class isn't agentic yet. Stay tuned.

Jay No CS department required

Three things you can do Monday.

  1. Pick one assignment and grade the system they built. Have them hand you the rules file with the paper. Read it and ask why each line is there.
  2. Have them write one rule a week, in a file the agent reads every time. The first good one will be "argue with me."
  3. Make it safe to break. A folder, fake data, permission prompts on. They're going to break something. Make it the fake data.
Jay

The best AI literacy I saw this year was a rule that told the agent to argue,
and a student who noticed when it stopped.

Mark Discussion · 6 min

Your turn.

  1. Where does the grade sit in your course right now, and what did it used to prove?
  2. What's the one assignment you'd move the grade on first?
  3. Who answers when a student's agent acts on your campus: honor code, IT, or the syllabus?

Jay Margalus · jmargalus@wlu.edu · Mark Rush · Washington & Lee · slides at jaymargalus.com/aacu · start building at jaymargalus.com/agents

Credits: Seymour Papert, Mindstorms · my students, de-identified, with thanks