◆ AAC&U Conference on AI and Higher Education · Atlanta · Fri Oct 30, 2026 · Innovation Session
Being users is not good enough.
Building the Agent · Jay Margalus & Mark Rush · Washington & Lee University
MarkDrawing one · how a course ran
The loop every course runs on. The grade sits on the artifact.
AssignTwo explanations of the same event. Which holds up, and why?
→
Student worksAlone, with the reading and whatever help they find.
→
The artifactEssay. Exam. Problem set.
→
FeedbackComments in the margin.
↺ feedback carries to the next artifact · the loop is the course
MarkDrawing one · what we graded, and why it worked
The essay was, and is, proof the thinking happened.
What I graded
An accurate account of the phenomenon, or the theory.
A thoughtful critique.
Clarity of expression.
Why that worked
The artifact was expensive, for both of us. Nobody wrote a competent con-law argument without doing the thinking, and careful grading took real time.
So the artifact stood in for the judgment, and I never had to grade judgment directly.
Feedback on the essay improved the next essay, if the student read past the grade.
When AI arrived I made the essay expensive again: in-class exams, blue books, oral exams. Authentic, slow, and the orals were a lottery.
JayThen the artifact got cheap
Same loop, but the middle box stopped proving anything.
Assign
→
Student worksWith an agent that builds, writes, and sometimes submits.
→
The artifactIt exists, it looks fine, and it says nothing about the thinking.
→
FeedbackOn what, exactly?
A whole platform built in four days over break, with a live storefront and AI ads. Ten visitors, four signups, zero conversions.
An alum asked a student about security. She "could only recount the prompts."
A browser agent wrote a student's reflection and submitted it before she could edit it.
JayWhat most campuses did about it
Ban it. Detect it. Teach them to prompt better.
All three leave the grade on the artifact. They're trying to make the essay expensive again.
A better user gets a better essay, and you still can't tell what they understood.
JayThe move
If the artifact no longer carries the thinking, grade the thing that does.
The system the student built to make it. And the record of what they decided along the way.
JayDrawing two · how the course runs now
The student builds the system. The system makes the artifact.
Student builds the systemA rules file. Skills they wrote. Fences on what it may touch.
→
The system makes the artifactFast and cheap, and average unless the rules are good.
→
Reality reactsA real person off campus who owns the job and can say no.
→
The decision logWhat I decided, what evidence I had, what I'm still guessing.
↺ feedback becomes a line in the file the agent reads every time it starts · so the fix sticks
JayDrawing two · what we grade, and why
Three things to grade, and none of them need you to code.
What
How you read it
Why it's the grade
The rules file
It's a paragraph. Read it like a syllabus and ask why each line is there.
It's the judgment, written down. "Argue with me" is a decision about what good looks like.
The log
Weekly, 500 words. What did you decide, on what evidence, and what are you still guessing?
"I tried it and it failed" passes. "Nothing this week" doesn't. The log is where the thinking shows.
Reality
Does it run on real information? Did the owner sign off? Twenty real inputs, five working days live.
The owner doesn't care how elegant the reasoning was. They care whether it works.
Satisfactory or Needs Revision against a checklist they saw on day one. No points, because points let you skip the log and make it up with a nice build.
JayWhy the work gets better · week 4, nobody assigned this
"Argue with me. Don't agree just because I said it."
Eight of twenty first-years wrote some version of this into their agent. One of them tested it. She restarted the session, fed it a bad assumption, and checked whether it pushed back. A few sessions later the praise crept back, and she caught it.
In drawing one
That's one good essay, and next week starts from zero.
In drawing two
That's a rule on every assignment for the rest of the term. The artifact got better because the system that makes it got better.
MarkWork the class before you work the exam · POL 1005, Sept 30
The model made the summary. I taught from it.
What I do now
Group work with a twist. Sullivan and Feldman's eight AI futures (Foreign Affairs, Jan 2026). Which made more sense, and why? Groups answer in class and send me their assessments. The model finds the median, the standouts, and the strays. Five minutes instead of eight hours.
Question
Split
How far does AI go?
9 superintelligence · 7 bounded
Can rivals catch up?
13 easy · 4 hard
Is China racing?
15 yes · 2 no
Not a grade yet. The class sees itself, and I run the Socratic questions off that picture.
What I want to build
An agent trained on how I read exams. It stops a student mid-essay and asks the follow-up I'd ask if I could sit beside all seventeen at once.
The Socratic method as a file. My judgment, made permanent.
JayWhere the line is · what students may hand to the agent
Always. Sometimes. Never.
Always
The thinking is already done. All that's left is the typing.
Scaffolding, formatting, boilerplate
First drafts of code
Transcribing the interview they ran
The tedious job itself, once the rule is written
Sometimes
Someone reads it before it counts.
Research and summaries
Drafting prose
Analysis of their own data
Two gates: Would I know a good answer if I saw one? Would I let a new hire do this unsupervised on day one?
Never
Judgment calls, and anything that goes out the door.
The decision: what to build, who it's for, whether it's good enough to ship
The log. It's their account of their own thinking.
Anything that goes out the door without a person signing off
Anything they can't explain when an alum asks
JayWhere to look · four checkpoints, cheapest first
Where the evaluation happens.
Checkpoint
What you do
Costs you
The file
Read the rules file. Ask why each line is there.
Five minutes
Monday demo
They show it running to a classmate, who signs the log as witness.
Class time you already had.
The log
Friday batch, same structure every week, so a student's trend shows.
An evening a week
Reality
The owner's sign-off. Before-and-after numbers. Twenty real inputs.
One sign-off per team
A checking agent reads each build before I do and gives the student 48 hours to fix it. It's one opinion. A build that passes the agent with a log that can't explain it still comes back.
Six weeks in: 18 of 20 running their own agent by end of week 1 · 18 of 20 with fences by week 3 · first graded builds Oct 19.
MarkOne box my drawing never needed
Who answers for the act of an agent?
Jay's agents act, and they act on boring stuff. Intake forms, spreadsheets, a reflection one of them submitted before the student could edit it. Mine would act on constitutional law: obscenity, profanity, pornography. Microsoft's Tay lasted sixteen hours.
Our honor code has no clause for that, and I'd bet yours doesn't either.
In Jay's class the answer is a fence the student writes. Nothing goes out the door until a person signs off. My class isn't agentic yet. Stay tuned.
JayNo CS department required
Three things you can do Monday.
Pick one assignment and grade the system they built. Have them hand you the rules file with the paper. Read it and ask why each line is there.
Have them write one rule a week, in a file the agent reads every time. The first good one will be "argue with me."
Make it safe to break. A folder, fake data, permission prompts on. They're going to break something. Make it the fake data.
Jay
The best AI literacy I saw this year was a rule that told the agent to argue, and a student who noticed when it stopped.
MarkDiscussion · 6 min
Your turn.
Where does the grade sit in your course right now, and what did it used to prove?
What's the one assignment you'd move the grade on first?
Who answers when a student's agent acts on your campus: honor code, IT, or the syllabus?