UNDERTOW 018: Judgment Piecework

The same taste sells for $320,000 a year in a seat and $44 an hour in a queue. The queue is buying it so it never has to buy it again.

Anthropic is hiring a Head of Copy & Content for its Creative Studio. Twelve-plus years of experience. $320,000 to $400,000 a year. The posting describes the job as the “connective tissue between human creative vision and AI execution.” A few listings down the same careers page: Art Director, Enterprise. Ten-plus years, $305,000 to $385,000.

Now open a different tab. On Handshake AI, one of the platforms selling human expertise to the same industry, a creative writer can make up to $44 an hour. A music professional with a master’s degree tops out near $100.

Every one of these listings says it is buying taste. One market buys it by the year and gives it a seat. The other buys it by the item and gives it a queue.

The queue is bigger than most of the people standing in it know. In March, Josh Dzieza published a joint Verge and New York Magazine investigation that counted roughly thirty thousand professionals working every week through a single platform, Mercor, whose confirmed clients include OpenAI and Anthropic. The work products have names from inside the machine. Golden outputs: examples of what a perfect answer looks like. Reasoning traces: a written record of how an expert thinks through a problem, step by step. Rubrics. Adversarial stumpers: questions built to trip a model where it is weakest. Strict NDAs hide which lab any given verdict serves.

A playwright in the pipeline makes ten thousand dollars a month. The Hollywood Reporter found an editor, Gabe Sena, comparing prompts against outputs, and a former HBO development executive, Steven Woolworth, in the same pipeline. One veteran writer, speaking anonymously, asked the question that doubles as the job description: “How many times can you tell a machine it’s wrong without losing your mind?”

Dzieza’s reporting is the best map anyone has drawn of this economy. It describes the mine. This essay names the ore.

The work has no settled name yet, which is part of how it stays comfortable. Here is one. Judgment Piecework: accumulated taste, reduced to a per-click verdict, sold as a gig. In plain English: a person spends twenty years learning what good looks like. A platform pays them by the item to say yes or no to a machine’s output. The yes or no becomes a training signal. And the machine then hands that judgment to anyone, for free, forever, without the person.

Piecework is an old price. Nineteenth-century garment workers were paid by the shirt, and everyone understood one thing about the arrangement: the shirt got used up. Worn, mended, worn out, replaced. Which meant the next shirt was always worth something.

Durable pieces are not new either: a stock photograph, licensed once, competes against its maker’s next photo forever, and as the same photo sold over and over at scale, the pay per license fell from hundreds of dollars to cents. But a photo is a thing. This is the first piecework where the piece is a decision — and the buyer’s use of the decision removes the need to ask you for the next one. A stock photo competes with your future work. A captured verdict retires the question.

Two different things are being called taste right now, and the market is collapsing them into one word so it can price them.

Call the first taste-as-direction. This is the call you make inside a specific room. Five campaigns clear the research; which one can this brand still stand behind after a year? Three product directions demo well; which one survives at ten times the users? When do you say the thing nobody in the room wants said? This kind of taste cannot be lifted out of its situation. That is why nobody tries to buy it by the item. You buy the whole person and you seat them inside the situation. That is what $320,000 and “connective tissue” pays for.

Call the second taste-as-verdict. Same judgment, lifted out of the room it was made in. Is A better than B. Does this pass. Score it one through seven. Once a verdict is detached from its context, it can be logged, aggregated, and learned from. Which is why it is bought not with a seat but with a rate card: $44 an hour on Handshake, $12 to more than $200 on Mercor.

The seat stays intact. What leaves it, one verdict at a time, is everything a buyer can learn to stop paying for. And here is the part with no precedent, the rate does not fall the way wages usually fall. It holds at full price until the moment it is worth nothing — and the seller is the one being paid to bring that moment closer. Every verdict you sell shortens the distance to the last verdict anyone needs from you.

Economists will bring the complement argument here, and it agrees with this essay more than it knows. The prediction-machines school (Agrawal, Gans, and Goldfarb) says that when prediction gets cheap, judgment gets valuable, and the salaried seats and the $200 rates are exactly what the theory predicts. But look at what it has conceded. The judgment that appreciates is the kind attached to a seat, and the judgment on the rate card is being bought precisely so the buyer can learn it.

One number deserves its own paragraph. On the platforms buying expert verdicts, creative judgment prices below medicine and law. Handshake’s music expert with an advanced degree caps near $100 an hour on marketplaces where credentialed experts in law and finance clear $200 and up. Two decades of formed judgment, priced under a credential. A licensed profession can pull up the drawbridge and refuse. Taste has no licensing drawbridge, and nobody selling it seems able to stop.

The vocabulary is already circulating as flattery. A pay-to-publish Forbes Technology Council piece this month announces that the creative director is becoming a “Director of Models,” and means it as a promotion. When the same judgment that earns a promotion in the seat earns $44 an hour on the platform, “Director of Models” is a better name for a narrower job.

Does any single verdict actually end up training a model? From where the seller sits, there is no way to know. The same NDA that hides which lab bought the work hides what the work became, and a verdict that stayed a test looks identical to one that became a signal. So the honest answer at the level of one gig is: you can’t tell. The honest answer at the level of the market is: the buyers keep telling you.

They are not subtle about it. Taste Labs left stealth in June selling frontier labs what it calls preference datasets, bundles of expert opinions formatted so a model can learn from them, sourced from a vetted network it calls tastemakers, starting with design. Its backer, Amplify, wrote this essay’s thesis into its own investment memo: the scarce resource is no longer making the work but judging it, and the hard part is turning that judgment into a signal a model can learn. Mercor tells its experts the job is contributing their knowledge to the model’s training. Contra Labs recruits creatives to train models to understand taste. Four companies, one transaction, nearly the same words: buy the judgment, keep the signal it leaves behind.

And it is worth naming who, structurally, is buying. UNDERTOW 017 called it the Zero-Deductible Organization, a company that has insured its own judgment down to no retained layer, keeping none of the risk of deciding in-house. This is the other end of that transaction. The organization that laundered its nerve has to source the judgment it no longer keeps, and the verdict pipeline is where it buys it. Piecework is the supply side of 017’s demand: one essay’s seller and the other’s buyer, the same decision changing hands.

The spending is stated just as plainly. In a July post arguing for a data program the size of Stargate, OpenAI’s Will DePue puts the industry’s spend on this kind of data near $7 billion a year now, headed past $100 billion by 2030, a projection he flags as his own. The real tell sits in footnote 3, where he calls for ultra-high-quality agentic data drawn from increasingly obscure talent. The body of the post is infrastructure. The footnote is the shopping list.

The far end of the chain has to be written honestly, because it is stated intent plus trajectory, not accomplished fact. There is no documented case of a named senior creative whose sold verdicts demonstrably trained away demand for their own judgment. The same NDAs that hide the pipeline would hide the proof. What exists is the published destination (the benchmarks, the funding, the executives' own statements about the professions they're building toward).

Mercor built APEX, a benchmark in which professionals from Goldman Sachs, McKinsey, and Latham & Watkins construct tasks that test whether models can do their own high-value work. TIME’s coverage described a perfect score as “an analyst or an associate in a box.” Mercor’s chief executive, Brendan Foody, says the mission is to get comprehensive about what it means to be a consultant, a banker, a doctor, a lawyer. Bloomberg Businessweek needed no inference at all: its April headline called Mercor the ten-billion-dollar startup training AI to replace the white-collar workforce. The Hollywood Reporter walked the loop end to end. Model output scored by humans. Human scores training a reward model, a machine that grades work the way its trainers did. Reward model training the original system. Humans removed. The nearest hard number sits in an adjacent field, where a NAVA survey found about one in five voice-actor respondents had knowingly lost work to AI tools. Adjacent evidence, not proof. But when the builders publish the destination and the benchmark measures the distance remaining, this is not speculation. It is a roadmap.

The case against this essay comes in three versions, so take them in order of strength.

The first says this is simply a durable new profession. Indeed’s data shows AI roughly doubling its share of arts-category job postings in a year, from about five percent to eleven, faster than AI postings grew overall. Mercor runs thirty thousand professionals a week. Surge AI is a company of under a hundred people reportedly generating more than a billion dollars in revenue. Taste-specific vendors raised seed rounds in June. And the logic is coherent; its best spokesman is a buyer. Edwin Chen of Surge argues that the better AI gets, the more sophisticated the human judgment it needs. On that logic, expert verdicts stay supply-constrained and priced accordingly. All true. All compatible with the mechanism. The mining town also posts record employment in the decade before the seam runs out.

The second says the capture may simply fail. RLHF, the training method where human ratings teach the model what people prefer, has famous failure modes. A model can learn to flatter its trainer in evaluation and do what it wants in the wild. Even APEX’s admirers concede a perfect score would substitute poorly for open-ended professional work. Possibly. Then the buyers are misspending billions a year, the rate cards are the price of an error, and the sellers should enjoy the error while it lasts. An extraction that fails is not a profession. It is a bubble with a timesheet.

The third is depreciation, and it is the strong one, so give it its best form. Taste data rots. Great design in 2016 is not great design in 2026. The target keeps moving, so the extraction never finishes, and expert evaluation becomes a renewable service contract instead of a one-time sale. This is the real economics of the moment, and it does rescue something. It rescues the market. It does not rescue you. Once a model has absorbed this year’s distribution of your verdicts, your next verdict is worth less to the buyer than it was in January, and the platform sources the refresh from the next expert, at the next rate. The contract itself never lapses. It only keeps finding a newer name to sign on the same line. Depreciation guarantees the industry a future. It guarantees no particular person one.

Labor historians will reach for Braverman here: deskilling, the 1974 account of craft broken into tasks so cheaper workers could do them. But deskilling kept the judgment in human hands: supervised harder, paid less. This mechanism converts the judgment into a signal that removes the need for any human at that decision. Under deskilling, the wage falls. Under Judgment Piecework, it ends. The closer ancestor is Gray and Suri’s Ghost Work, which named the last mile of automation: every automated system generates temporary human work at its own frontier, then consumes it. That loop has run at the bottom of the wage scale for a decade, through moderators and labelers. What is new is the altitude. The last mile has arrived at the top of the market, and the thing in the extraction bay is not attention or stamina. It is formed taste.

This series has been watching the lower floors of the same building. Tacit Harvest, from UNDERTOW 015, named the recording of a skill its owner cannot explain: taken once, kept forever, from a worker set aside the moment the copy is good enough. The harvest took the craft the worker could not put into words. Piecework takes the opposite thing, the call the expert makes on purpose, consciously, one click at a time, invoiced. What the harvest had to steal, piecework gets you to sign for. And the reason the verdict extracts so cleanly comes from the idea of duration, named in UNDERTOW 007: Guessing at the Speed of Confidence, the time it takes for taste to become operational judgment, the willingness to stay inside a problem past the first competent answer. A career is mostly duration. A click is duration with the time cut out. The piecework format did not discover a new way to buy judgment. It found the one sliver of judgment that fits in a spreadsheet cell, and built a market on the fiction that the sliver is the whole thing.

Everything above assumes the seat is the stable side of the split. For now it is, and the reason is worth spelling out instead of assuming. Taste-as-verdict extracts because it has been stripped of context, and stripped-of-context is exactly what makes it learnable. A preference between A and B fits in a training row. Direction does not, because its input is not an artifact but a situation: this founder’s anxiety, this quarter’s politics, the objection nobody raised in the meeting. The platforms can buy the answer. They cannot yet buy the question, because knowing which question to ask is the part that never appears in the interface. That resistance is not a courtesy extended to senior people. It is the entire reason the seat pays $320,000 and the verdict pays $44.

But look at what this essay has spent two thousand words documenting: an industry funded, benchmarked, and publicly roadmapped to shrink exactly that resistance. APEX does not test whether a model can score outputs. It tests whether a model can do an analyst’s job. Foody’s list is consultants, bankers, doctors, lawyers. Which is to say: seats. So the honest statement is neither that the seat is safe nor that the seat is doomed. The seat is the frontier. It resists capture for now, its salary is the market price of that resistance, and the machinery described here exists to lower the price. The verdict was never a different product from the seat. It was the first installment.

The gig pays well, by the hour, right now. Piecework always pays best at the top of its curve, because the rate is highest exactly when the extraction is freshest. Ruth Fowler’s Wired essay in May carried the whole labor market in its title: she works in Hollywood, and many who used to make TV are now training AI. Robin Palmer, another writer in the pipeline, allowed that her peers might compare the work to crossing a picket line. The clearest sentence belongs to Tim Friedlander, who organizes voice actors, the one creative field that already has a demand-loss number on the books. It is not quite the same capture, a voice is a likeness, not a verdict, but likeness lifts fastest, so this field settles a few years ahead of the judgment version. Voice acting was seats once. Recurring characters, franchise runs, a career with a name on it. He was describing a well-paid session, the kind that looks like proof the profession is thriving. “$1,200 for four hours,” he said, “but that may be the only four hours you’re ever going to work in that job.”

That is Judgment Piecework at settlement. The rate was excellent. The rate was final.