Journal/Postmortem/Notes from a quoting automation rollout.
Notes from a quoting automation rollout.
What eleven weeks inside a mid-sized fabricator taught us about evals, change management, and why the receptionist always knows first.
Published
Mar 04, 2026
Reading time
14 minutes
Category
Postmortem
Fig. 06 · Object OBJ-0304Trajectory: 11 weeks
This is a sanitized account of an eleven-week engagement. Names, sectors, and numbers have been changed enough that the firm cannot be identified, and any client we work with is welcome to read this without finding themselves in it. The lessons are the part that travels.
The wedge was simple: incoming requests for quotation arrived in three places — email, a web form, and the phone. Each one was re-keyed into a quoting tool by hand, by one of three people, with answers turned around in twenty-four to seventy-two hours. The CFO wanted the turnaround under four. We agreed that was achievable. Eleven weeks later, average turnaround was sixteen minutes. Here is what we learned along the way.
01. Week one was not what we expected
We arrived ready to talk about models, retrieval, and the rule book the team used to price jobs. The first useful conversation happened with the receptionist.
She knew, before anyone in operations, which incoming requests were "easy" — repeats from existing customers, standard part numbers, predictable margins — and which were "hard" — new customers, custom specs, anything urgent. The pricing team had never asked her. We did, and she gave us, in a single afternoon, a more accurate triage taxonomy than anything in the existing CRM.
Every operations diagnostic ends in the same place: the person closest to the door knows things the org chart does not.
— project journal, week one
This is the first lesson of the rollout, and it is not a technical one. Talk to the people who handle the inputs first. They have been compressing your business into mental models for years. Those mental models are the eval set you are about to need.
02. The eval was boring and saved us twice
Before any model touched a real RFQ, we built a set of forty historical quotes — selected with the receptionist and the head of pricing — with their actual outcomes. The agent had to produce a quote within a tolerance of the historical answer. Below tolerance: it could ship. Above: it could not.
The eval caught two regressions we would otherwise have shipped. The first was a category of stainless-steel jobs where the agent confidently underquoted by a number that would have, over a quarter, eaten more margin than the project saved. The second was a class of urgent jobs the agent priced at standard rate because it had no notion of urgency from the input fields.
Without the eval, both would have launched. With the eval, both surfaced in week six and were patched before the rollout.
03. The hardest part was not the model
Once the agent worked — at week seven, comfortably — we hit the part of the project nobody had budgeted for: change management. The pricing team did not initially trust the system. This is rational. They had been priced wrong by humans for years; now a machine was doing it.
We solved this in three steps, none of which were technical:
The agent's outputs were drafts, not decisions. Every quote had a "send" button that required a human to press it. For four weeks, that button was the project.
We built a one-screen audit trail. Every quote the agent produced was visible alongside the chunks of the rulebook it had used. The pricing lead could spot-check ten a day in fifteen minutes.
We let the team override and log the override. Each override became an eval case. The agent improved, fast, on the cases the humans cared most about.
By week nine, the pricing team was sending agent drafts unedited for the easy categories and reviewing only the hard ones. By week eleven, sixteen-minute turnaround was the operating reality.
04. The two things that almost killed it
In a project like this, the obvious risks — model drift, hallucination, integration failure — are not what kill the rollout. The two near-misses we had were:
The CFO's deadline slipped early. A board meeting was moved up by three weeks, and a "pilot" became a "result we need to report." We pushed back hard, and the CFO held the line. If she hadn't, the project would have shipped without the eval and we would be writing a different postmortem.
The pricing lead nearly left. Four weeks in, she was unsure whether the project was a help or a quiet plan to replace her. Her manager handled this — directly, in a one-on-one. The lesson is not that you should manage your people well. It is that you should budget for the conversation, because it is going to happen, and avoiding it is the failure mode.
Working principle: the people closest to the workflow you are automating need to be told — early, plainly — what changes for them. The conversation is part of the rollout, not a thing you do after.
05. What the number actually meant
Sixteen minutes was the headline. The CFO got her board win. But the numbers we cared more about, internally, were:
The receptionist's afternoon. It came back. She stopped re-keying RFQs and started, of her own initiative, flagging customer behavior the CRM did not capture. That is now a separate, smaller wedge.
Pricing team headcount. Did not change. The hours they used to spend re-keying went to harder pricing decisions on the jobs that needed judgment.
Margin variance on stainless-steel jobs. Tightened by three points. Almost entirely because of the regression caught by the eval in week six.
The headline number ships the project. The shadow numbers are why the project was worth doing.
A short closing
If you are about to run a project like this one, the things we wish we had known are not technical. They are:
Talk to the people closest to the input first. They know more than the org chart says.
Build the eval set with them, not separately.
Treat the change-management conversation as a deliverable, not a side effect.
The shadow numbers — the time given back, the variance tightened — are what compound.
The model was the easiest week of the project. The other ten weeks were people, process, and the unfashionable habits that make a rollout survive.
Ovo je sanirana priča o jedanaestonedeljnom angažmanu. Imena, sektori i brojevi dovoljno su izmenjeni da se firma ne može identifikovati, i svaki klijent sa kojim radimo dobrodošao je da ovo pročita ne pronalazeći sebe u tekstu. Lekcije su deo koji putuje.
Zahvat je bio jednostavan: dolazni zahtevi za ponudu stizali su na tri mesta — email, web forma, telefon. Svaki je bio ručno prebačen u alat za izradu ponuda, od strane jedne od tri osobe, sa odgovorom unutar dvadeset četiri do sedamdeset dva sata. CFO je htela vreme odgovora ispod četiri sata. Složili smo se da je to izvodljivo. Jedanaest nedelja kasnije, prosečno vreme odgovora bilo je šesnaest minuta. Evo šta smo naučili na putu.
01. Prva nedelja nije bila ono što smo očekivali
Stigli smo spremni da pričamo o modelima, preuzimanju i o knjizi pravila po kojoj je tim određivao cene. Prvi koristan razgovor desio se sa recepcionerkom.
Znala je, pre bilo koga u operacijama, koji su dolazni zahtevi „laki" — ponovljene narudžbine od postojećih klijenata, standardni brojevi delova, predvidive marže — a koji „teški" — novi klijenti, prilagođene specifikacije, bilo šta hitno. Tim za određivanje cena je nije pitao. Mi jesmo, i dala nam je za jedno popodne tačniju triažnu taksonomiju nego sve što je bilo u postojećem CRM-u.
Svaka operativna dijagnostika završi se na istom mestu: osoba najbliža vratima zna stvari koje organizaciona šema ne zna.
— projektni dnevnik, prva nedelja
Ovo je prva lekcija puštanja, i nije tehnička. Razgovarajte prvo sa ljudima koji obrađuju ulaze. Oni godinama kompresuju vaše poslovanje u mentalne modele. Ti mentalni modeli su skup za evaluaciju koji vam upravo treba.
02. Evaluacija je bila dosadna i spasila nas dvaput
Pre nego što je ijedan model dotakao stvaran zahtev, napravili smo set od četrdeset istorijskih ponuda — odabranih sa recepcionerkom i šeficom cenovne politike — sa stvarnim ishodima. Agent je morao da proizvede ponudu u toleranciji u odnosu na istorijski odgovor. Ispod tolerancije: može da ide. Iznad: ne može.
Evaluacija je uhvatila dve regresije koje bismo inače pustili. Prva je bila kategorija poslova sa nerđajućim čelikom gde je agent samouvereno potcenjivao cenu za broj koji bi, kroz kvartal, pojeo više marže nego što je projekat uštedeo. Druga je bila klasa hitnih poslova koje je agent procenjivao po standardnoj tarifi jer nije imao pojam o hitnosti iz ulaznih polja.
Bez evaluacije, oboje bi bilo pušteno. Sa evaluacijom, oboje je izronilo u šestoj nedelji i zakrpljeno pre puštanja.
03. Najteži deo nije bio model
Kada je agent proradio — u sedmoj nedelji, udobno — naišli smo na deo projekta koji niko nije budžetirao: upravljanje promenom. Cenovni tim na početku nije verovao sistemu. Ovo je racionalno. Bili su pogrešno tarifirani od strane ljudi godinama; sada to radi mašina.
Rešili smo to u tri koraka, nijedan tehnički:
Izlazi agenta bili su nacrti, ne odluke. Svaka ponuda imala je dugme „pošalji" koje je zahtevalo da ga čovek pritisne. Četiri nedelje, to dugme je bilo projekat.
Napravili smo jednostran trag revizije. Svaka ponuda koju je agent proizveo bila je vidljiva uz delove pravilnika koje je koristio. Vodja cenovne politike mogao je da uradi nasumičnu proveru deset ponuda dnevno za petnaest minuta.
Pustili smo tim da nadjača i zabeležimo nadjačavanje. Svako nadjačavanje postalo je slučaj evaluacije. Agent se brzo poboljšavao na slučajevima do kojih je ljudima najviše bilo stalo.
Do devete nedelje, cenovni tim je slao nacrte agenta neizmenjene za lake kategorije i pregledao samo teške. Do jedanaeste nedelje, šesnaest minuta je bila operativna stvarnost.
04. Dve stvari koje su skoro ubile projekat
U projektu kao što je ovaj, očigledni rizici — drift modela, halucinacije, neuspeh integracije — nisu ono što ubije puštanje. Dva bliska promašaja koja smo imali bili su:
CFO-in rok je rano otklizao. Sednica odbora pomerena je tri nedelje ranije, i „pilot" je postao „rezultat koji moramo da prijavimo". Snažno smo se odupirali, i CFO je držala liniju. Da nije, projekat bi bio pušten bez evaluacije i pisali bismo drugačiji postmortem.
Vođa cenovne politike skoro je dala otkaz. Četiri nedelje unutra, nije bila sigurna da li je projekat pomoć ili tihi plan da je zamene. Njen menadžer je to rešio — direktno, u jednoj-na-jednu. Lekcija nije da treba da dobro vodite svoje ljude. Lekcija je da treba da budžetirate taj razgovor, jer će se desiti, a izbegavanje je način otkazivanja.
Radni princip: ljudima najbližima toku rada koji automatizujete treba reći — rano, otvoreno — šta se za njih menja. Razgovor je deo puštanja, ne nešto što radite posle.
05. Šta je broj zapravo značio
Šesnaest minuta je bio naslov. CFO je dobila pobedu pred odborom. Ali brojevi do kojih nam je interno više bilo stalo bili su:
Recepcionerkino popodne. Vratilo se. Prestala je da preunosi zahteve i počela je, na sopstvenu inicijativu, da označava ponašanje klijenata koje CRM nije hvatao. To je sada zaseban, manji zahvat.
Broj zaposlenih u cenovnom timu. Nije se promenio. Sati koje su provodili u preunošenju otišli su na teže odluke o cenama na poslovima kojima je trebala procena.
Varijansa marže na poslovima sa nerđajućim čelikom. Sužena za tri poena. Skoro u potpunosti zbog regresije koju je evaluacija uhvatila u šestoj nedelji.
Naslovni broj isporuči projekat. Brojevi u senci su razlog zbog kojeg je projekat vredeo.
Kratko zatvaranje
Ako se spremate da pokrenete projekat poput ovog, stvari za koje bismo voleli da smo znali nisu tehničke. Bile su:
Razgovarajte prvo sa ljudima najbližim ulazu. Znaju više od onoga što kaže organizaciona šema.
Izgradite skup za evaluaciju sa njima, ne odvojeno.
Tretirajte razgovor o upravljanju promenom kao isporuku, ne kao sporedni efekat.
Brojevi u senci — vreme vraćeno, varijansa sužena — su ono što se akumulira.
Model je bio najlakša nedelja projekta. Ostalih deset nedelja bili su ljudi, proces, i nemoderne navike koje čine da puštanje preživi.
Filed under: POSTMORTEM · METHOD First published: Mar 04, 2026
