Case study: 90%+ sustained accuracy on a Transperfect Dataforce interpolation annotation program

Transperfect Dataforce is the data annotation arm of Transperfect, a publicly known global language and content services company. The Dataforce division runs annotation, evaluation, and data collection programs for foundation model and vertical AI training, and operates in 20+ countries. The vendor is named in this case study because the program is referenced on their public materials and the operational data below is consistent with what they publish.

The program is the Indonesian leg of a multi-region annotation cohort covering sports motion (basketball, football, gymnastics), training motion (yoga, strength training, dance), and automotive motion (pedestrian tracking, in-cabin driver monitoring). The Indonesian pod handles the Asia-region and EU-prime-time shift work; the same SOP and gold set are used by parallel pods in other regions.

For each video clip, the annotator receives 24–48 frames, marks the object state on frame N, predicts the state on frame N+24, draws the appropriate bounding box, polygon, or keypoint set, and tags edge cases: motion blur, occlusion, scene transition, multiple objects of the same class, partial visibility. The output is a JSON manifest per clip with frame-level coordinates plus a confidence and motion-class tag per object.

Three sub-streams feed the program. Sports motion accounts for roughly 45% of the volume, training motion for 35%, and automotive motion for 20%. The sports stream is the highest volume but the lowest error rate; the automotive stream is the lowest volume but the highest error rate because of small-object detection (pedestrians at distance, dashboard reflections).

Transperfect's contractual accuracy floor is 90% on the weekly client-side QA sample. Below 90% for two consecutive weeks triggers a remediation plan; below 88% for one week triggers an immediate escalation. The 90% target is the line at which the labeled data is useful for the client's downstream model training; below 90%, label noise is too high to train on.

Zipang's actual sustained accuracy on the program has been above 90% since week 12. The rolling 12-week average has been between 91% and 93% across 2024 and 2025, finishing 2025 at 92.1%. The accuracy curve from week 1 (78%) to week 12 (92%) is the operational story of the program and is documented below.

Zipang's 5-gate funnel was applied with a gate 3 quiz specific to interpolation annotation: 100 short video clips (sports, training, automotive) for which the candidate must produce the frame-N and frame-N+24 labels per the client SOP. The quiz is scored against a frozen answer set; pass threshold is 80% on quiz accuracy, which approximates the 78% week 1 production accuracy plus a small headroom for ramp.

Aggregate pass rates for this program: gate 1 retained 26% of applicants, gate 2 retained 47%, gate 3 retained 54%, gate 4 retained 71%, gate 5 retained 81%. End-to-end, 60 of approximately 1,800 applicants (3.3%) reached the paid trial cohort. Of the 60, 20 converted to full-time production (33% trialist-to-full-time conversion). The lower conversion versus the France retail program reflects the harder task; the program needs fewer, sharper operators, not a larger cohort.

The active production cohort is 20 full-time agents — a deliberately small group. The Transperfect program rewards sustained accuracy and the client prefers fewer sharper operators over a larger cohort, partly because the task does not scale linearly: the QA reviewer and gold-set cycle is the bottleneck, and a 50-operator cohort would need two full-time QA reviewers, whereas a 20-operator cohort needs one.

The 20 agents are distributed across Jakarta, Bandung, and Yogyakarta. Weekly gold-set calibration runs on a 200-case frozen set: every Friday, the full cohort re-labels the 200 cases, scores are computed against the gold answer, and any operator whose accuracy drifts below 89% on the gold set is pulled into a 1:1 retraining session before live production the following week.

Three review layers run on every clip. First, peer review at frame N: a second operator labels frame N independently, and any disagreement greater than 5% on bounding-box IoU triggers a re-annotation. Second, supervisor review at frame N+24: a QA supervisor reviews every N+24 prediction against the source video, scoring motion-continuity and edge-case tagging. Third, weekly gold-set calibration on the 200-case frozen set, scored by the full cohort and compared to the previous week.

Microsecond-level KPI tracking means every label, every disagreement, every re-annotation is timestamped and attributable. The operations team can answer questions like 'which operator drifted on the automotive sub-stream after the gold set was updated in week 18' in a single dashboard query. Without the timestamp granularity, the weekly calibration cycle would be running blind.

Week 1 production accuracy landed at 78% on the first 2,000-clip batch — well below the 90% target. The error analysis showed three clusters: motion-blur edge cases (operators were drawing boxes too tight, losing the object when it crossed a frame boundary), automotive small-object detection (pedestrians at distance were missed entirely), and scene-transition frames (operators were labeling the post-transition frame instead of the transition itself).

Weeks 2–4: targeted retraining on the three error clusters, using the gold set as the teaching rubric. Production accuracy climbed to 82% by week 4. Weeks 5–8: introduction of the peer-review-at-frame-N layer, which caught disagreements before they reached the client. Accuracy climbed to 88% by week 8. Weeks 9–12: full introduction of the supervisor review at frame N+24, plus a 200-case weekly gold set. Accuracy crossed 90% in week 11 and landed at 92% in week 12.

The curve has been stable since week 12, oscillating between 91% and 93% on the rolling 12-week average. The 92.1% figure for end-2025 is within that band.

What the client got: 20 full-time Indonesian production agents holding 90%+ sustained accuracy on interpolation annotation across sports, training, and automotive streams. Weekly KPI dashboards. Microsecond-level attribution of every label, disagreement, and re-annotation. Monthly calibration reports showing gold-set scores, drift, and retraining sessions. The accuracy curve from 78% to 92% across the first 12 weeks, with the methodology that produced it.

What the client did not get: public naming of the individual operators or trainers on the program. The agents are Indonesian remote workers whose individual identities are not published. The trainers and QA supervisors are Zipang staff and are also not individually named. The program is referenced in the aggregate (432 deployed, 90%+ sustained, 60 → 20 conversion), not by name.

The program runs on an annual contract, auto-renewing in 12-month terms unless either party gives 60 days' notice. Pricing is per-task-delivered with a quality bonus that triggers when the rolling 12-week accuracy holds above 91%. The 30-day replacement guarantee covers any agent who falls below the gold-set threshold for two consecutive weeks or who triggers an escalation event. The 90-day window covers non-performance issues that surface later (consistently slow throughput, repeated QA flags, or failure to maintain shift coverage).

Replacement is the operator's responsibility, not the client's. The client does not re-run hiring; Zipang pulls from the active shortlist pool (fed by the same 5-gate funnel) and onboards the replacement within 5 business days. Across the life of the program, 7 of the original 20 full-time agents have been replaced; the current 20 are a mix of original and replacement operators, with the longest-tenured operator at 28 months.

The program is in its third contract year. Sustained accuracy is at 92.1% on the rolling 12-week average. The 20 full-time cohort is stable. Replacement is functioning. The client has not re-bid the program since the original contract, and the parallel pods in other regions are calibrated against the Indonesian pod's gold set as a baseline.

The 90% number is a contractual floor; the 92.1% is what the program actually delivers. Both are defensible. The accuracy curve from 78% in week 1 to 92% in week 12 is the operational story of how a 5-gate funnel, a frozen gold set, a three-layer review, and weekly calibration combine to land at a number the client can train models on.