For this benchmark, we submitted the same 429 documents to Academi.cx and Pangram through their APIs, then compared each tool’s scores with the documents’ actual Turnitin AI scores. Together those documents add up to close to 944,000 words, averaging about 2,200 words each, ranging from a 311 word short answer to a 25,000 word thesis.
How close were the predictions
Across all 429 documents, Academi.cx averaged 1.92 points off the real Turnitin score. Pangram averaged 13.09 points off.
Interestingly, Pangram's median error is 0, same as Academi.cx. That means Pangram often lands exactly on the number, but a smaller share of much bigger misses is enough to drag its average error up sharply. A tool can be right most of the time and still carry a double digit mean error if it's occasionally very wrong.
Breaking it down by document type
The 429 documents split into 220 that Turnitin scored as 0% AI, 99 it scored as 100% AI, and 110 that fell somewhere in between.
On the 0% documents, Academi.cx averaged 0.18 points of error, Pangram averaged 7.6. On the 100% documents the gap narrows, Academi.cx averaged 3.53 points off versus Pangram at 4.23, its closest showing of the three groups. The mixed bucket is where the gap opens widest, Academi.cx averaged 3.95 points of error there while Pangram jumped to 32.04, by far its weakest range and the main driver behind its overall average.
False flags and missed AI
Academi.cx had a 0% false flag rate and a 0% missed AI rate across all 429 documents, meaning it never scored a genuinely 0% document above 20%, and never scored a genuinely 100% document below 80%. Pangram came in at an 11.4% false flag rate and a 6.1% missed AI rate on the same set.
Sentence level accuracy
Pangram's API only returns a single document level score, not sentence by sentence highlighting, so it can't be measured against Turnitin's sentence calls the way the other tools in this series can. That comparison is left out here rather than estimated.
The takeaway
Across all 429 documents, Academi.cx stayed close to the actual Turnitin percentage almost without exception, while Pangram handled the easy extremes reasonably well but lost accuracy fast once documents moved into the mixed range, where its 32.04 point average error suggests it struggles more with sizing partial AI content correctly than with missing the call outright.
See the full benchmark results.
