AI Content Detector: What Tool is the Most Accurate?
By Peter Boyd on January 31, 2023 in Content Content Writing
PaperStreet writes an average of 360 pages of content for law firms throughout the country every month. Let us clarify…. “PaperStreet writes,” meaning real people with actual JD degrees. In our office, a real human develops a topic, a JD writer follows instructions and composes the articles, and an editor proofreads and optimizes the material – all real people with a heartbeat.
The same people who write that content each month are trying to wrap their heads around what the world of law firm content writing is going to look like in three years thanks to ChatGPT, an automation tool launched in November 2022 that basically does the work of writers and bloggers in about 10 seconds flat. No brain power necessary. The tool generates automatic text based on a query you provide, and the results could pass as a basic article (though not one our clients would love or appreciate) or at least part of one.
At PaperStreet it leaves us questioning how we can determine what’s AI-generated to assure our clients they are still getting unique content from an accurate person with a brain vs. a bot. So we did an experiment to determine the best AI content detection tools.
AI Content Detection Tools
We tested five popular AI content detection tools to see if they could accurately differentiate between our lawyer content writing and content that ChatGPT delivered.
Through each AI content detection tool, we tested 20 human-written pages and 20 pages of bot-generated content.
AI-Generated Content Examples
We prompted ChatGPT to write articles based on some of the topics we write for often. Some examples:
- how to incorporate your business in florida and the various business entities.
- a guide about dui arrest in florida
- summarizing the alternative dispute resolution process and mediation before trial
- divorce uncontested divorces in texas.
- medical malpractice in Ohio and section 2305.113
The responses averaged 384 words. The highest word count was 482 and the lowest was 283. Some examples of the verbiage provided include:
“summarizing the alternative dispute resolution process and mediation before trial”
“a guide about dui arrest in florida”
“how to incorporate your business in florida and the various business entities.”
Law Firm Content Writing Examples
For the examples of content written by our team of legal writers, we used a mixture of blog posts and web pages across different practice areas.
Boca Raton wrongful death practice area page
Long Island elder law practice area page
Securities law firm blog post
These posts vary in length so we did adjust for size as gptzero.me has a 5,000 character limit and Writer.com AI Detect has a 1500 character limit.
Best AI Content Detection Tools
The entire experiment took less than four hours, which should say something about how easy this content is to generate and test, and we did come to a conclusion and some key findings.
Our AI Content Detector Test Results
- The detection tools can identify human-generated content fairly accurately.
- Gptzero.me resulted in some false positives during the human writing test, but we like that it highlights the areas in question.
- When identifying AI-generated content, gptzero.me was also the most accurate.
- As a secondary tool, GPT2 would suffice.
Download the full results of our test of the lawyer written content.
Download the full results of our test of the bot written content.
How to Detect AI-Generated Content
The most accurate way to identify AI-generated content is to use an online AI content detector. Based on our experiment the most accurate AI-content detector is Gptzero.me. When using this AI content detector you copy and paste the material from your written work into the dialog box and click “Ger Results.” You also have the option of uploading a file. To use the tool, you must agree to the terms of service.
Key Takeaways
As an industry, we’re wading into uncharted territory when we consider how the use of automated content will impact the entire content-writing industry. Lawyers and other professionals will need to decide the value of bot-generated content vs. writings from someone with actual real-life experience.
At PaperStreet, we would be fools to ignore this latest tech discovery, but we are confident it won’t replace our writing team. The law firms we work with expect intelligent and well-thought-out writings on their firm, practices, and latest happenings. One of the ways we differentiate ourselves from other law firm marketing agencies is by working with only lawyer writers. While a bot may be able to contribute some opening text or summarize a new law, it can’t add the value that we believe comes from real people.
- You are able to discern if there is bot content on most articles.
- Gptzero.me is the most accurate AI content detector.
- The more detailed the content, the more likely it will be rated as human.
- Specific known facts and listicles are more likely to be considered as AI generated text.
- The tools with word limits take more time because you have to cut and paste.
- It takes less than 60 seconds to manually scan a piece of writing across the five tools.
- AI-generated content is super easy to create, taking less than a minute.
