AI Content Detector: What Tool is the Most Accurate?

A table showing ChatGPT's examples, capabilities, and limitations. Examples include providing creative ideas. Capabilities include remembering user context. Limitations include possibly generating incorrect information.

PaperStreet writes an average of 360 pages of content for law firms throughout the country every month. Let us clarify…. “PaperStreet writes,” meaning real people with actual JD degrees. In our office, a real human develops a topic, a JD writer follows instructions and composes the articles, and an editor proofreads and optimizes the material – all real people with a heartbeat.

The same people who write that content each month are trying to wrap their heads around what the world of law firm content writing is going to look like in three years thanks to ChatGPT, an automation tool launched in November 2022 that basically does the work of writers and bloggers in about 10 seconds flat. No brain power necessary. The tool generates automatic text based on a query you provide, and the results could pass as a basic article (though not one our clients would love or appreciate) or at least part of one. 

At PaperStreet it leaves us questioning how we can determine what’s AI-generated to assure our clients they are still getting unique content from an accurate person with a brain vs. a bot. So we did an experiment to determine the best AI content detection tools.

AI Content Detection Tools

We tested five popular AI content detection tools to see if they could accurately differentiate between our lawyer content writing and content that ChatGPT delivered. 

  1. GPT2 
  2. Copyleaks
  3. Writer.com AI Detect
  4. AI Text Classifier
  5. gptzero.me

Through each AI content detection tool, we tested 20 human-written pages and 20 pages of bot-generated content. 

AI-Generated Content Examples

We prompted ChatGPT to write articles based on some of the topics we write for often. Some examples: 

  • how to incorporate your business in florida and the various business entities.
  • a guide about dui arrest in florida
  • summarizing the alternative dispute resolution process and mediation before trial
  • divorce uncontested divorces in texas.
  • medical malpractice in Ohio and section 2305.113

The responses averaged 384 words. The highest word count was 482 and the lowest was 283. Some examples of the verbiage provided include: 

“summarizing the alternative dispute resolution process and mediation before trial”

Image of a text document describing Alternative Dispute Resolution (ADR), its mediation process, advantages, and confidentiality aspects. The text details how mediation helps resolve disputes outside of court.

 

“a guide about dui arrest in florida”

Detailed text guidelines on what to do if arrested for a DUI in Florida, including tips on legal representation, rights during an arrest, and consequences such as fines, jail time, and license suspension.

 

“how to incorporate your business in florida and the various business entities.”

An image of text explaining various business entities in Florida, including sole proprietorships, general partnerships, limited partnerships, LLCs, and corporations, their benefits, and the registration process.

Law Firm Content Writing Examples

For the examples of content written by our team of legal writers, we used a mixture of blog posts and web pages across different practice areas. 

 

Boca Raton wrongful death practice area page

Screenshot of a webpage from The Grife Law Firm discussing Florida law on wrongful death claims, with a sidebar form for requesting a free consultation. The headline reads, "Florida Law Limits Who Can Pursue a Wrongful Death Claim.

 

Long Island elder law practice area page

Website screenshot for the law firm "Davidow, Davidow, Siegel & Stern, LLP" offering elder law services for New Yorkers, including a contact form for inquiries and various ways to reach their office.

 

Securities law firm blog post

Screenshot of a blog page titled "Latest Headlines in Investment Fraud – August 2022" by Zamansky LLC. The page highlights a warning from FinCEN about the "Rising Trend of Elder Financial Exploitation.

These posts vary in length so we did adjust for size as gptzero.me has a 5,000 character limit and Writer.com AI Detect has a 1500 character limit.

 

Best AI Content Detection Tools

The entire experiment took less than four hours, which should say something about how easy this content is to generate and test, and we did come to a conclusion and some key findings.

Our AI Content Detector Test Results

  • The detection tools can identify human-generated content fairly accurately. 
  • Gptzero.me resulted in some false positives during the human writing test, but we like that it highlights the areas in question. 
  • When identifying AI-generated content, gptzero.me was also the most accurate. 
  • As a secondary tool, GPT2 would suffice. 

 

Download the full results of our test of the lawyer written content.

 

 

Download the full results of our test of the bot written content. 

 

How to Detect AI-Generated Content

The most accurate way to identify AI-generated content is to use an online AI content detector. Based on our experiment the most accurate AI-content detector is Gptzero.me. When using this AI content detector you copy and paste the material from your written work into the dialog box and click “Ger Results.” You also have the option of uploading a file. To use the tool, you must agree to the terms of service. 

Key Takeaways

As an industry, we’re wading into uncharted territory when we consider how the use of automated content will impact the entire content-writing industry. Lawyers and other professionals will need to decide the value of bot-generated content vs. writings from someone with actual real-life experience. 

At PaperStreet, we would be fools to ignore this latest tech discovery, but we are confident it won’t replace our writing team. The law firms we work with expect intelligent and well-thought-out writings on their firm, practices, and latest happenings. One of the ways we differentiate ourselves from other law firm marketing agencies is by working with only lawyer writers. While a bot may be able to contribute some opening text or summarize a new law, it can’t add the value that we believe comes from real people. 

  • You are able to discern if there is bot content on most articles.
  • Gptzero.me is the most accurate AI content detector.
  • The more detailed the content, the more likely it will be rated as human.
  • Specific known facts and listicles are more likely to be considered as AI generated text.
  • The tools with word limits take more time because you have to cut and paste.
  • It takes less than 60 seconds to manually scan a piece of writing across the five tools.
  • AI-generated content is super easy to create, taking less than a minute.
Share This

Join the Sales and Marketing News, receive our last insights, tips and best practices.

Our 7 Guarantees

Keeping 2,000+ Clients Happy Since 2001.

1
You Will Love Your Design We design to please you and your clients
2
Same-Day Support 24-hour turnaround edits during business hours
3
Free Education We provide knowledge to help you expand
4
No Hidden Charges We quote flat-rate projects
5
Own Your Site No strings attached
6
We Create Results SEO, PPC, content + design = clients
7
We Make Life Easier One agency for web, branding and marketing