Building an AI text detector from scratch

Moving from abstract concepts to a practical implementation

AI researcher Sebastian Raschka has published a guide on building a local AI text detector. He wrote this in response to Substack deploying a similar feature on its platform. The outcome is not just an academic example, but a fully functional API with a user interface. The architecture relies on a DistilBERT classifier that scores the likelihood of synthetic patterns on a scale from zero to one hundred percent.

Small language models find their specific purpose

The project illustrates a shift from massive LLMs toward smaller, narrowly targeted tools. The local detector is not merely a passive checking tool for human editors. It makes more sense as an automated verifier within a broader development cycle, providing feedback to other models so they avoid generating machine sounding phrases.

The cat and mouse game remains structurally unsolvable

Raschka explicitly warns against overestimating detection scores. Classifiers only recognize the statistical patterns they encountered in their training data. Any new model that bypasses these patterns renders the existing detector blind. Furthermore, the risk of false positives penalizes human authors who simply used a standard grammar checker to polish their writing.

The actual value is not in catching internet spam

The ultimate proof for these local tools won't be a perfectly clean internet completely free of synthetic noise. Their real power emerges when developers integrate rapid automated feedback (RLVR) into their training pipelines, bypassing the need to send sensitive training data to an external cloud API.

Lilith's verdict

AI text detectors are not a defense against spammers. They are control mechanisms for developers themselves, indicating how far their own models have drifted from natural human language.