Comparing Language Detection Libraries (& API) Using Java/ColdFusion/CFML

Language detection is a feature that we needed in a past project. I wrote an article in 2020 regarding the use of kju2 fork of the Optimaize Language Detector java library. The Optimaize library hasn't been updated since 2015 and the kju2 fork was placed in read-only mode on Apr 16, 2023.

I evaluated the Lingua java library. It claims to be the "The most accurate natural language detection library for Java and the JVM, suitable for long and short text alike" and also appears to be actively updated & supported. In my small unit test, Lingua seemed to be slightly slower and couldn't correctly identify Malay text.

The detection time for both java libraries was fairly random for English. Sometimes it would return a response in 295 ms and other times it would be 48,000+ ms. (Maybe it's just my developer PC.) kju2 seemed to be faster on average.

I also found a third-party Detect Language API that supports 165 languages and claims to have "high accuracy". It requires an API key and offers both free & premium plans.

If you're performing language detection with Java and/or ColdFusion/CFML, what are you using?

THAITHAI1THAI2th78MALAYMALAY2UNKNOWN291id88Pesan moral dari Cerita Rakyat Bawang Merah Bawang Putih adalah Jangan terya tamak dan serakah. Sepiap orang sudah memiliki rezekinya masing-masing. Orang yang teribu serakah akan mendapat Balasan yang setimpal dengan perbuatanya. Selalu berbuat baik lah Dalam ikiap hanpah laku, maka kita akan mendapat kebidahan dan happanika. Source Code

Should Be	kju2 lang	kju2 ms	lingua lang	lingua ms	api lang	api ms	Sample Text
ENGLISH	ENGLISH	2272	ENGLISH	570	en	537	A great way to learn Spanish vocabulary is by reading texts, stories or articles that are completely in the language. That is why we have written are own short reading passages in Spanish about different topics.
GREEK	GREEK	6	GREEK	12	el	105	Βίβλος γενέσεως Ἰησοῦ Χριστοῦ υἱοῦ Δαυεὶδ υἱοῦ Ἀβραάμ.
FRENCH	FRENCH	61	FRENCH	78	fr	70	En hiver, il fait froid en France. Le soleil se lève tard. Il fait encore nuit quand je vais au travail. Parfois, il y a même de la neige.
HEBREW	HEBREW	3	HEBREW	11	iw	110	כל ישראל יש להם חלק לעולם הבא, שנאמר ועמך כולם צדיקים, לעולם יירשו ארץ, נצר מטעי מעשה ידי להתפאר.
ARABIC	ARABIC	2	ARABIC	19	ar	81	عندما يريد العالم أن ‪يتكلّم ‬ ، فهو يتحدّث بلغة يونيكود. تسجّل الآن لحضور المؤتمر الدولي العاشر ليونيكود
CHINESE	CHINESE	2	CHINESE	8	zh	74	虽然它长得不好看，但是它有一颗无比善良的心。小猴子乐乐的家被大水冲垮了，无家可归。丑丑就让乐乐住在自己的家，还把自己最喜欢吃的巧克力分给乐乐吃。不仅如此，谁头痛、生病了，没钱买药，它都会尽其所能进行帮助。
KOREAN	KOREAN	12	KOREAN	3	ko	120	안녕하십니까 할리데이비슨 대구점 MC 우제헌입니다. 포티에잇 문의 전달받고 전화 드렸습니다만 연결되지 않아 문자 드립니다.
SPANISH	SPANISH	0	SPANISH	116	es	92	Habitualmente este término se aplica a todas las pistas donde aterrizan aviones, sin embargo el término correcto es aeródromo.
THAI	THAI	1	THAI	14	th	105	ข้อ 1 มนุษย์ทั้งหลายเกิดมามีอิสระและเสมอภาคกันในเกียรติศักด[เกียรติศักดิ์]และสิทธิ ต่างมีเหตุผลและมโนธรรม และควรปฏิบัติต่อกันด้วยเจตนารมณ์แห่งภราดรภาพ
VIETNAMESE	VIETNAMESE	2	VIETNAMESE	14	vi	98	Tất cả mọi người sinh ra đều được tự do và bình đẳng về nhân phẩm và quyền lợi. Mọi con người đều được tạo hóa ban cho lý trí và lương tâm và cần phải đối xử với nhau trong tình anh em.
TURKISH	TURKISH	3	TURKISH	93	tr	212	Yukarda mavi gök, asağıda yağız yer yaratıldıkta; ikisinin arasında insan oğlu yaratılmış. İnsan oğulları üzerine ecdadım Bumın hakan, İstemi hakan tahta oturmuş; oturarak Türk milletinin ülkesini, türesini, idare edivermiş, tanzim edivermis. Dört taraf hep düşman imiş. Asker sevk edip dört taraftaki kavmi hep (itaati altına) almış hep muti kılmış. Başlılara baş eğdirmiş, dizlilere diz çöktürmüş.
JAPANESE	JAPANESE	3	JAPANESE	7	ja	194	幸運こううんにも、息子むすこはこの四月しがつから保育園ほいくえんに入はいることができ、私わたしはまた働はたらき始はじめた。
RUSSIAN	RUSSIAN	2	RUSSIAN	1393	ru	96	All people are born free and equal in their dignity and rights. They are endowed with reason and conscience and must act in relation to each other in the spirit of brotherhood.
FINNISH	FINNISH	6	FINNISH	383	fi	125	Everyone has the right to receive education. Education must be free of charge, at least for primary and basic education. Primary education must be compulsory. Technical and vocational education must be universally available, and higher education must be equally open to all according to their abilities.
URDU	URDU	1	URDU	22	ur	104	इक मल्क पर शर्ष गिर बश्धास्थ करता करता.
						Hello World

https://gist.github.com/JamoCA/b883fbddf0303df8f4b0d597cfc2ae25

The above is the detailed content of Comparing Language Detection Libraries (& API) Using Java/ColdFusion/CFML. For more information, please follow other related articles on the PHP Chinese website!