1. Gabatarwa
Gyaran Rubutun Sinanci (CSC) aikin NLP ne na asali da aka yi niyya don gano da kuma gyara kurakuran rubutu a cikin rubutun Sinanci. Yana da muhimmanci ga aikace-aikace kamar Gane Sunan Mahalli, Gane Haruffa ta Gani (OCR), da kuma binciken gidan yanar gizo. Hanyar da ta fi yawa ita ce a ɗauki CSC a matsayin aikin alamar jerin, daidaita samfuran BERT akan nau'ikan jumla biyu. Duk da haka, wannan takarda ta gano wata muhimmiyar aibi a cikin wannan tsari kuma ta ba da shawarar wata sabuwar mafita: Tsarin Harshe na Sake Tsarawa (ReLM).
2. Hanyar Aiki
2.1 Laifin Alamar Jerin
Babban hujja akan hanyar alamar jerin shine tsarin koyon sa wanda bai dace da hankali ba. A cikin CSC, yawancin haruffa tsakanin jumlolin tushe da manufa suna daidai. Wannan yana ba wa samfuran damar "yin magudi" ta hanyar haddace ma'auratan haruffan kuskure-gyara na musamman kuma kawai su kwafi sauran, suna samun maki masu yawa ba tare da fahimtar ma'anar jumla da gaske ba. Gyaran ya zama mai matukar dogaro da tsarin kuskure da kansa, maimakon ma'anar gabaɗaya na jumla. Wannan yana haifar da rashin haɗawa da canja wuri, musamman a cikin yanayin zero-shot ko ƴan-shot inda ba a gani ba tsarin kuskure ya bayyana.
Hoto na 1 yana kwatanta wannan aibi. Samfurin da aka horar akan ma'auratan ("age" -> "remember") zai yi kuskuren gyara sabon misali na "age" zuwa "remember" ko da yake mahallin (misali, "not to dismantle the engine") a fili yana buƙatar wani gyara daban ("not"). Wannan yana nuna gazawar haɗa ma'anar mahallin.
2.2 Tsarin ReLM
ReLM yana ba da shawarar canjin tsari: a ɗauki gyaran rubutu a matsayin aikin sake tsarawa na jumla, yana kwatanta tsarin fahimtar ɗan adam. Maimakon alamar haruffa zuwa haruffa, ana horar da samfurin don sake tsara dukan jumla ta hanyar cika ramukan da aka rufe bisa ga ma'anar ma'ana da aka ɓoye na jumlar tushe. Wannan yana tilasta samfurin gina cikakkiyar fahimtar jumla kafin samar da gyare-gyare, yana karya dogaro da yawa akan tsarin kuskure da aka haddace.
3. Cikakkun Bayanai na Fasaha
3.1 Tsarin Ƙirar Model
An gina ReLM akan tsarin ƙirar BERT. Jumlar tushe $S = \{c_1, c_2, ..., c_n\}$ an fara ɓoye ta zuwa wakilcin ma'ana mai mahallin ta amfani da mai ɓoyewa na BERT. Muhimmanci, wuraren haruffan da aka gano a matsayin madaidaicin kurakurai (misali, ta hanyar wani na'urar gano daban ko ta hanyar rufe duk wurare) ana maye gurbinsu da wata alama ta musamman `[MASK]`.
3.2 Manufar Horarwa
Ana horar da samfurin don sake gina jumlar manufa daidai $T = \{t_1, t_2, ..., t_n\}$ ta hanyar hasashen alamomin don wuraren da aka rufe, bisa ga mahallin da ba a rufe ba. Manufar horarwa ita ce asarar yaren da aka rufe (MLM) na yau da kullun, amma ana amfani da ita da dabara don tilasta sake tsarawa:
$\mathcal{L} = -\sum_{i \in M} \log P(t_i | S_{\\backslash M})$
inda $M$ shine saitin wuraren da aka rufe (madaidaicin kurakurai) kuma $S_{\\backslash M}$ shine jumlar tushe tare da waɗannan wuraren da aka rufe. Wannan manufar tana ƙarfafa samfurin yin amfani da ma'anar jumla ta duniya, ba kawai ma'auratan haruffa na gida ba, don hasashen cikawa daidai.
4. Gwaje-gwaje & Sakamako
4.1 Aikin Ma'auni
An kimanta ReLM akan ma'auni na CSC na yau da kullun kamar SIGHAN. Sakamakon ya nuna ya sami sabon aiki mafi kyau na zamani, ya fi samfuran da suka gabata waɗanda suka dogara da alamar jerin (misali, waɗanda suka haɗa da siffofin sauti) da babban tazara. Wannan yana tabbatar da ingancin tsarin sake tsarawa.
Ma'auni Mai Muhimmanci (Misali): Gano F1 ya inganta da kusan ~2.5%; Daidaitaccen Gyara ya inganta da kusan ~3.1% akan mafi kyawun samfurin da ya gabata.
4.2 Haɗawa Ba tare da Horarwa ba (Zero-Shot)
Gwaji mai mahimmanci shine aikin zero-shot akan bayanan da ke ɗauke da tsarin kuskure da ba a gani yayin horarwa ba. ReLM ya nuna haɗawa mafi girma idan aka kwatanta da samfuran alama, waɗanda suka fuskanci faɗuwar aiki mai mahimmanci. Wannan yana magance kai tsaye babban aibin da aka gano a baya, yana tabbatar da ReLM yana koyon ƙarin ilimin harshe mai canzawa.
5. Tsarin Bincike & Nazarin Lamari
Babban Fahimta: Babban nasarar takardar shine gane CSC a matsayin matsalar samarwa da ke yin kama da matsalar alamar. Samfuran alama suna da banbancewa—suna rarraba kowane hali. ReLM ya sake tsara shi a matsayin samarwa mai sharadi—ƙirƙirar jumla da aka gyara daga wacce ta lalace. Wannan ya yi daidai da nasarar samfuran samarwa a wasu ayyukan NLP kamar fassarar inji (misali, tsarin Transformer) da cika rubutu (misali, T5). Fahimtar ita ce gyaran gaskiya yana buƙatar amincin ma'ana ga niyya, ba kawai daidaita tsarin gida ba.
Tsarin Ma'ana: Hujjar tana da kaifi sosai: 1) Gano toshewar (haddacewa a cikin alama). 2) Ba da shawarar madadin da ya dace da fahimta (sake tsarawa kamar ɗan adam). 3) Aiwatar da shi ta amfani da tsarin ƙira da aka tabbatar (BERT MLM). 4) Tabbatar da shi tare da ma'auni masu wuya (SOTA akan daidaitacce da zero-shot). Gudun daga binciken matsalar zuwa ƙirar mafita yana da haɗin kai kuma yana jan hankali.
Ƙarfi & Aibobi: Babban ƙarfi shine kyawun ra'ayi da tabbacin gwaji. Yana magance matsalar gaske tare da sauƙi amma mai ƙarfi canji. Amfani da BERT ya sa ya zama mai amfani kuma mai maimaitawa. Duk da haka, wani yuwuwar aibi shine dogaro da wata hanyar gano kuskure ta daban ko dabarar "rufe-duk" mai ƙarfi yayin ƙididdigewa, wanda zai iya zama mara inganci. Takardar za ta iya bincika ƙarin dabarun rufewa masu zurfi, masu koyawa kamar Gano Alamar da aka Maye gurbin ELECTRA. Bugu da ƙari, yayin da yake inganta haɗawa, aikinsa akan kurakurai da ba a saba gani ba ko masu shakku sosai a cikin mahallin rikitarwa har yanzu tambaya ce da ba a buɗe ba.
Fahimta Mai Aiki: Ga masu aiki, wannan alama ce bayyananna don ƙaura daga samfuran alama kawai don CSC. Tsarin ReLM yana da sauƙin daidaitawa. Aikin gaba ya kamata ya mai da hankali kan: 1) Gano & Gyara Haɗe: Haɗa wani ɓangare mai horarwa don yanke shawara abin da za a rufe, ya wuce dabarun dabaru. 2) Amfani da Manyan LMs: Aiwatar da wannan tsarin sake tsarawa ga ƙarin samfuran samarwa masu ƙarfi kamar GPT-3.5/4 ko LLaMA don CSC na ƴan-shot. 3) Canja wurin Tsakanin Harsuna: Gwada ko hanyar sake tsarawa ta haɗu da gyaran rubutu a wasu harsuna masu zurfin rubutu, kamar Jafananci ko Thai. 4) Aiwatarwa a Duniyar Gaske: Kimanta jinkiri da buƙatun albarkatu don aikace-aikace na ainihi kamar masu gyara hanyar shigarwa ko dandamali na hira.
Nazarin Lamari (Babu lamba): Yi la'akari da jumlar kuskure: "这个苹果很营样" (Wannan apple yana da gina jiki mai yawa?). Samfurin alama zai iya ganin "营"->"营" (daidai) da "样"->"养" (gina jiki) daban. Zai iya yin kuskuren fitarwa "这个苹果很营养" (daidai) amma kuma zai iya ruɗe. ReLM, ta hanyar rufe "营样" da sake tsara ɓangaren a cikin mahallin "苹果" (apple) da "很" (sosai), yana iya samar da kalmomin da suka dace kuma daidai "营养" kai tsaye, yayin da yake amfani da cikakkiyar ma'anar jumla don zaɓar mafi kyawun kalmar haɗe.
6. Aikace-aikace na Gaba & Hanyoyi
- Mataimakan Rubutu Mai Hikima: Haɗawa cikin na'urori masu sarrafa kalmomi da hanyoyin shigarwa don ainihi, mai fahimtar mahalli gyaran rubutu da kurakuran nahawu na Sinanci.
- Fasahar Ilimi: Ƙarfafa ƙarin tsarin maki da tsarin amsa mai zurfi don masu koyon harshen Sinanci, yana bayyana gyare-gyare bisa ga mahallin ma'ana.
- Maido da Takardu: Haɓaka OCR da hanyoyin ƙididdiga na takardun tarihi ta hanyar gyara kurakuran dubawa ba kawai bisa ga siffar hali ba, amma akan mahallin takarda.
- Tsakanin Yanayin CSC: Tsawaita ra'ayin sake tsarawa don gyara kurakuran da suka taso daga tsarin magana-zuwa-rubutu, inda kurakurai suke sauti, suna buƙatar fahimtar rafin ma'anar da aka faɗa.
- Tushe don Ƙarfin NLP: Amfani da ReLM a matsayin kayan aikin koyarwa ko haɓaka bayanai don ƙirƙirar ƙarin samfuran masu jure wa hayaniya don ayyuka na gaba kamar nazarin ra'ayi ko fassarar inji.
7. Nassoshi
- Liu, L., Wu, H., & Zhao, H. (2024). Chinese Spelling Correction as Rephrasing Language Model. arXiv preprint arXiv:2308.08796v3.
- Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT.
- Clark, K., Luong, M. T., Le, Q. V., & Manning, C. D. (2020). ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. ICLR.
- Raffel, C., et al. (2020). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. JMLR.
- Touvron, H., et al. (2023). LLaMA: Open and Efficient Foundation Language Models. arXiv preprint arXiv:2302.13971.
- Yu, J., & Li, Z. (2014). Chinese Spelling Error Detection and Correction Based on Language Model, Pronunciation, and Shape. Proceedings of the Third CIPS-SIGHAN Joint Conference on Chinese Language Processing.