Teburin Abubuwan Ciki
1. Gabatarwa
Gyaran Kuskuren Rubutun Sinanci (CSC) aiki ne mai mahimmanci na NLP wanda ke mai da hankali kan gano da kuma gyara kurakuran rubutu a cikin rubutun Sinanci. Yana aiki azaman tushen ginshiƙi don aikace-aikace kamar Gane Sunayen Abubuwa (NER), Sarrafa Rubutun Gani (OCR) bayan aiki, da kuma inganta injin bincike. Hanyoyin zamani na gargajiya suna sanya CSC a matsayin matsalar yiwa alama, suna daidaita model irin su BERT don sanya haruffan kuskure zuwa na daidai. Duk da haka, wannan takarda ta gano iyaka ta asali a cikin wannan hanya: tana yin gyare-gyare da yawa bisa tsarin kuskure da kansa, maimakon ma'anar jimla gaba ɗaya, wanda ke haifar da rashin amfani a kan kurakuran da ba a gani ba.
2. Hanyar Aiki
2.1. Laifin Tsarin Yiwa Alama
Takardar ta yi iƙirarin cewa tsarin yiwa alama da ya yaɗu bai dace da gyaran ɗan adam ba. Mutane suna fahimtar ma'anar jimla da farko sannan su sake tsara ta daidai bisa ilimin harshe, ba ta hanyar haddace madaidaicin haruffa kai tsaye ba. Duk da haka, model ɗin yiwa alama na iya samun maki masu yawa ta hanyar kawai haddace nau'ikan kuskure-gyara da suka yawaita daga bayanan horo da kuma kwafin haruffan da ba su canza ba, sun kasa daidaitawa da mahallin lokacin da sabbin kurakura suka bayyana. Hoto na 1 a cikin PDF ya kwatanta wannan tare da misali inda model ya canza "age" zuwa "remember" bisa tsarin da aka haddace, yayin da ɗan adam zai gyara shi zuwa "not" bisa ma'anar jimla.
2.2. Tsarin ReLM
Don magance wannan, marubutan sun ba da shawarar Tsarin Harshe na Sake Tsarawa (ReLM). Maimakon yiwa haruffa alama zuwa haruffa, ReLM an horar da shi don sake tsara dukkan jimlar shigarwa. An saka jimlar tushe cikin wakilcin ma'ana. Daga nan model ɗin yana samar da jimlar da aka gyara ta hanyar "cika" takamaiman ramukan rufe fuska a cikin wannan mahallin ma'ana. Wannan yana tilasta model ɗin ya dogara da fahimtar jimla gabaɗaya maimakon haddace kuskure a wuri.
3. Cikakkun Bayanai na Fasaha
3.1. Tsarin Lissafi
Idan aka ba da jimlar tushe $X = \{x_1, x_2, ..., x_n\}$ mai ɗauke da yuwuwar kurakurai, manufar ita ce samar da jimlar da aka gyara $Y = \{y_1, y_2, ..., y_m\}$. A cikin tsarin yiwa alama, manufar sau da yawa ana siffanta ta azaman $P(Y|X) = \prod_{i=1}^{n} P(y_i | x_i, \text{mahalli})$, yana ɗaure $y_i$ sosai da $x_i$.
ReLM ya sake tsara wannan. Da farko yana ƙirƙirar wani ɓangare na rufe fuska na $X$, wanda aka nuna shi da $X_{\text{mask}}$, inda wasu alamun (mai yuwuwar kurakurai) aka maye gurbinsu da takamaiman alamar [MASK]. Manufar horarwa ita ce sake gina $Y$ daga $X_{\text{mask}}$ bisa cikakken mahalli:
$$P(Y|X) \approx P(Y | X_{\text{mask}}) = \prod_{j=1}^{m} P(y_j | X_{\text{mask}}, y_{ An gina ReLM akan mai saka BERT da aka riga aka horar. An saka jimlar shigarwa ta BERT. Don samarwa, ana amfani da mai karantawa (ko kuma shugaban harshe mai rufe fuska) don hasashen alamun don wuraren da aka rufe fuska ta hanyar kai-da-kai ko a layi daya, dangane da takamaiman dabarun cikawa. An daidaita model ɗin akan tarin rubutun da ke da kuskure da na daidai. An kimanta ReLM akan ma'auni na CSC kamar SIGHAN 2013, 2014, da 2015. Sakamakon ya nuna cewa ReLM ya sami sabon aiki mafi kyau, ya fi na baya model ɗin yiwa alama (misali, model ɗin da ke haɗa siffofin sauti kamar SpellGCN) gaba ɗaya. An danganta nasarorin aikin da ƙarfin ikonsa na sarrafa gyare-gyaren da suka dogara da mahalli. Gwaji mai mahimmanci shine aikin zero-shot akan tarin bayanai masu ɗauke da tsarin kuskure da ba a gani yayin horo ba. ReLM ya nuna ingantaccen amfani sosai idan aka kwatanta da model ɗin yiwa alama. Wannan shaida ce kai tsaye cewa manufarsa ta sake tsarawa tana haifar da koyo da ƙarin ilimin harshe mai iya canzawa maimakon madaidaicin kuskure. Tsari: Don kimanta ƙarfin model ɗin CSC, muna ba da shawarar bincike mai ginshiƙi biyu: Haddacewa vs. Fahimta da Hankalin Mahalli. Nazarin Lamari (Babu Code): Yi la'akari da misalin daga PDF: Shigarwa: "Age to dismantle the engine when it fails." Model ɗin yiwa alama da aka horar akan nau'i ("age" -> "remember") zai iya fitar da "Remember to dismantle...", yana amfani da ƙa'idar da aka haddace ba daidai ba. Mutum ko ReLM, yana fahimtar ma'anar (shawara game da gazawar injin), zai iya fitar da "Not to dismantle..." ko "Do not dismantle...". Wannan lamari yana gwada ikon model ɗin na soke tsarin da aka haddace tare da fahimtar mahalli, wani muhimmin bambanci ga ReLM. Tsarin sake tsarawa na ReLM yana da aikace-aikace masu ban sha'awa fiye da CSC: Fahimta ta Asali: Babban nasarar takardar ba kawai sabon makin SOTA ba ne; shine gyaran falsafa ga yadda muke ƙirar gyaran harshe. Marubutan sun gano daidai cewa ɗaukar CSC a matsayin matsalar "kuskuren rubutu" (yiwa alama) kuskuren rukuni ne. Gyaran harshe a asalinsa aiki ne na samarwa, mai sanin ma'ana. Wannan ya yi daidai da manyan yanayin AI suna motsawa daga model ɗin nuna bambanci zuwa na samarwa, kamar yadda aka gani a canjin daga CNN na rarrabuwa zuwa model ɗin samar hoto kamar DALL-E ko tsarin da ke ayyana tsari kamar CycleGAN (Isola et al., 2017), wanda ya sake tsara fassarar hoto a matsayin matsalar sake gini mai daidaitaccen zagayawa maimakon madaidaicin pixel a layi daya. Kwararar Ma'ana: Hujjar tana da kaifi sosai: 1) Nuna cewa hanyoyin na yanzu suna aiki amma saboda dalilan da ba daidai ba (haddacewa). 2) Gano tushen dalili (makantar manufar yiwa alama). 3) Ba da madadin da ya dace da fahimta (sake tsarawa). 4) Tabbatar da cewa wannan madadin ba kawai yana aiki ba ne amma yana magance laifin da aka gano (ingantaccen amfani). Amfani da gwajin zero-shot yana da kyau musamman—shi daidai gwargwado ne na bugun kashewa. Ƙarfi & Kurakurai: Babban ƙarfin shine kyawun ra'ayi da tabbatarwa ta gwaji. Manufar sake tsarawa ta fi dacewa da ainihin yanayin aikin. Duk da haka, yuwuwar kuskuren takardar shine rashin ƙayyadaddun aiki na "sake tsarawa." Ta yaya ake zaɓar ramukan rufe fuska? Shin koyaushe cikawa ce ɗaya-zuwa-ɗaya, ko tana iya sarrafa saka/ goge? Farashin lissafi na samarwa idan aka kwatanta da yiwa alama shima yana da yuwuwar ya fi girma, wanda kawai aka nuna shi. Yayin da suka ambaci albarkatu kamar kwas ɗin Stanford NLP don ilimin Transformer na tushe, kwatantawa mai zurfi tare da model ɗin mai saka-mai karantawa don bitar rubutu (kamar T5) zai ƙarfafa matsayi. Fahimta Mai Aiki: Ga masu aiki: Nan da nan rage fifikon model ɗin yiwa alama kawai don kowane aikin gyaran harshe da ke buƙatar mahalli. Tsarin ReLM shine sabon ma'auni. Ga masu bincike: Wannan aikin ya buɗe ƙofa. Matakai na gaba suna bayyana: 1) Girma: Aiwatar da wannan manufa ga LLMs masu karantawa kawai (misali, umarni-daidaita GPT-4 don gyara). 2) Yi amfani gabaɗaya: Gwada wannan akan gyaran kuskuren nahawu (GEC) don Ingilishi da sauran harsuna—yuwuwar yana da girma. 3) Inganta: Ƙirƙirar dabarun cikawa masu inganci don rage kayan aikin jinkiri. Wannan takarda ba ƙarshen labari ba ce; ita ce babi na farko mai jan hankali na sabuwar hanya don gina ingantattun tsarin gyaran harshe kamar na ɗan adam.3.2. Tsarin Ƙirar Model
4. Gwaje-gwaje & Sakamako
4.1. Aikin Ma'auni
4.2. Amfani Ba tare da Horarwa Ba (Zero-Shot)
5. Tsarin Bincike & Nazarin Lamari
6. Ayyuka na Gaba & Jagorori
7. Nassoshi
8. Binciken Kwararru & Fahimta