Tsarin Ma'ana Mai Zurfi don Binciken Ilimi: Nazarin Maganin Duolingo SLAM na 2018

Tsarin Abubuwan Ciki

---

1. Gabatarwa & Bayyani

Wannan takarda tana gabatar da maganin marubucin ga Aikin Raba na Duolingo na 2018 akan Tsarin Samun Harshe na Biyu (SLAM). Babban kalubalen shine binciken ilimi a matakin kalma: hasashen ko ɗalibi zai rubuta kalmomin sabon jumla daidai, idan aka yi la'akari da bayanan ƙoƙarinsa na tarihi akan dubunnan jimloli da aka yiwa lakabi da siffofin ƙamus, ilimin halittar jiki, da na nahawu.

Maganin da aka gabatar yana amfani da Tsarin Ma'ana Mai Zurfi (DeepFM), wani tsari gauraye wanda ya haɗa sashi mai fadi (Tsarin Ma'ana) don koyon hulɗar siffofi biyu-biyu da sashi mai zurfi (Cibiyar Jijiya Mai Zurfi) don koyon hulɗar siffofi mafi girma. Tsarin ya sami AUC na 0.815, wanda ya fi na tushen koma baya na lissafi (AUC 0.774) amma bai kai na tsarin da ya fi yin aiki (AUC 0.861) ba. Aikin ya sanya DeepFM a matsayin tsari mai sassauƙa wanda zai iya haɗa tsarin ilimi na gargajiya kamar Ka'idar Amsar Abubuwan Tambaya (IRT).

2. Ayyukan Da Suka Gabata & Tushen Ka'idoji

Takardar ta sanya gudummawarta a cikin faɗin yanayin tsarin ɗalibi da binciken ilimi.

2.1. Ka'idar Amsar Abubuwan Tambaya (IRT)

IRT wani tsari ne na ilimin halayyar ɗan adam wanda ke tsarin yuwuwar amsa daidai a matsayin aiki na iyawar ɗalibi ($\theta$) da sigogin abu (misali, wahala $b$). Wani tsari na gama-gari shine tsarin lissafi mai sigogi biyu (2PL): $P(\text{daidai} | \theta) = \sigma(a(\theta - b))$, inda $a$ shine nuna bambanci kuma $\sigma$ shine aikin lissafi. Takardar ta lura cewa IRT ta zama tushe mai ƙarfi, mai bayyani amma yawanci ba ta haɗa cikakkun bayanan gefe ba.

2.2. Juyin Halittar Binciken Ilimi

Binciken ilimi yana mai da hankali kan tsarin juyin halittar ilimin ɗalibi akan lokaci.

• Binciken Ilimi na Bayesian (BKT): Yana tsarin mai koyo a matsayin Tsarin Markov Boye tare da jihohin ilimi masu ɓoye.
• Binciken Ilimi Mai Zurfi (DKT): Yana amfani da Cibiyoyin Jijiya Maimaitawa (RNNs), kamar LSTMs, don tsarin jerin lokutan hulɗar ɗalibi. Takardar ta ambaci aikin Wilson et al. (2016) wanda ya nuna cewa bambance-bambancen IRT na iya fi tsarin DKT na farko, yana nuna buƙatar ingantattun tsare-tsare masu sanin siffofi.

2.3. Koyo Mai Fadi & Mai Zurfi

Takardar ta ginu akan tsarin Koyo Mai Fadi & Mai Zurfi wanda Cheng et al. (2016) suka gabatar a Google. Tsarin layi "mai fadi" yana ƙwaƙwalwar haɗuwar siffofi akai-akai, yayin da cibiyar jijiya "mai zurfi" ke yin gama-gari zuwa haɗuwar siffofi da ba a gani ba. Guo et al. (2017) sun ba da shawarar maye gurbin tsarin layi mai fadi da Tsarin Ma'ana (FM), wanda ke tsarin duk hulɗar siffofi biyu-biyu ta hanyar sigogi masu rarrabuwa, wanda ya kai ga tsarin DeepFM.

3. Tsarin Ma'ana Mai Zurfi (DeepFM) don Binciken Ilimi

Takardar ta daidaita tsarin DeepFM don fagen binciken ilimi.

3.1. Tsarin Tsari & Tsarawa

DeepFM ya ƙunshi sassa biyu masu layi daya wanda aka haɗa sakamakonsu:

1. Sashin FM: Yana tsarin hulɗar siffofi na layi da biyu-biyu. Don ƙarar siffa $\mathbf{x}$, sakamakon FM shine: $y_{FM} = w_0 + \sum_{i=1}^n w_i x_i + \sum_{i=1}^n \sum_{j=i+1}^n \langle \mathbf{v}_i, \mathbf{v}_j \rangle x_i x_j$, inda $\mathbf{v}_i$ suke ƙungiyoyin ma'ana masu ɓoye.
2. Sashin Zurfi: Cibiyar jijiya ta gaba da gaba ta al'ada wacce ke ɗaukar haɗaɗɗun bayanan siffa a matsayin shigarwa kuma tana koyon ƙa'idodi masu rikitarwa, masu matsayi mafi girma.

Hasashen ƙarshe shine: $p(\mathbf{x}) = \psi(y_{FM} + y_{DNN})$, inda $\psi$ shine aikin haɗi (misali, sigmoid $\sigma$ ko CDF na al'ada $\Phi$).

3.2. Rufe Bayanai & Haɗaɗɗun Bayanai

Wani muhimmin gudummawa shine kula da siffofi. Tsarin yana la'akari da Rukunan C na siffofi (misali, user_id, item_id, fasaha, ƙasa, lokaci). Kowane ƙima mai iyaka a cikin rukuni (misali, mai amfani=123, ƙasa='FR') ko ƙimar ci gaba da kanta ana kiranta mahaluƙi. Kowane daga cikin N mahaluƙi masu yuwuwa ana ba shi ƙarar haɗawa mai koyawa. Misali (misali, ɗalibi yana amsa kalma) ana rufe shi azaman ƙarar bayanai mara kyau $\mathbf{x}$ mai girman N, inda aka saita sassan zuwa 1 (don mahaluƙi masu iyaka da suke nan), ƙimar ainihi (don siffofi masu ci gaba), ko 0.

4. Aiwatarwa zuwa Aikin SLAM

4.1. Shirya Bayanai

Don aikin Duolingo SLAM, siffofi sun haɗa da ID mai amfani, abu na ƙamus (kalma), siffofinsa na harshe (sashin magana, ilimin halittar jiki), mahallin jumla, da bayanan lokaci. An canza waɗannan zuwa tsarin mara kyau na tushen mahaluƙi da DeepFM ke buƙata. Wannan rufewa yana ba da damar tsarin koyon hulɗa tsakanin kowane mahaluƙi biyu, kamar (mai amfani=Alice, kalma="ser") da (kalma="ser", lokaci=na baya).

4.2. Saitin Gwaji

An horar da tsarin don hasashen sakamako mai lamba biyu (daidai/ba daidai ba) don ɗalibi yana rubuta takamaiman kalma. An yi amfani da AUC (Yankin Ƙarƙashin Lankwasa ROC) a matsayin ma'aunin kimantawa na farko, daidaitaccen ma'auni don ayyukan rarrabuwa mai lamba biyu tare da bayanai marasa daidaituwa da suka zama ruwan dare a cikin saitunan ilimi.

5. Sakamako & Nazarin Aiki

Tsarin DeepFM ya sami gwajin AUC na 0.815. Wannan yana wakiltar ci gaba mai mahimmanci akan tushen koma baya na lissafi (AUC 0.774), yana nuna ƙimar tsarin hulɗar siffofi. Duk da haka, bai kai maki mafi girma na 0.861 ba. Takardar tana nuna cewa wannan yana bayyana "dabarun ban sha'awa don gina akan tsarin ka'idar amsar abubuwan tambaya," yana nuna cewa yayin da DeepFM ke ba da tsari mai ƙarfi, mai cike da siffofi, akwai wurin haɗa ƙarin ka'idar ilimi ko al'amuran tsarin jeri waɗanda tsarin mafi girma zai iya kama.

6. Nazari Mai Tsauri & Ra'ayoyin Kwararru

Babban Fahimta: Wannan takarda ba game da sabon algorithm mai ban mamaki ba ce, amma mai hikima, mai aiki tuƙuru aiwatar da tsarin tsarin shawara na masana'antu da ya wanzu (DeepFM) zuwa wani filin matsalar da ba a saba da shi ba: binciken ilimi mai zurfi, mai cike da siffofi. Matakin marubuci yana da ma'ana—sun ƙetare zagayowar tashin hankali na ilimi game da koyo mai zurfi kawai don ilimi (kamar DKT na farko) a maimakon haka sun sake amfani da tsarin da aka tabbatar a cikin kasuwanci na e-commerce don kama rikitarwar hulɗar mai amfani-abu-siffa. Ainihin fahimta ita ce tsara binciken ilimi ba kawai a matsayin matsalar hasashen jeri ba, amma a matsayin matsala ta hulɗar siffofi mai girma, mara kyau, kamar yadda ake hasashen dannawa a cikin talla.

Kwararar Hankali & Matsayin Dabarun: Hankali yana da ban sha'awa. 1) Tsare-tsare na gargajiya (IRT, BKT) suna da bayyani amma sun iyakance ga hulɗar da aka ƙayyade a baya, ƙananan girma. 2) Tsarin koyo mai zurfi na farko (DKT) yana kama jerin gwano amma yana iya zama mai ƙoshin bayanai kuma mara kyau, wani lokacin yana ƙasa da yin aiki kamar yadda Wilson et al. suka lura. 3) Aikin SLAM yana ba da tarin dukiya na bayanan gefe (siffofin harshe). 4) Saboda haka, yi amfani da tsarin da aka ƙera a fili don wannan: DeepFM, wanda ke haɗa ƙwaƙwalwar hulɗar ma'ana biyu-biyu (sashin FM, mai kama da hulɗar ɗalibi-abu na IRT) tare da ikon gama-gari na DNN. Takardar ta nuna da wayo yadda za a iya ganin IRT a matsayin wani yanayi na musamman, mai sauƙi na wannan tsarin, don haka yana da'awar babban matsayi na gaba ɗaya.

Ƙarfi & Kurakurai: Babban ƙarfin shine aiki tuƙuru da amfani da siffa. DeepFM tsari ne mai ƙarfi, na kasuwanci don amfani da cikakkiyar tsarin siffa na aikin SLAM. Kurakurarsa, kamar yadda sakamakon ya bayyana, shine cewa yana yiwuwa tsarin da suka fi kama yanayin lokaci na asali a cikin koyo ya fi shi. Tsarin tushen LSTM ko tsarin canzawa (kamar waɗanda aka yi amfani da su daga baya a cikin KT, misali, SAKT ko AKT) zai iya haɗa tarihin jeri da kyau. AUC na takardar na 0.815, duk da yake ingantacciyar ci gaba akan tushe, ya bar tazarar 0.046 zuwa ga wanda ya ci nasara—tazarar da wataƙila ta wakilci farashin da ba a ba da fifiko ga yanayin lokaci ba. Kamar yadda bincike daga Kalubalen AI na Riiid! da ayyuka daga baya suka nuna, haɗa tsare-tsare masu sanin siffa kamar DeepFM tare da ingantattun tsarin jeri shine hanyar cin nasara.

Fahimta Mai Aiki: Ga masu aiki da masu bincike: 1) Kada ku yi watsi da injiniyan siffa. Nasarar amfani da DeepFM ta jaddada cewa a cikin bayanan ilimi, "bayanin gefe" (alamomin fasaha, wahala, lokacin amsawa, siffofin harshe) sau da yawa shine babban bayanin. 2) Dubi filayen da ke kusa. Tsarin shawara sun shafe shekaru goma suna magance matsalolin makamancin haka na sanyin farawa, rashin kyau, da hulɗar siffa; kayan aikinsu (FM, DeepFM, DCN) ana iya canjawa kai tsaye. 3) Gaba shine gauraye. Mataki na gaba a bayyane yake: haɗa ikon hulɗar siffa na DeepFM tare da ingantaccen tsarin jeri. Ka yi tunanin "DeepFM na Lokaci" inda sashin zurfi shine LSTM ko Transformer wanda ke sarrafa jerin wakilcin hulɗar ma'ana. Wannan ya yi daidai da yanayin da aka gani a cikin ayyuka kamar "Cibiyar Juyin Halittar Sha'awa Mai Zurfi" (DIEN) a cikin talla, wanda ke haɗa hulɗar siffa tare da tsarin jeri na juyin halittar sha'awar mai amfani—kwatankwacin cikakke don juyin halittar ilimi.

7. Cikakkun Bayanai na Fasaha & Tsarin Lissafi

Tushen DeepFM yana cikin tsarinsa na sassa biyu. Bari shigarwar ta zama ƙarar siffa mara kyau $\mathbf{x} \in \mathbb{R}^n$.

Sashin Tsarin Ma'ana (FM):
$y_{FM} = w_0 + \sum_{i=1}^{n} w_i x_i + \sum_{i=1}^{n} \sum_{j=i+1}^{n} \langle \mathbf{v}_i, \mathbf{v}_j \rangle x_i x_j$
Anan, $w_0$ shine bambancin duniya, $w_i$ su ne ma'auni don sharuɗɗan layi, kuma $\mathbf{v}_i \in \mathbb{R}^k$ shine ƙarar ma'ana mai ɓoye don siffa ta i. Cikin samfurin $\langle \mathbf{v}_i, \mathbf{v}_j \rangle$ yana tsarin hulɗa tsakanin siffa $i$ da $j$. Ana ƙididdige wannan cikin sauƙi a cikin lokacin $O(kn)$.

Sashin Zurfi:
Bari $\mathbf{a}^{(0)} = [\mathbf{e}_1, \mathbf{e}_2, ..., \mathbf{e}_m]$ ya zama haɗakar ƙungiyoyin haɗawa don siffofin da ke cikin $\mathbf{x}$, inda ake duba $\mathbf{e}_i$ daga matrix haɗawa. Ana ciyar da wannan ta jerin yadudduka masu cikakken haɗin kai:
$\mathbf{a}^{(l+1)} = \sigma(\mathbf{W}^{(l)} \mathbf{a}^{(l)} + \mathbf{b}^{(l)})$
Sakamakon Layer na ƙarshe shine $y_{DNN}$.

Hasashen Ƙarshe:
$\hat{y} = \sigma(y_{FM} + y_{DNN})$
Ana horar da tsarin daga ƙarshe zuwa ƙarshe ta hanyar rage asarar giciye mai lamba biyu.

8. Tsarin Nazari & Misali na Ra'ayi

Yanayi: Hasashen ko Dalibi_42 zai fassara kalmar "was" (jigo: "be", lokaci: na baya) daidai a cikin darasin Sifen.

Mahaluƙin Siffa & Rufe:

• user_id=42 (Mai iyaka)
• word_lemma="be" (Mai iyaka)
• grammar_tense="past" (Mai iyaka)
previous_accuracy=0.85 (Ci gaba, daidaitacce)

Ƙarar shigarwa mara kyau $\mathbf{x}$ zai sami 1s a cikin wuraren da suka dace da mahaluƙi masu iyaka, ƙimar 0.85 don siffar ci gaba, da 0s a wasu wurare.

Fassarar Tsari:

• Sashin FM zai iya koyon cewa ma'aunin hulɗa $\langle \mathbf{v}_{user42}, \mathbf{v}_{tense:past} \rangle$ mara kyau ne, yana nuna Dalibi_42 yana fama da lokacin baya gaba ɗaya.
• A lokaci guda, zai iya koyon $\langle \mathbf{v}_{lemma:be}, \mathbf{v}_{tense:past} \rangle$ mara kyau ne sosai, yana nuna "be" a lokacin baya yana da wahala musamman ga duk ɗalibai.
• Sashin Zurfi zai iya koyon ƙa'ida mafi rikitarwa, mara layi: misali, babban previous_accuracy da aka haɗa da takamaiman tsarin kurakurai na baya akan fi'ili mara ka'ida yana daidaita hasashen ƙarshe, yana kama hulɗar mafi girma fiye da biyu-biyu.

Wannan yana nuna yadda DeepFM zai iya kama sauƙaƙan alaƙa masu bayyana (kamar IRT) da ƙa'idodi masu rikitarwa, marasa layi a lokaci guda.

9. Aiwatarwa na Gaba & Hanyoyin Bincike

Aiwatar da DeepFM zuwa binciken ilimi yana buɗe hanyoyi masu ban sha'awa da yawa:

1. Haɗawa da Tsarin Jeri: Madaidaicin ƙari shine haɗa yanayin lokaci. DeepFM zai iya zama injin hulɗar siffa a kowane lokaci, tare da ciyar da sakamakonsa cikin RNN ko Transformer don tsarin juyin halittar yanayin ilimi akan lokaci, yana haɗa ƙarfin tsare-tsare masu sanin siffa da masu sanin jerin gwano.
2. Shawarar Abun Ciki Na Musamman: Bayan hasashe, haɗaɗɗun bayanan da aka koya don masu amfani, fasaha, da abubuwan abun ciki na iya ƙarfafa ingantattun tsarin shawara a cikin dandamalin koyo masu daidaitawa, suna ba da shawarar motsa jiki na gaba mafi kyau ko albarkatun koyo.
3. Canja wurin Koyo Tsakanin Yankuna: Haɗaɗɗun bayanan mahaluƙi da aka koya daga bayanan koyon harshe (misali, haɗaɗɗun bayanai don ra'ayoyin nahawu) za a iya canjawa wuri ko daidaitawa don wasu yankuna kamar koyar da lissafi ko kimiyya, yana haɓaka haɓakar tsarin inda bayanai suka yi ƙaranci.
4. Bayanani & Shiga Tsakani: Duk da yake yana da bayyani fiye da DNN kawai, bayanin DeepFM har yanzu yana dogara ne akan abubuwan ɓoye. Aikin nan gaba zai iya mai da hankali kan haɓaka hanyoyin bayani na bayan haka don fassara hulɗar ma'ana zuwa fahimta mai aiki ga malamai (misali, "Dalibi yana fama musamman da hulɗar tsakanin muryar m da cikakken lokaci na baya").
5. Gwaji Mai Daidaitawa na Ainihi-Lokaci: Ingantaccen sashin FM yana sa ya dace da tsarin ainihin-lokaci. Za a iya tura shi a cikin yanayin gwaji na kwamfuta mai daidaitawa (CAT) don zaɓar tambaya ta gaba bisa ga ƙididdigar da aka sabunta akai-akai na iyawar ɗalibi da hulɗar siffa-abu.

10. Nassoshi

1. Corbett, A. T., & Anderson, J. R. (1994). Binciken ilimi: Tsarin samun ilimin tsari. Tsarin mai amfani da daidaitawar mai amfani, 4(4), 253-278.
2. Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L. J., & Sohl-Dickstein, J. (2015). Binciken ilimi mai zurfi. Ci gaba a cikin tsarin sarrafa bayanai na jijiya, 28.
3. Wilson, K. H., Karklin, Y., Han, B., & Ekanadham, C. (2016). Komawa ga tushe: Ƙarin Bayesian na IRT sun fi cibiyoyin jijiya don ƙididdige ƙwarewa. A Cikin Ma'adinan Bayanan Ilimi.
4. Cheng, H. T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., ... & Shah, H. (2016, Satumba). Koyo mai fadi & mai zurfi don tsarin shawara. A cikin Gabatarwar taron bita na 1 akan koyo mai zurfi don tsarin shawara (shafi na 7-10).
5. Guo, H., Tang, R., Ye, Y., Li, Z., & He, X. (2017). DeepFM: cibiyar jijiya mai tushen tsarin ma'ana don hasashen CTR. arXiv preprint arXiv:1703.04247.
6. Vie, J. J., & Kashima, H. (2018). Injunan Binciken Ilimi: Injunan Ma'ana don Binciken Ilimi. arXiv preprint arXiv:1811.03388.
7. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Tushen ka'idar amsar abu. Sage.
8. Settles, B., Brust, C., Gustafson, E., Hagiwara, M., & Madnani, N. (2018). Tsarin samun harshe na biyu. A cikin Gabatarwar Taron Bit na NAACL-HLT akan Amfani da Sabon Salo na NLP don Gina Ayyukan Ilimi.