Mena kua tukuna e koe tētahi tauira i tino miharo i roto i tētahi pukatuhi engari i hē te hanga, e mōhio ana koe ki te mea ngaro: te ine i te mahi AI . He pūnaha tirotiro e hono ana ki ngā whāinga o te ao tūturu. He ātaahua te tika. He pai ake te pono, te haumaru, me te pānga pakihi.
Ngā tuhinga ka pai pea koe ki te pānui i muri i tēnei:
🔗 Me pehea te korero ki a AI
He aratohu mō te whakawhitiwhiti kōrero whai hua me te AI mō ngā hua pai tonu.
🔗 He aha te AI akiaki
E whakamārama ana i te hanga a ngā akiaki i ngā urupare AI me te kounga putanga.
🔗 He aha te tapanga raraunga AI
He tirohanga whānui mō te tuku tapanga tika ki ngā raraunga mō ngā tauira whakangungu.
🔗 He aha te matatika AI
He kupu whakataki ki ngā mātāpono matatika e ārahi ana i te whakawhanaketanga me te whakatinanatanga o te AI haepapa.
He aha te mea e pai ai te mahi a te AI? ✅
Putanga poto: ko te mahi pai o te AI he tikanga he whai hua tō pūnaha, he pono, ā, ka taea te whakahoki i raro i ngā āhuatanga pōrearea, hurihuri. Mōhiotanga:
-
Te kounga o te mahi - ka whiwhi i ngā whakautu tika mō ngā take tika.
-
Whakatikatika - ka rite ngā kaute māia ki te mea tūturu, kia taea ai e koe te mahi mohio.
-
Pakari - ka mau ki te nekenekehanga, ki ngā taha o te whenua, me te pōuriuri o ngā hoariri.
-
Haumaru me te tika - ka karo i ngā whanonga kino, whakaaro pōhēhē, kore rānei e tutuki i ngā whakaritenga.
-
Whai huatanga - he tere, he iti te utu, ā, he pumau hei whakahaere i te tauine nui.
-
Pānga pakihi - ka nekehia te KPI e aro nui ana koe.
Mena kei te hiahia koe i tētahi tohutoro whaimana mō te whakataurite i ngā inenga me ngā mōrearea, he whetu raki pakari te Anga Whakahaere Mōrearea NIST AI
Ko te tunu taumata-tiketike mō te ine i te mahi a te AI 🍳
Whakaarohia kia toru ngā paparanga :
-
Ngā inenga mahi - te tika mō te momo mahi: te whakarōpūtanga, te whakatauira, te whakatauranga, te whakaputa, te whakahaere, me ētahi atu.
-
Ngā inenga pūnaha - te whakaroa, te nui o te mahi, te utu mō ia waeatanga, ngā reiti rahunga, ngā whakaoho nekehanga, ngā SLA wā mahi.
-
Ngā inenga putanga - ngā putanga pakihi me ngā putanga kaiwhakamahi e tino hiahiatia ana e koe: te tahuritanga, te pupuri, ngā aituā haumaru, te kawenga arotake-ā-ringa, te rahinga tīkiti.
Mā te mahere ine pai ka whakaranua ngā mea e toru. Ki te kore, ka whiwhi koe i tētahi rōkete e kore e wehe atu i te papa whakarewatanga.
Ngā inenga matua mā te momo raruraru - me te wā hei whakamahi i tēhea 🎯
1) Whakarōpūtanga
-
Te Tika, Te Whakamahara, F1 - te tokotoru o te rā tuatahi. Ko te F1 te toharite ōrite o te tika me te whakamahara; he mea whai hua ina kore e taurite ngā akomanga, ina kore rānei e ōrite ngā utu. [2]
-
ROC-AUC - te whakatauranga paepae-kore o ngā kaiwhakarōpū; mēnā he onge ngā hua pai, tirohia hoki te PR-AUC . [2]
-
Te tika taurite - te toharite o te maumahara puta noa i ngā akomanga; he pai mō ngā tapanga piko. [2]
Mātakitaki i te mahanga: ko te tika anake ka tino hē, ka kore hoki e taurite. Mena he 99% o ngā kaiwhakamahi he pono, ka whiwhi te tauira kuware-tika tonu i te 99%, ā, ka hinga i a rātou tō rōpū tinihanga i mua i te tina.
2) Te Hokinga Whakamuri
-
MAE mō ngā hapa e mārama ana ki te tangata; RMSE ina hiahia koe ki te whiu i ngā hapa nui; R² mō te rerekētanga kua whakamāramahia. Kātahi ka tirohia te hauora o ngā tohatoha me ngā kauwhata toenga. [2]
(Whakamahia ngā waeine e pai ana ki te rohe kia taea ai e ngā kaipupuri hea te kite i te hapa.)
3) Te whakarōpūtanga, te kimi, ngā taunakitanga
-
nDCG - e aro ana ki te tūranga me te hiranga o te whakatauranga; he paerewa mō te kounga rapu.
-
MRR - e aro ana ki te tere o te putanga mai o te tūemi tuatahi e tika ana (he pai mō ngā mahi "kimihia tētahi whakautu pai").
(Kei roto i ngā whare pukapuka ine matua ngā tohutoro whakatinanatanga me ngā tauira kua mahihia.) [2]
4) Te whakaputa tuhinga me te whakarāpopototanga
-
BLEU me ROUGE - ngā inenga taupatupatu matarohia; he mea whai hua hei tūāpapa.
-
o ngā inenga e hangai ana ki te whakaurunga (hei tauira, BERTScore ) ki te whakaaro tangata; me hono tonu ki ngā whakatauranga tangata mō te momo, te pono, me te haumaru. [4]
5) Whakautu pātai
-
te Exact Match me te tohu-taumata F1 mō ngā QA tango; ki te mea me whakahua ngā pūtake i ngā whakautu, me ine hoki i te tūāpapa (ngā tirotiro tautoko whakautu).
Te whakatikatika, te māia, me te arotahi Brier 🎚️
Kei roto i ngā kaute whakawhirinaki te wāhi e takoto puku ai ngā pūnaha maha. Me whai tūponotanga e whakaata ana i te mooni kia taea ai e ngā kaiwhakahaere te whakatakoto i ngā paepae, te huarahi ki te tangata, te mōrearea utu rānei.
-
Ngā ānau whakatikatika - whakaata i te tūponotanga i matapaetia me te auautanga ā-wheako.
-
Kaute Brier - he ture tatau tika mō te tika o te tūponotanga; he pai ake te iti iho. He tino whai hua ina aro nui koe ki te kounga o te tūponotanga, kaua ko te tūnga anake. [3]
Tuhipoka ā-marae: mā te whakatikatika F1 "kino ake" engari he pai ake ka tino pai ake te aromatawai - nā te mea ka taea e te tangata te whakawhirinaki ki ngā kaute.
Te haumaru, te whakaaro pōhēhē, me te tika - inehia te mea nui 🛡️⚖️
Ka taea e te pūnaha te tika whānui, engari ka kino tonu ki ētahi rōpū. Aroturukihia rōpū me ngā paearu tika:
-
Te ōritetanga o te taupori - ngā reiti pai ōrite puta noa i ngā rōpū.
-
Ngā tūponotanga ōrite / Ngā whai wāhitanga ōrite - ngā reiti hapa ōrite, ngā reiti pono-pai rānei puta noa i ngā rōpū; whakamahia ēnei hei kimi me te whakahaere i ngā taupatupatu, kaua hei tohu kotahi-whiu mō te pāhi-kore. [5]
Tohutohu whaihua: tīmata ki ngā papaaho e wehewehe ana i ngā inenga matua mā ngā huanga matua, kātahi ka tāpirihia ngā inenga tika motuhake e ai ki ngā whakaritenga a ō kaupapa here. He ahua uaua te ahua, engari he iti ake te utu i tētahi aituā.
Ngā LLM me te RAG - he pukapuka mahi ine e tino whai hua ana 📚🔍
He… kōpikopiko te ine i ngā pūnaha whakaputa uri. Me mahi tēnei:
-
Tautuhia ngā putanga mō ia whakamahinga: te tika, te āwhina, te kore kino, te piripono ki te momo kāhua, te reo e pā ana ki te waitohu, te pūtake o te tohutoro, te kounga o te whakakorenga.
-
Aunoatia ngā arotakenga tūāpapa me ngā anga pakari (hei tauira, ngā taputapu aromatawai i roto i tō puranga) me te pupuri kia rite ki ngā putanga o ō huinga raraunga.
-
Tāpirihia ngā inenga tikanga (e hangai ana ki te whakauru) me ngā inenga taupatupatu (BLEU/ROUGE) kia mārama ai. [4]
-
Te tūāpapa o ngā taputapu i roto i te RAG: te tere o te tiki, te tika/te maumahara o te horopaki, te taupatupatu o ngā whakautu me te tautoko.
-
Arotake tangata me te whakaae - ine i te ōritetanga o te kaiarotake (hei tauira, te κ a Cohen, te κ a Fleiss rānei) kia kore ai ō tapanga e rite ki te āhua.
Tāpiri: ngā ōrau whakaroa takiuru me te utu tohu, utu rorohiko rānei mō ia mahi. Kāore he tangata e pai ki te whakautu rotarota ka tae mai i te Tūrei e tū mai nei.
Te ripanga whakatairite - ngā taputapu hei āwhina i a koe ki te ine i te mahi a te AI 🛠️📊
(Āe, he āhua pōrearea te āhua - he pōrearea ngā nota tūturu.)
| Utauta | Te hunga whakarongo pai rawa atu | Utu | He aha i whai hua ai - he tere te tango |
|---|---|---|---|
| ngā inenga ako-scikit | Ngā kaimahi ML | Koreutu | Ngā whakatinanatanga ā-ture mō te whakarōpūtanga, te whakatauira, te whakatauranga; he māmā te whakauru ki ngā whakamātautau. [2] |
| Arotake MLflow / GenAI | Ngā kaipūtaiao raraunga, MLOps | Kore utu + utu | Ngā oma pokapū, ngā inenga aunoa, ngā kaiwhakawā LLM, ngā kaitatau kaute ritenga; ka tuhi marie i ngā taonga. |
| Te āhua nei | Ngā tīma e hiahia ana kia tere te hanga papaaho | OSS + kapua | Neke atu i te 100 ngā inenga, ngā pūrongo kounga me te nekehanga, ngā matau aroturuki - he ataahua ngā tirohanga ina hiahiatia. |
| Ngā Taumaha me ngā Pānga | Ngā whakahaere e nui ana te whakamātautau | Taumata kore utu | Ngā whakatairite taha-ki-taha, ngā huinga raraunga arotake, ngā kaiwhakawā; he maeneene ngā ripanga me ngā tohu. |
| LangSmith | Ngā kaihanga taupānga LLM | Utu | Whaia ia taahiraa, whakaranua te arotake tangata me ngā kaiaromatawai ture, LLM rānei; he pai mō te RAG. |
| TruLens | Ngā kaiwhaiwhai arotake LLM pūtake tuwhera | OSS | Ko ngā mahi urupare hei whakatau i te paitini, te tū pakari, te hiranga; hei whakauru ki hea noa atu. |
| Ngā Tumanako Nui | Ngā whakahaere e whakatairanga ana i te kounga raraunga | OSS | Whakapūmautia ngā tumanakohanga mō ngā raraunga - nā te mea ka pakaru katoa ngā inenga i ngā raraunga kino. |
| Ngā Tirotiro Hōhonu | Te whakamātautau me te CI/CD mō te ML | OSS + kapua | Ngā whakamātautau kei roto i ngā pākahiko mō te nekenekehanga raraunga, ngā take tauira, me te aroturukitanga; he pai ngā ārai ārai. |
Ka rerekē ngā utu - tirohia ngā tuhinga. Āe, ka taea e koe te whakaranu i ēnei me te kore e puta mai ngā pirihimana taputapu.
Ngā paepae, ngā utu, me ngā kōpiko whakatau - te mea ngaro 🧪
He mea rerekē engari he mea pono: e rua ngā tauira me te ROC-AUC kotahi, he rerekē rawa te uara pakihi i runga i tō paepae me ngā ōwehenga utu .
He pepa tere hei hanga:
-
Whakatakotoria te utu mō te hua pai rūpahu me te hua kino rūpahu i roto i te moni, i te wā rānei.
-
Tātarihia ngā paepae me te tatau i te utu e tumanakohia ana mō ia 1k whakataunga.
-
Kōwhiria te utu iti rawa e tumanakohia ana , kātahi ka maukati mā te aroturuki.
Whakamahia ngā kōpiko PR ina he onge ngā hua pai, ngā kōpiko ROC mō te āhua whānui, me ngā kōpiko whakatikatika ina whakawhirinaki ngā whakataunga ki ngā tūponotanga. [2][3]
Mini-case: he tauira whakarōpūtanga tīkiti tautoko me te F1 iti engari he tino pai te whakatikatika, ka tapahia ngā ara ā-ringa i muri i te whakawhiti o ngā mahi mai i te paepae mārō ki te ara taumata (hei tauira, "whakatau-aunoa," "arotake-tangata," "whakapiki") e herea ana ki ngā roopu kaute kua whakatikatikaina.
Te aroturuki ipurangi, te nekeneke, me te whakatūpato 🚨
Ko ngā arotake tuimotu te tīmatanga, ehara i te mutunga. I roto i te whakaputanga:
-
Aroturuki i te rerekētanga tāuru , te rerekētanga putanga , me te hekenga mahi mā te wāhanga.
-
Whakatakotoria ngā arowhai ārai - te tere mōrahi o te moemoeā, ngā paepae paitini, ngā taumata ōrite o te māramatanga.
-
Tāpirihia ngā papaaho Canary mō te whakaroa o te p95, ngā wā whakamutu, me te utu mō ia tono.
-
Whakamahia ngā whare pukapuka i hangaia mō tētahi kaupapa motuhake hei tere ake i tēnei; ka tukuna e rātou ngā momo putanga hou, kounga, me ngā momo aroturuki taketake.
He kupu whakarite iti nei he hapa: whakaarohia tō tauira he rite ki te tīmatanga o te paraoa rewena - kāore koe e tunu kotahi noa iho ka haere atu; ka whāngai koe, ka mātakitaki, ka hongi, ā, i ētahi wā ka tīmata anō.
Aromatawai tangata e kore e pakaru 🍪
Ina whakamākahia ngā putanga e te tangata, he mea nui ake te tukanga i tō whakaaro.
-
Tuhia ngā aratohu matatau me ngā tauira mō te eke panuku, te eke hōhonu, me te eke kore angitu.
-
Whakarōpūtia ngā tauira matapōkeretia, ā, whakamahia he matapo ina taea.
-
Inehia te whakaae a ngā kaitātari (hei tauira, ko te κ a Cohen mō ngā kaitātari e rua, ko te κ a Fleiss mō te tokomaha) ka whakahou i ngā tūtohi mēnā ka hē te whakaae.
Mā tēnei ka kore e rere ngā tapanga tangata i runga i te wairua, i te nui rānei o te kawhe.
Ruku hohonu: me pēhea te ine i te mahi AI mō ngā LLM i roto i te RAG 🧩
-
Te kounga o te whakahokinga mai - te whakahokinga mai, te tika o te mea, te nDCG; te kapinga o ngā meka koura. [2]
-
Te pono o ngā whakautu - ngā arowhai whakahua-me-manatoko, ngā kaute tūāpapa, ngā rangahau tautohe.
-
Te pai o te kaiwhakamahi - ngā painga, te otinga o te mahi, te tawhiti whakatika mai i ngā tauira kua tūtohutia.
-
Haumaru - paitini, turuturu PII, tutuki i ngā kaupapa here.
-
Te utu me te whakaroa - ngā tohu, ngā pānga keteroki, ngā whakaroa p95 me te p99.
Honoa ēnei ki ngā mahi pakihi: ki te heke te tūnga whenua ki raro i te rārangi, hurihia aunoa ki te aratau tino pakari, ki te arotake tangata rānei.
He pukapuka tākaro māmā hei tīmatanga i tēnei rā 🪄
-
Tautuhia te mahi - tuhia tētahi rerenga kōrero: he aha te mahi a te AI, ā, mā wai hoki.
-
Kōwhiria kia 2–3 ngā inenga mahi - me te whakataurite me te iti rawa kia kotahi te wāhanga tika. [2][3][5]
-
Whakatauhia ngā paepae mā te whakamahi i te utu - kaua e whakaaro noa.
-
Waihangahia he huinga aromatawai iti - 100–500 tauira kua tohua e whakaata ana i te ranunga whakaputa.
-
Aunoatia ō arotakenga - te aromatawai/aroturuki waea ki roto i te CI kia rite tonu te whakahaere o ia huringa.
-
Aroturuki i roto i ngā haki hua - te nekenekehanga, te whakaroa, te utu, ngā haki aituā.
-
Arotakehia ia marama - tapahia ngā inenga kāore e whakamahia ana e tetahi; tāpirihia ngā inenga e whakautu ana i ngā pātai tūturu.
-
Tuhia ngā whakataunga - he kāri kaute ora ka pānuihia e tō tīma.
Āe, koinā tonu. Ā, e whai hua ana.
Ngā hapa noa me pēhea te karo i aua hapa 🕳️🐇
-
Te tāpiri atu ki tētahi inenga kotahi - whakamahia he kete inenga e ōrite ana ki te horopaki whakatau. [1][2]
-
Te kore aro ki te whakatikatika - ko te maia me te kore whakatikatika he whakapehapeha noa iho. [3]
-
Kāore he wehewehenga - me wehewehe tonu mā ngā rōpū kaiwhakamahi, whenua, taputapu, reo. [5]
-
Ngā utu kāore i tautuhia - ki te kore koe e whakatakoto hapa utu, ka whiriwhiria e koe te paepae hē.
-
Te rerekētanga o te aromatawai tangata - te ine i te whakaae, te whakahou i ngā paerewa, te whakangungu anō i ngā kaiarotake.
-
Kāore he taputapu haumaru - tāpirihia te tika, te paitini, me ngā arowhai kaupapa here ināianei, kaua i muri mai. [1][5]
Ko te kīanga i haere mai koe mō: me pēhea te ine i te mahi AI - te Roa Rawa, Kāore Au i Pānui 🧾
-
Tīmata ki ngā putanga mārama , kātahi ka whakarārangihia mahi , pūnaha , me te pakihi . [1]
-
Whakamahia ngā inenga tika mō te mahi - F1 me ROC-AUC mō te whakarōpūtanga; nDCG/MRR mō te whakatauranga; taupatupatu + inenga tikanga mō te whakaputa (kua honoa ki te tangata). [2][4]
-
Whakatauritehia ō tūponotanga me te utu i ō hapa hei whiriwhiri i ngā paepae. [2][3]
-
Tāpirihia tika me ngā wāhanga rōpū, ā, whakahaeretia ngā taupatupatu kia mārama. [5]
-
Aunoatia ngā arotakenga me ngā aroturukitanga kia taea ai e koe te mahi anō me te kore e mataku.
E mōhio ana koe ki te āhua - inehia te mea nui, ki te kore ka whakapai ake koe i te mea kāore e nui.
Ngā Tohutoro
[1] NIST. Anga Whakahaere Mōrearea AI (AI RMF). pānuihia atu
[2] scikit-learn. Aromatawai Tauira: te ine i te kounga o ngā matapae (Aratohu Kaiwhakamahi). pānuihia atu
[3] scikit-learn. Te Whakataurite Tūponotanga (ngā ānau whakataurite, kaute Brier). pānuihia atu
[4] Papineni et al. (2002). BLEU: he Tikanga mō te Aromatawai Aunoa o te Whakamāoritanga Mīhini. ACL. pānuihia atu
[5] Hardt, Price, Srebro (2016). Te Ōritetanga o te Whai Wāhitanga i roto i te Ako e Aroturukihia ana. NeurIPS. pānuihia atu