Whakautu poto: Hei arotau i ngā tauira AI, whiriwhiria tētahi herenga matua (whakaroa, utu, mahara, kounga, pumau, putanga rānei), kātahi ka hopu i tētahi turanga pono i mua i te whakarerekē i tētahi mea. Tangohia ngā aukati paipa i te tuatahi, kātahi ka whakamahia ngā painga iti-mōrearea pērā i te tika whakauru me te puranga; ki te mau tonu te kounga, neke atu ki te taputapu whakahiato/wā-oma, kātahi ka whakaitihia te rahi o te tauira mā te ine ine, te whakatōtō rānei ina hiahiatia.
Ngā kōrero matua hei whakaaroaro:
Herenga : Kōwhiria kia kotahi, kia rua rānei ngā inenga whāinga; he whenua taupatupatu te arotautanga, ehara i te wikitoria kore utu.
Ine : Tātarihia ngā kawenga mahi tūturu me te p50/p95/p99, te putanga mahi, te whakamahinga, me ngā tihi mahara.
Paipa : Whakatikahia te tohu, ngā uta raraunga, te tukatuka-mua, me te puranga i mua i te pā ki te tauira.
Te Tuku : Whakamahia te pupuri raraunga, te puranga ā-tauira, te whakatikatika i te wā kotahi, me te aroturuki tonu i te whakaroa o te hiku.
Ngā Ārai : Whakahaerehia ngā tohutoro koura, ngā inenga mahi, me ngā arowhai ā-wāhi i muri i ia huringa mahi.
🔗 Me pēhea te aromatawai tika i ngā tauira AI
Ngā paearu matua me ngā mahi hei aromatawai tika me te pono i ngā tauira.
🔗 Me pēhea te ine i te mahi a te AI me ngā inenga tūturu
Whakamahia ngā tohu paearu, te whakaroa, te utu, me ngā tohu kounga hei whakatairite.
🔗 Me pēhea te whakamātautau i ngā tauira AI i mua i te whakaputanga.
Te rere mahi whakamātautau mahi: ngā wehewehenga raraunga, ngā take ahotea, me te aroturuki.
🔗 Me pēhea te whakamahi i te AI mō te waihanga ihirangi
Hurihia ngā whakaaro hei tauira tere ake mā te whakamahi i ngā tohutohu hanganga me te whakahou.
1) Te Tikanga o te "Whakaarotau" i roto i te Mahi (Nā te mea he rerekē te whakamahinga a te katoa) 🧠
Ina kī te tangata "arotau i tētahi tauira AI," ko tā rātou tikanga pea ko:
-
Kia tere ake (kia iti ake te whakaroa)
-
Me whakaiti te utu (he iti ake ngā hāora GPU, he iti ake te whakapaunga kapua)
-
Whakaitihia (te tapuwae mahara, te whakatinanatanga o te taha)
-
Kia tika ake (kia pai ake te kounga, kia iti ake ngā moemoeā)
-
Kia pumau ake (kia iti ake te rerekētanga, kia iti ake ngā hapa i roto i te whakaputanga)
-
Me ngāwari ake te tuku (te nui o te mahi, te puranga, te mahinga matapae)
Anei te pono e ahua whakarihariha ana: kāore e taea te whakanui i ēnei katoa i te wā kotahi. He rite te arotau ki te pēhi i tētahi poihau - panaia tētahi taha ki roto, ka puta mai tētahi atu taha. Ehara i te mea i ngā wā katoa, engari he maha ngā wā ka whakamahere koe mō ngā taupatupatu.
Nō reira, i mua i te pā atu ki tētahi mea, whiriwhiria tō herenga matua :
-
Mena kei te mahi ora koe ki ngā kaiwhakamahi, he mea nui ki a koe te whakaroa p95 ( ngā ōrau AWS CloudWatch ) me te mahi hiku ( te mahi pai "whakaroa hiku" ) 📉
-
Mena kei te whakangungu koe, he mea nui ki a koe te wā-ki-te-kounga me te whakamahinga o te GPU 🔥
-
Mena kei te whakamahi koe i ngā taputapu, he mea nui ki a koe te RAM me te mana 🔋
2) He aha te āhua o te putanga pai o te arotautanga tauira AI ✅
Ko te putanga pai o te arotautanga ehara i te "whakamahi i te ine me te inoi" anake. He pūnaha. Ko ngā whakaritenga pai rawa atu:
-
He tūāpapa e whakawhirinaki ana koe
Ki te kore e taea e koe te whakaputa anō i ō hua o nāianei, kāore koe e mōhio kua whakapai ake koe i tetahi mea. He māmā noa iho… engari ka pekehia e te tangata. Kātahi ka huripokitia. -
te inenga whāinga
"Tere ake". He whāinga tūturu te "Whakaitihia te whakaroa p95 mai i te 900ms ki te 300ms i te kaute kounga ōrite". -
Ngā ārai mō te kounga.
I ia wikitoria mahi, ka tūpono te hekenga o te kounga. Me whai whakamātautau, me arotake, me whai taputapu hauora rānei koe. -
Mōhiotanga taputapu
Ka taea e te tauira "tere" i runga i tētahi GPU te ngoki i runga i tētahi atu. He momo pōrearea motuhake anō ō ngā CPU. -
Ngā huringa auau, ehara i te tuhi anō nui.
Ina huri koe i ngā mea e rima i te wā kotahi, ā, ka pai ake te mahi, kāore koe e mōhio he aha. He mea… whakararuraru tēnei.
Me rite te arotautanga ki te whakatā i tētahi kitā - he iti noa ngā whakarerekētanga, whakarongo pai, tāruatia 🎸. Ki te rite koe ki te tākaro maripi, he hapa kei roto.
3) Ripanga Whakataurite: Ngā Kōwhiringa Rongonui hei Arotau i ngā Tauira AI 📊
Kei raro nei tētahi ripanga whakatairite tere, ā, he āhua pōrearea hoki, o ngā taputapu/huarahi arotau noa. Kāo, ehara i te mea "tika" - kāore hoki te ao tūturu i te pērā.
| Utauta / Kōwhiringa | hunga whakarongo | Utu | He aha i mahi ai |
|---|---|---|---|
PyTorch torch.compile ( tuhinga PyTorch ) |
E te iwi PyTorch | Koreutu | Ka taea e te hopu kauwhata me ngā mahi tinihanga a te kaiwhakaemi te whakaiti i te utu… i ētahi wā he makutu ✨ |
| Wā Whakahaere ONNX ( ngā tuhinga Wā Whakahaere ONNX ) | Ngā tīma whakatinanatanga | Āhua kore utu | He kaha te arotautanga o ngā whakatau, he whānui te tautoko, he pai mō te tuku paerewa |
| TensorRT ( ngā tuhinga NVIDIA TensorRT ) | Te whakatinanatanga o NVIDIA | Ngā kare ā-roto utu (he maha ngā wā ka tāpirihia) | He whakakotahitanga kernel kaha + he whakahaere tika, he tino tere ina pāwhiritia |
| Tere Hōhonu ( tuhinga ZeRO ) | Ngā tīma whakangungu | Koreutu | Ngā whakapainga mahara + te tukunga raraunga (ZeRO me ētahi atu). Ka rite ki te miihini rererangi |
| FSDP (PyTorch) ( Ngā tuhinga FSDP a PyTorch ) | Ngā tīma whakangungu | Koreutu | Ngā tawhā/ngā whakarōpūtanga maramara, ka iti ake te whakamataku o ngā tauira nunui |
| te ine i ngā bitsandbytes ( bitsandbytes ) | Ngā kaimahi hangarau LLM | Koreutu | He iti te taumaha o te moka, he nui te penapena mahara - kei te kounga te mea nui, engari whew 😬 |
| Te Whakatōtō ( Hinton et al., 2015 ) | Ngā tīma hua | "Te utu wā" | Ka riro i te tauira ākonga iti ake ngā whanonga, ko te ROI pai rawa atu mō te wā roa |
| Te tapahi ( akoranga tapahi PyTorch ) | Rangahau + hua | Koreutu | Ka tango i te taumaha mate. Ka pai ake te mahi ina honoa ki te whakangungu anō |
| Arotahi Maamaa / kākano hanumi ( pepa Arotahi Maamaa ) | Ngā tohunga mahi toi | Koreutu | He tere ake te aro, he pai ake te whanonga mahara. He wikitoria tūturu mō ngā transformers |
| Tūmau Whakatau Triton ( Puranga Hihiri ) | Ngā mahi/whakararo | Koreutu | Te mahi whakaputa, te puranga, ngā paipa maha-tauira - he āhua hinonga |
Whakaae mō ngā āhuatanga rerekē o te hōputu: He pōrearea te "Utu" nā te mea ka taea tonu e te pūmanawa tuwhera te utu mō te wiki mutunga o te rapu hapa, he utu nui tēnā. 😵💫
4) Tīmata ki te Ine: Whakamāramahia tō whakaaro kia rite ki tō tikanga 🔍
Ki te kotahi noa te mea ka mahia e koe i tēnei aratohu katoa, me mahi tēnei: me ine tika.
I roto i aku ake whakamātautau, ko ngā "huarahi hou mō te arotautanga" i ahu mai i te kitenga i tētahi mea māmā noa iho pēnei i:
-
te uta raraunga e matekai ana te GPU
-
Te aukati i te tukatuka-mua o te CPU
-
te iti o ngā rahinga puranga e hua ai te nui o te whakarewatanga o te kernel
-
puhoi te tuku tohu (ka taea e ngā kaiwhakauru tohu te noho hei tangata kino puku)
-
te wehewehenga o te mahara ( ngā tuhipoka tohatoha mahara CUDA a PyTorch )
-
he rorohiko papa kotahi e whakahaere ana i te mana
Ngā mea hei ine (tautuhinga iti rawa)
-
Te Whakaroa (p50, p95, p99) ( SRE i runga i ngā ōrau whakaroa )
-
Te rere o te raraunga (tohu/hēkona, tono/hēkona)
-
Te whakamahinga o te GPU (rorohiko + mahara)
-
Ngā tihi VRAM / RAM
-
Te utu mō ia tohu 1k (mō ia whakatau rānei)
Te whakaaro whaihua mō te whakatauira
-
Tuhia tētahi horopaki e aro nui ana koe (kaua he tohutohu taonga takaro).
-
Tuhia ngā mea katoa ki roto i tētahi "pukapuka mahi tino pai" iti nei.
Ae, he hoha... engari ka ārai i a koe i te whakamātau nui i a koe anō i muri mai.
(Mēnā kei te hiahia koe i tētahi taputapu tūturu hei tīmatanga: ko PyTorch Profiler ( torch.profiler docs ) me Nsight Systems ( NVIDIA Nsight Systems ) ngā mea e tino whakaarohia ana.)
5) Arotautanga Raraunga + Whakangungu: Te Mana Nui Mārire 📦🚀
Ka aro nui te tangata ki te hoahoanga tauira, ā, ka wareware ki te paipa hiko. I taua wā, ka tahuna marietia e te paipa hiko te haurua o te GPU.
Ngā wikitoria ngāwari e tere ana te puta
-
Whakamahia te tino tika (FP16/BF16 mēnā he pumau) ( PyTorch AMP / torch.amp )
He tere ake te nuinga, he pai tonu - engari kia tūpato ki ngā āhuatanga rerekē o te tau. -
Te kohikohinga o ngā rōnaki ina iti te rahi o te puranga ( 🤗 Aratohu whakateretere )
Ka noho pumau te arotautanga me te kore e pahū te mahara. -
Te arowhai i te rōnaki ( torch.utils.checkpoint )
Ka whakawhiti i te tatau mō te mahara - ka taea ai ngā horopaki nui ake. -
Te tuku tohu whai hua ( 🤗 Ngā Kaiwhakaatu Tohu )
Ka taea e te tuku tohu te noho hei arai i te tauine. Ehara i te mea ataahua; he mea nui. -
Te whakahāngai i te uta raraunga
He maha atu ngā kaimahi, te mahara kua pinea, te tiki-mua - kāore i te whakaatu engari he whai hua 😴➡️💪 ( Aratohu Whakahāngai Mahi PyTorch )
Te whakatikatika pai-tawhā
Mena kei te whakatikatika koe i ngā tauira nunui, ka taea e ngā tikanga PEFT (pēnei i ngā urutau momo LoRA) te whakaiti nui i te utu whakangungu me te noho kaha tonu ( 🤗 Aratohu PEFT Transformers , pepa LoRA ). Koinei tētahi o ngā wā "he aha tātou i kore ai e mahi i tēnei i mua ake nei?".
6) Arotautanga Taumata Hanganga: Whakatikahia te Rahi o te Tauira 🧩
I ētahi wā ko te huarahi pai ki te arotau ko… te whakamutu i te whakamahi i tētahi tauira he nui rawa mō te mahi. E mōhio ana ahau, he mahi whakahāwea 😄.
Waea atu mō ētahi kaupapa matua:
-
Whakatauhia mēnā me whai pūkenga whānui koe, me whai tohunga rānei.
-
Kia nui te matapihi horopaki e tika ana, kaua e rahi rawa.
-
Whakamahia he tauira kua whakangungua mō te mahi e mahi ana (ngā tauira whakarōpūtanga mō ngā mahi whakarōpūtanga, me ētahi atu).
Ngā rautaki whai hua mō te whakatikatika rahi
-
Whakawhiti ki tētahi tauira iti ake mō te nuinga o ngā tono.
Kātahi ka tukuna ngā "uiui uaua" ki tētahi tauira nui ake. -
Whakamahia he tatūnga e rua ngā wāhanga.
He tere te tauira tauira, he kaha ake te manatoko tauira, te whakatika rānei.
He rite ki te tuhituhi tahi me tētahi hoa he whiriwhiri - he hoha, engari he whai hua. -
Whakaitihia te roa o te putanga
He nui te moni me te wā e pau ana ki ngā tohu putanga. Ki te kore e pai te haere o tō tauira, ka utua e koe te utu mō te haere.
Kua kite ahau i ngā tīma e whakaheke ana i ngā utu mā te whakapoto i ngā putanga. He ahua iti noa iho. He pai te mahi.
7) Ngā Arotautanga o te Kaihanga + Kauwhata: Nō hea te Tere 🏎️
Koinei te paparanga "kia atamai ake te mahi a te rorohiko".
Ngā tikanga noa:
-
Hononga kaiwhakahaere (whakakotahi i ngā kernel) ( NVIDIA TensorRT "hononga paparanga" )
-
Te takai pumau (ngā uara pumau kua tatauhia i mua) ( Ngā arotautanga kauwhata ONNX Runtime )
-
te kōwhiringa kernel ki te taputapu
-
Te hopu kauwhata hei whakaiti i te utu nui o Python (
torch.compileoverview )
He māmā noa iho: tera pea he tere te pāngarau o tō tauira, engari he puhoi te mahi. Ka whakatikahia ētahi o ēnā e ngā kaiwhakaemi.
Ngā tuhipoka whai hua (arā, ngā koha)
-
Ka pāngia pea ēnei whakapainga e ngā huringa āhua o te tauira.
-
He tere rawa ētahi tauira, ko ētahi he paku nekeneke noa iho.
-
I ētahi wā ka tere ake te oma, ā, ka pōhēhē hoki - pērā i te gremlin kua nekehia mai 🧌
Heoi anō, ina mahi, ko tētahi o ngā wikitoria tino pai.
8) Te Whakawhānui, te Tapahi, te Whakatōtō: Iti ake, kāore e tangi (Nui rawa) 🪓📉
Koinei te wāhanga e hiahiatia ana e te tangata… nā te mea he rite ki te mahi kore utu. Tērā pea, engari me mahi pērā i te pokanga.
Te ine ine (ngā taumaha/whakahohenga iti iho te tika)
-
He tino pai mō te tere whakatau me te mahara
-
Mōrearea: ka heke te kounga, inā koa i ngā āhuatanga taha
-
Te mahi pai: aromatawaihia i runga i tētahi huinga whakamātautau tūturu, kaua i runga i ngā kare ā-roto
Ngā reka noa ka rangona e koe:
-
INT8 (he maha ngā wā he totoka) ( Ngā momo TensorRT kua inehia )
-
INT4 / iti-moka (he penapena nui, he piki ake te mōrearea kounga) ( te ine inenga k-moka bitsandbytes )
-
Te rahinga whakauru (kāore e hiahiatia ana te tika ōrite mō ngā mea katoa)
Te tapahi (tango i ngā tawhā)
-
Ka tango i ngā taumaha, i ngā hanganga rānei "kāore i te tino hira" ( akoranga tapahi PyTorch )
-
I te nuinga o te wā me whakangungu anō kia ora ai te kounga
-
He pai ake te mahi i tā te tangata e whakaaro ana… ina oti pai
Te whakatōtō (ka ako te ākonga i te kaiako)
Koinei taku tino pai mō te whakamahi i te rīwhi mō te wā roa. Ka taea e te whakatōtō te whakaputa i tētahi tauira iti ake e mahi ana i te āhua ōrite, ā, he maha ngā wā he pumau ake i te whakatōtōtanga nui rawa ( Te Whakatōtō i te Mātauranga i roto i te Whatunga Io ).
He kupu whakarite kāore i te tino tika: he rite te whakatō wai ki te ringihia he hupa uaua mā roto i te tātari, ā, ka puta he… hupa iti ake. Ehara i te mea koinā te mahi a te hupa, engari kua mārama koe ki te whakaaro 🍲.
9) Te Mahi me te Whakatau: Te Rohe Pakanga Tuturu 🧯
Ka taea e koe te "arotau" i tētahi tauira, ā, kāore tonu e pai te mahi. Ko te mahi te wāhi e tino kitea ai te roa me te utu.
He mea nui te wikitoria i te mahi tuku
-
te puranga
i te tere o te mahi. Engari ka whakanui ake i te roa o te mahi mēnā ka nui rawa. Tauritehia. ( Te puranga hihiri a Triton ) -
Te Kete I te
Kete i te Prompt me te whakamahi anō i te KV-cache ka nui te mahi mō ngā horopaki e whakahuatia ana. ( He whakamārama mō te KV cache ) -
Putanga rerema
Ahakoa he rite te roa o te wā, e whakaaro ana ngā kaiwhakamahi he tere ake. He mea nui te tirohanga 🙂. -
Te whakaiti i te utu mō ia tohu
Ka nui ake te mahi a ētahi puranga mō ia tohu. Whakaitihia taua utu, ā, ka nui tō wikitoria.
Kia tūpato ki te whakaroa o te hiku
Tērā pea ka ātaahua tō toharite i te wā e pōrearea ana tō p99. Engari, kei te hiku tonu ngā kaiwhakamahi. (Te "Tail latency" me te take e takoto ai ngā toharite )
10) Te Arotau i te Pūmārō: Whakatauritehia te Tauira ki te Mīhini 🧰🖥️
He rite te arotau i te tere o te waka reihi me te kore e tirotiro i ngā potae, ki te whakatikatika i te pūnaha me te kore e tirotiro i ngā potae. Ae, ka taea e koe, engari he ahua kuware.
Ngā whakaaro mō te GPU
-
Ko te whānui o te mahara te mea e whakawhāiti ana i te nuinga o te wā, ehara i te rorohiko mata
-
Ka taea e ngā rahinga puranga nui ake te āwhina, kia kore rā anō e āwhina
-
He mea nui te whakakotahitanga o te kernel me te arotautanga mō ngā transformers ( FlashAttention: IO-aware exact attention )
Ngā whakaaro mō te CPU
-
He mea nui te miro, te whakapūtātanga, me te rohe mahara
-
Ka taea e te utu nui mō te tokenization te whakahaere ( 🤗 Ngā tokenizers "Tere" )
-
Tērā pea me rerekē ngā rautaki ine ine i tō te GPU
Ngā whakaaro mō te taha/pūkoro
-
Ko te tapuwae mahara te mea nui rawa atu
-
He mea nui te rerekētanga o te whakaroa nā te mea he… pōuri ngā taputapu
-
He maha ngā wā ka hinga ake ngā tauira iti, motuhake i ngā tauira whānui nunui
11) Ngā Ārai Kounga: Kaua e "Whakapaipai" i a Koe Hei Pūngāwerewere 🧪
Me haere tahi mai me te tirotiro kounga ia wikitoria tere. Ki te kore ka whakanui koe, ka tautoko, kātahi ka puta he karere pēnei i te "he aha te kaiawhina i kōrero ohorere ai pērā i te pāhihi?" 🏴☠️
Ngā ārai whaihua:
-
Ngā tohu koura (huinga tohu pumau ka whakamatautauria tonutia e koe)
-
Ngā inenga mahi (tika, F1, BLEU, te mea e tika ana)
-
Ngā tirotiro ā-tangata (āe, tino pono)
-
Ngā paepae whakamuri ("kāore e whakaaetia kia neke atu i te X% te hekenga")
Aroturukihia hoki ngā aratau rahunga:
-
te nekeneke hōputu
-
ngā huringa whanonga whakakorekore
-
auau o te moemoeā
-
te pikinga roa o te urupare
Ka taea e te arotautanga te whakarerekē i ngā whanonga i ngā huarahi whakamiharo. He mea motuhake. He mea whakapataritari. E ai ki te tirohanga whakamuri, e taea ana te matapae.
12) Rārangi Arowhai: Me pēhea te Arotau i ngā Tauira AI, Hipanga-i-te-Hipanga ✅🤖
Ki te hiahia koe ki tētahi raupapa mārama o ngā mahi mō te Me Pēhea te Arotau i ngā Tauira AI , anei te tukanga mahi hei pupuri i te tangata kia noho mārire:
-
Tautuhia te angitu
Kōwhiria kia 1-2 ngā inenga matua (te whakaroa, te utu, te whakaputanga, te kounga). -
Inehia te tūāpapa. Tātaihia
ngā kawenga mahi tūturu, tuhia te p50/p95, te mahara, te utu. ( PyTorch Profiler ) -
Whakatikahia ngā arai o ngā paipa
Te uta raraunga, te tohu, te tukatuka i mua, te puranga. -
Whakamahia ngā wikitoria rorohiko iti-mōrearea.
He tika whakaranu, he arotautanga kernel, he puranga pai ake. -
Whakamātauria ngā arotautanga o te kaikohikohi/wā whakahaere:
te hopu kauwhata, ngā wā whakahaere whakatau, te whakakotahitanga kaiwhakahaere. ( akorangatorch.compile, ngā tuhinga ONNX Runtime ) -
Whakaitihia te utu mō te tauira.
Tātarihia marietia, whakaranua mēnā ka taea, tapatapahia mēnā e tika ana. -
Te pupuri ratonga
rangitahi, te whakamātau kawenga, ngā whakatikatika whakaroa hiku. -
Whakamanahia te kounga
Whakahaerehia ngā whakamātautau whakatauira me te whakatairite i ngā putanga taha-ki-te-taha. -
Whakahouhia
Ngā panonitanga iti, ngā tuhipoka mārama, whakahokia. Kāore e whakaatu - whai hua.
Āe, koinei tonu te Me pēhea te Arotau i ngā Tauira AI ahakoa he rite tonu ki te "Me pēhea te mutu te takahi i ngā rake". He rite tonu.
13) Ngā Hapa Noa (Kia Kaua e Whakahokia Pērā i a Tātou Katoa) 🙃
-
Te arotau i mua i te ine
Ka moumou taima koe. Kātahi ka arotau koe i te mea hē me te māia… -
Te whai i tētahi paearu kotahi
Kei te ngaro ngā paearu. Ko tāu kawenga mahi te pono. -
Te kore aro ki te mahara
Ka puta ngā whakaroa, ngā tukinga, me te wiri i ngā raruraru mahara. ( Te Mārama ki te whakamahinga mahara CUDA i roto i te PyTorch ) -
te ine nui rawa i te wā tino wawe.
ine iti-moka, engari tīmata ki ngā mahi haumaru i te tuatahi. -
Kāore he mahere huri whakamuri.
Ki te kore e taea te hoki wawe, ka noho ahotea ia whakatinanatanga. Ka puta he pepeke i te ahotea.
Ngā Tuhipoka Whakamutunga: Te Huarahi Tangata ki te Arotau 😌⚡
te Arotau i ngā Tauira AI i te mahi kotahi. He tukanga paparanga: ine, whakatika i te paipa, whakamahi i ngā kaikohikohi me ngā wā oma, whakahāngai i te tuku, kātahi ka whakaiti i te tauira mā te ine ine, te whakatōtō rānei mēnā e tika ana. Mahia i ia taahiraa, i ia taahiraa, kia mau ki ngā ārai kounga, kaua hoki e whakawhirinaki ki te "tere ake te ahua" hei ine (he ataahua ō kare ā-roto, ehara ō kare ā-roto i te kaitātari).
Ki te hiahia koe ki te kai poto rawa hei tango:
-
Inehia i te tuatahi 🔍
-
Whakapai ake i te paipa e whai ake nei 🧵
-
Kātahi ka arotauhia te tauira 🧠
-
Kātahi ka arotau i te tuku 🏗️
-
Me tirotiro tonu i te kounga ✅
Ā, ki te āwhina, kia maumahara koe: ehara te whāinga i te "tauira tino tika". Ko te whāinga he tauira tere, he utu tika, he pono hoki e taea ai e koe te moe i te pō… i te nuinga o ngā pō 😴.
Ngā Pātai Auau
Te tikanga o te arotau i tētahi tauira AI i roto i te mahi
Ko te tikanga o te "arotau" ko te whakapai ake i tētahi herenga matua: te whakaroa, te utu, te tapuwae mahara, te tika, te pumau, te kaha tuku ratonga rānei. Ko te wāhanga uaua ko ngā taupatupatu - mā te pana i tētahi wāhi ka pakaru tētahi atu. Ko tētahi huarahi mahi ko te whiriwhiri i tētahi whāinga mārama (pēnei i te whakaroa p95, te wā-ki-te-kounga rānei) me te arotau ki taua whāinga. Ki te kore he whāinga, he ngāwari te "whakapai" engari ka ngaro tonu.
Me pēhea te arotau i ngā tauira AI me te kore e whara i te kounga
Whakaarohia ia tere, ia huringa utu rānei he hokinga puku pea. Whakamahia ngā ārai pēnei i ngā tohu koura, ngā inenga mahi, me ngā tirotiro tere a te tangata. Whakatakotoria he paepae mārama mō te rerekētanga kounga e whakaaetia ana, ā, whakatairitea ngā putanga taha-ki-taha. Mā tēnei ka ārai i te hurihanga o te "tere ake" ki te "he aha i rerekē ai i te wā whakaputa?" i muri i tō tukunga.
Ngā mea hei ine i mua i te tīmatanga o te arotau
Tīmata ki ngā ōrau whakaroa (p50, p95, p99), te putanga (tohu/hēkona, tono/hēkona rānei), te whakamahinga GPU, me te VRAM/RAM tihi. Aroturukihia te utu mō ia whakatau, mō ia tohu 1k rānei mēnā he herenga te utu. Tātaihia he horopaki tūturu e tukuna ana e koe, kaua he tohu takaro. Mā te pupuri i tētahi "pukapuka mahi" iti ka āwhina i a koe ki te karo i te whakapae me te whakahoki anō i ngā hapa.
Ngā wikitoria tere, iti te mōrearea mō te mahi whakangungu
Ko te tino tika (FP16/BF16) te mea tere rawa atu i te nuinga o te wā, engari kia tūpato ki ngā āhuatanga tau. Mena he iti te rahi o te puranga, ka taea e te kohikohinga rōnaki te whakau i te arotautanga me te kore e pakaru te mahara. Ka whakawhitihia e te arowhai rōnaki te tatau taapiri mō te mahara iti, ka taea ai ngā horopaki nui ake. Kaua e warewarehia te tokenization me te whakatikatika uta raraunga - ka taea e rātou te whakangāwari i te GPU.
Āhea me whakamahi i te torch.compile, te ONNX Runtime, te TensorRT rānei
Ko te arotahi o ēnei taputapu ko te hopu kauwhata, te whakakotahitanga o te kernel, me te arotautanga o te kauwhata wā-oma. Ka taea e rātou te tuku tere ake i te whakatau, engari he rerekē ngā hua i runga i te āhua o te tauira me te taputapu. Ko ētahi o ngā tatūnga he rite ki te makutu; ko ētahi he iti noa te neke. Kia mataara ki te aro ki ngā huringa āhua me ngā hapa "gremlin" i ētahi wā - ine i mua me muri i runga i tō kawenga mahi tūturu.
He pai te ine ine, me pēhea te karo i te haere rawa
Ka taea e te ine rahinga te whakaiti i te mahara me te tere ake i te whakatau, inā koa me te INT8, engari ka heke te kounga i ngā take taha. Ka nui ake ngā penapena mā ngā kōwhiringa iti-moka (pēnei i te INT4/k-moka) me te mōrearea nui ake. Ko te tikanga haumaru ko te aromatawai i runga i tētahi huinga whakamātautau tūturu me te whakataurite i ngā putanga, kaua i te kare ā-roto. Tīmata ki ngā mahi haumaru ake i te tuatahi, kātahi ka iti ake te tika mēnā e tika ana.
Te rerekētanga i waenga i te kutikuti me te whakahoro hei whakaiti i te rahi o te tauira
Ka tangohia e te tapahi ngā tawhā "taumaha mate" ā, he maha ngā wā me whakangungu anō kia ora ai te kounga, inā koa ka mahia kia kaha. Ka whakangungua e te whakahoromata he tauira ākonga iti ake kia whai i ngā whanonga a te kaiako nui ake, ā, ka taea e ia te whai huatanga roa ake i te ine nui rawa atu. Mena kei te hiahia koe ki tētahi tauira iti ake e mahi ana i te āhua ōrite, ā, e mau tonu ana te pumau, ko te whakahoromata te huarahi ma.
Me pēhea te whakaiti i te utu whakatau me te whakaroa mā te whakapai ake i te ratonga
Ko te mahi ratonga te wāhi e kitea ai te arotautanga: ka whakanui ake te puranga i te tere o te tuku engari ka pā kino pea ki te roa o te wā mēnā ka nui rawa te mahi, nō reira kia āta whakatikatika. Ka nui te mahi pupuri (te pupuri tere me te whakamahi anō i te KV-cache) ina hoki anō ngā horopaki. Ka whakapai ake te putanga rere i te tere e kitea ana ahakoa he rite te wā katoa. Tirohia hoki te utu nui mō ia tohu i roto i tō puranga - ka tere te tāpiri o te mahi iti mō ia tohu.
He aha te hiranga o te whakaroa o te hiku i te wā e arotau ana i ngā tauira AI
He ātaahua te āhua o ngā toharite i te wā e raru ana te p99, ā, ko te nuinga o ngā kaiwhakamahi kei te noho i roto i te hiku. Ko te roa o te wā e raru ai te hiku ka puta mai i te wiri: te pakaru o te mahara, te pikinga o te tukatuka i mua i te CPU, te puhoi o te tohu, te whanonga puranga ngoikore rānei. Koia te take e aro nui ana te aratohu ki ngā ōrau me ngā kawenga mahi tūturu. Mena ka arotau noa koe i te p50, ka taea tonu e koe te tuku i tētahi wheako "he puhoi noa iho te ahua."
Ngā Tohutoro
-
Amazon Web Services (AWS) - Ngā ōrau o te AWS CloudWatch (ngā whakamāramatanga tatauranga) - docs.aws.amazon.com
-
Google - Te Hiku i te Tauine (te mahi pai mō te whakaroa hiku) - sre.google
-
Google - Ngā Whāinga Taumata Ratonga (Pukapuka SRE) - ngā ōrau whakaroa - sre.google
-
PyTorch - torch.compile - docs.pytorch.org
-
PyTorch - FullyShardedDataParallel (FSDP) - docs.pytorch.org
-
PyTorch - PyTorch Profiler - docs.pytorch.org
-
PyTorch - Ngā tikanga CUDA: te whakahaere mahara (ngā tuhipoka tohatoha mahara CUDA) - docs.pytorch.org
-
PyTorch - Te Tika Whakaranu Aunoa (torch.amp / AMP) - docs.pytorch.org
-
PyTorch - torch.utils.checkpoint - docs.pytorch.org
-
PyTorch - Aratohu Whakatūnga Mahi - docs.pytorch.org
-
PyTorch - Akoranga Tapahi - docs.pytorch.org
-
PyTorch - Te Mārama ki te Whakamahinga Mahara CUDA i roto i te PyTorch - docs.pytorch.org
-
PyTorch - akoranga / tirohanga whānui mō te torch.compile - docs.pytorch.org
-
Wā Whakahaere ONNX - Tuhinga Wā Whakahaere ONNX - onnxruntime.ai
-
NVIDIA - Ngā tuhinga mō TensorRT - docs.nvidia.com
-
NVIDIA - Ngā momo ine TensorRT - docs.nvidia.com
-
NVIDIA - Ngā Pūnaha Nsight - developer.nvidia.com
-
NVIDIA - Tūmau Whakatau Triton - puranga hihiri - docs.nvidia.com
-
DeepSpeed - Ngā tuhinga mō te ZeRO Wāhanga 3 - deepspeed.readthedocs.io
-
bitsandbytes (turanga-bitsandbytes) - bitsandbytes - github.com
-
Te Awhi i te Kanohi - Whakateretere: Aratohu Whakaemi Rōrahi - huggingface.co
-
Te Awhi i te Kanohi - Ngā tuhinga mō ngā Tokenizers - huggingface.co
-
Kanohi Awhi - Transformers: Aratohu PEFT - huggingface.co
-
Kanohi Āwhi - Transformers: Whakamārama mō te KV cache - huggingface.co
-
Kanohi Āwhi - Ngā Whakawhiti: Ngā Kaiwhakamahi Tohu "Tere" (ngā akomanga kaiwhakamahi tohu) - huggingface.co
-
arXiv - Te Whakamaeneene i te Mātauranga i roto i te Whatunga Io (Hinton et al., 2015) - arxiv.org
-
arXiv - LoRA: Te Whakarerekētanga Iti-Pae o ngā Tauira Reo Nui - arxiv.org
-
arXiv - FlashAttention: Te Aroturuki Tika Tere me te Whai Hua i te Mahara me te Mōhiotanga IO - arxiv.org