Me pēhea te arotau i ngā tauira AI [Ataata me te Pātaitai]

Whakautu poto: Hei arotau i ngā tauira AI, whiriwhiria tētahi herenga matua (whakaroa, utu, mahara, kounga, pumau, putanga rānei), kātahi ka hopu i tētahi turanga pono i mua i te whakarerekē i tētahi mea. Tangohia ngā aukati paipa i te tuatahi, kātahi ka whakamahia ngā painga iti-mōrearea pērā i te tika whakauru me te puranga; ki te mau tonu te kounga, neke atu ki te taputapu whakahiato/wā-oma, kātahi ka whakaitihia te rahi o te tauira mā te ine ine, te whakatōtō rānei ina hiahiatia.

Ngā kōrero matua hei whakaaroaro:

Herenga: Kōwhiria kia kotahi, kia rua rānei ngā inenga whāinga; he whenua taupatupatu te arotautanga, ehara i te wikitoria kore utu.

Ine: Tātarihia ngā kawenga mahi tūturu me te p50/p95/p99, te putanga mahi, te whakamahinga, me ngā tihi mahara.

Paipa: Whakatikahia te tohu, ngā uta raraunga, te tukatuka-mua, me te puranga i mua i te pā ki te tauira.

Te Tuku: Whakamahia te pupuri raraunga, te puranga ā-tauira, te whakatikatika i te wā kotahi, me te aroturuki tonu i te whakaroa o te hiku.

Ngā Ārai: Whakahaerehia ngā tohutoro koura, ngā inenga mahi, me ngā arowhai ā-wāhi i muri i ia huringa mahi.

🔗 Me pēhea te aromatawai tika i ngā tauira AI
Ngā paearu matua me ngā mahi hei aromatawai tika me te pono i ngā tauira.

🔗 Me pēhea te ine i te mahi a te AI me ngā inenga tūturu
Whakamahia ngā tohu paearu, te whakaroa, te utu, me ngā tohu kounga hei whakatairite.

🔗 Me pēhea te whakamātautau i ngā tauira AI i mua i te whakaputanga.
Te rere mahi whakamātautau mahi: ngā wehewehenga raraunga, ngā take ahotea, me te aroturuki.

🔗 Me pēhea te whakamahi i te AI mō te waihanga ihirangi
Hurihia ngā whakaaro hei tauira tere ake mā te whakamahi i ngā tohutohu hanganga me te whakahou.

1) Te Tikanga o te "Whakaarotau" i roto i te Mahi (Nā te mea he rerekē te whakamahinga a te katoa) 🧠

Ina kī te tangata "arotau i tētahi tauira AI," ko tā rātou tikanga pea ko:

• Kia tere ake (kia iti ake te whakaroa)

• Me whakaiti te utu (he iti ake ngā hāora GPU, he iti ake te whakapaunga kapua)

• Whakaitihia (te tapuwae mahara, te whakatinanatanga o te taha)

• Kia tika ake (kia pai ake te kounga, kia iti ake ngā moemoeā)

• Kia pumau ake (kia iti ake te rerekētanga, kia iti ake ngā hapa i roto i te whakaputanga)

• Me ngāwari ake te tuku (te nui o te mahi, te puranga, te mahinga matapae)

Anei te pono e ahua whakarihariha ana: kāore e taea te whakanui i ēnei katoa i te wā kotahi. He rite te arotau ki te pēhi i tētahi poihau - panaia tētahi taha ki roto, ka puta mai tētahi atu taha. Ehara i te mea i ngā wā katoa, engari he maha ngā wā ka whakamahere koe mō ngā taupatupatu.

Nō reira, i mua i te pā atu ki tētahi mea, whiriwhiria tō herenga matua:

• Mena kei te mahi ora koe ki ngā kaiwhakamahi, he mea nui ki a koe te whakaroa p95 (ngā ōrau AWS CloudWatch) me te mahi hiku (te mahi pai "whakaroa hiku") 📉

• Mena kei te whakangungu koe, he mea nui ki a koe te wā-ki-te-kounga me te whakamahinga o te GPU 🔥

• Mena kei te whakamahi koe i ngā taputapu, he mea nui ki a koe te RAM me te mana 🔋

2) He aha te āhua o te putanga pai o te arotautanga tauira AI ✅

Ko te putanga pai o te arotautanga ehara i te "whakamahi i te ine me te inoi" anake. He pūnaha. Ko ngā whakaritenga pai rawa atu:

• He tūāpapa e whakawhirinaki ana koe
  Ki te kore e taea e koe te whakaputa anō i ō hua o nāianei, kāore koe e mōhio kua whakapai ake koe i tetahi mea. He māmā noa iho… engari ka pekehia e te tangata. Kātahi ka huripokitia.

• te inenga whāinga
  "Tere ake". He whāinga tūturu te "Whakaitihia te whakaroa p95 mai i te 900ms ki te 300ms i te kaute kounga ōrite".

• Ngā ārai mō te kounga.
  I ia wikitoria mahi, ka tūpono te hekenga o te kounga. Me whai whakamātautau, me arotake, me whai taputapu hauora rānei koe.

• Mōhiotanga taputapu
  Ka taea e te tauira "tere" i runga i tētahi GPU te ngoki i runga i tētahi atu. He momo pōrearea motuhake anō ō ngā CPU.

• Ngā huringa auau, ehara i te tuhi anō nui.
  Ina huri koe i ngā mea e rima i te wā kotahi, ā, ka pai ake te mahi, kāore koe e mōhio he aha. He mea… whakararuraru tēnei.

Me rite te arotautanga ki te whakatā i tētahi kitā - he iti noa ngā whakarerekētanga, whakarongo pai, tāruatia 🎸. Ki te rite koe ki te tākaro maripi, he hapa kei roto.

3) Ripanga Whakataurite: Ngā Kōwhiringa Rongonui hei Arotau i ngā Tauira AI 📊

Kei raro nei tētahi ripanga whakatairite tere, ā, he āhua pōrearea hoki, o ngā taputapu/huarahi arotau noa. Kāo, ehara i te mea "tika" - kāore hoki te ao tūturu i te pērā.

Whakaae mō ngā āhuatanga rerekē o te hōputu: He pōrearea te "Utu" nā te mea ka taea tonu e te pūmanawa tuwhera te utu mō te wiki mutunga o te rapu hapa, he utu nui tēnā. 😵💫

4) Tīmata ki te Ine: Whakamāramahia tō whakaaro kia rite ki tō tikanga 🔍

Ki te kotahi noa te mea ka mahia e koe i tēnei aratohu katoa, me mahi tēnei: ​​me ine tika.

I roto i aku ake whakamātautau, ko ngā "huarahi hou mō te arotautanga" i ahu mai i te kitenga i tētahi mea māmā noa iho pēnei i:

• te uta raraunga e matekai ana te GPU

• Te aukati i te tukatuka-mua o te CPU

• te iti o ngā rahinga puranga e hua ai te nui o te whakarewatanga o te kernel

• puhoi te tuku tohu (ka taea e ngā kaiwhakauru tohu te noho hei tangata kino puku)

• te wehewehenga o te mahara (ngā tuhipoka tohatoha mahara CUDA a PyTorch)

• he rorohiko papa kotahi e whakahaere ana i te mana

Ngā mea hei ine (tautuhinga iti rawa)

• Te Whakaroa (p50, p95, p99) (SRE i runga i ngā ōrau whakaroa)

• Te rere o te raraunga (tohu/hēkona, tono/hēkona)

• Te whakamahinga o te GPU (rorohiko + mahara)

• Ngā tihi VRAM / RAM

• Te utu mō ia tohu 1k (mō ia whakatau rānei)

Te whakaaro whaihua mō te whakatauira

• Tuhia tētahi horopaki e aro nui ana koe (kaua he tohutohu taonga takaro).

• Tuhia ngā mea katoa ki roto i tētahi "pukapuka mahi tino pai" iti nei.
  Ae, he hoha... engari ka ārai i a koe i te whakamātau nui i a koe anō i muri mai.

(Mēnā kei te hiahia koe i tētahi taputapu tūturu hei tīmatanga: ko PyTorch Profiler (torch.profiler docs) me Nsight Systems (NVIDIA Nsight Systems) ngā mea e tino whakaarohia ana.)

5) Arotautanga Raraunga + Whakangungu: Te Mana Nui Mārire 📦🚀

Ka aro nui te tangata ki te hoahoanga tauira, ā, ka wareware ki te paipa hiko. I taua wā, ka tahuna marietia e te paipa hiko te haurua o te GPU.

Ngā wikitoria ngāwari e tere ana te puta

• Whakamahia te tino tika (FP16/BF16 mēnā he pumau) (PyTorch AMP / torch.amp)
  He tere ake te nuinga, he pai tonu - engari kia tūpato ki ngā āhuatanga rerekē o te tau.

• Te kohikohinga o ngā rōnaki ina iti te rahi o te puranga (🤗 Aratohu whakateretere)
  Ka noho pumau te arotautanga me te kore e pahū te mahara.

• Te arowhai i te rōnaki (torch.utils.checkpoint)
  Ka whakawhiti i te tatau mō te mahara - ka taea ai ngā horopaki nui ake.

• Te tuku tohu whai hua (🤗 Ngā Kaiwhakauru Tohu)
  Ka taea e te tuku tohu te noho hei arai i te tauine. Ehara i te mea ataahua; he mea nui.

• Te whakahāngai i te uta raraunga
  He maha atu ngā kaimahi, te mahara kua pinea, te tiki-mua - kāore i te whakaatu engari he whai hua 😴➡️💪 (Aratohu Whakahāngai Mahi PyTorch)

Te whakatikatika pai-tawhā

Mena kei te whakatikatika koe i ngā tauira nunui, ka taea e ngā tikanga PEFT (pēnei i ngā urutau momo LoRA) te whakaiti nui i te utu whakangungu me te noho kaha tonu (🤗 Aratohu PEFT Transformers, pepa LoRA). Koinei tētahi o ngā wā "he aha tātou i kore ai e mahi i tēnei i mua ake nei?".

6) Arotautanga Taumata Hanganga: Whakatikahia te Rahi o te Tauira 🧩

I ētahi wā ko te huarahi pai ki te arotau ko… te whakamutu i te whakamahi i tētahi tauira he nui rawa mō te mahi. E mōhio ana ahau, he mahi whakahāwea 😄.

Waea atu mō ētahi kaupapa matua:

• Whakatauhia mēnā me whai pūkenga whānui koe, me whai tohunga rānei.

• Kia nui te matapihi horopaki e tika ana, kaua e rahi rawa.

• Whakamahia he tauira kua whakangungua mō te mahi e mahi ana (ngā tauira whakarōpūtanga mō ngā mahi whakarōpūtanga, me ētahi atu).

Ngā rautaki whai hua mō te whakatikatika rahi

• Whakawhiti ki tētahi tauira iti ake mō te nuinga o ngā tono.
  Kātahi ka tukuna ngā "uiui uaua" ki tētahi tauira nui ake.

• Whakamahia he tatūnga e rua ngā wāhanga.
  He tere te tauira tauira, he kaha ake te manatoko tauira, te whakatika rānei.
  He rite ki te tuhituhi tahi me tētahi hoa he whiriwhiri - he hoha, engari he whai hua.

• Whakaitihia te roa o te putanga
  He nui te moni me te wā e pau ana ki ngā tohu putanga. Ki te kore e pai te haere o tō tauira, ka utua e koe te utu mō te haere.

Kua kite ahau i ngā tīma e whakaheke ana i ngā utu mā te whakapoto i ngā putanga. He ahua iti noa iho. He pai te mahi.

7) Ngā Arotautanga o te Kaihanga + Kauwhata: Nō hea te Tere 🏎️

Koinei te paparanga "kia atamai ake te mahi a te rorohiko".

Ngā tikanga noa:

• Te whakakotahitanga o ngā kaiwhakahaere (te whakakotahitanga o ngā kernel) (te "whakakotahitanga paparanga" NVIDIA TensorRT)

• Te takai pumau (ngā uara pumau kua tatauhia i mua) (Ngā arotautanga kauwhata ONNX Runtime)

• te kōwhiringa kernel ki te taputapu

• Te hopu kauwhata hei whakaiti i te utu nui o Python (torch.compile overview)

He māmā noa iho: tera pea he tere te pāngarau o tō tauira, engari he puhoi te mahi. Ka whakatikahia ētahi o ēnā e ngā kaiwhakaemi.

Ngā tuhipoka whai hua (arā, ngā koha)

• Ka pāngia pea ēnei whakapainga e ngā huringa āhua o te tauira.

• He tere rawa ētahi tauira, ko ētahi he paku nekeneke noa iho.

• I ētahi wā ka tere ake te oma, ā, ka pōhēhē hoki - pērā i te gremlin kua nekehia mai 🧌

Heoi anō, ina mahi, ko tētahi o ngā wikitoria tino pai.

8) Te Whakawhānui, te Tapahi, te Whakatōtō: Iti ake, kāore e tangi (Nui rawa) 🪓📉

Koinei te wāhanga e hiahiatia ana e te tangata… nā te mea he rite ki te mahi kore utu. Tērā pea, engari me mahi pērā i te pokanga.

Te ine ine (ngā taumaha/whakahohenga iti iho te tika)

• He tino pai mō te tere whakatau me te mahara

• Mōrearea: ka heke te kounga, inā koa i ngā āhuatanga taha

• Te mahi pai: aromatawaihia i runga i tētahi huinga whakamātautau tūturu, kaua i runga i ngā kare ā-roto

Ngā reka noa ka rangona e koe:

• INT8 (he maha ngā wā he totoka) (Ngā momo TensorRT kua inehia)

• INT4 / iti-moka (he penapena nui, he piki ake te mōrearea kounga) (te ine inenga k-moka bitsandbytes)

• Te rahinga whakauru (kāore e hiahiatia ana te tika ōrite mō ngā mea katoa)

Te tapahi (tango i ngā tawhā)

• Ka tango i ngā taumaha, i ngā hanganga rānei "kāore i te tino hira" (akoranga tapahi PyTorch)

• I te nuinga o te wā me whakangungu anō kia ora ai te kounga

• He pai ake te mahi i tā te tangata e whakaaro ana… ina oti pai

Te whakatōtō (ka ako te ākonga i te kaiako)

Koinei taku tino pai mō te whakamahi i te rīwhi mō te wā roa. Ka taea e te whakatōtō te whakaputa i tētahi tauira iti ake e mahi ana i te āhua ōrite, ā, he maha ngā wā he pumau ake i te whakatōtōtanga nui rawa (Te Whakatōtō i te Mātauranga i roto i te Whatunga Io).

He kupu whakarite kāore i te tino tika: he rite te whakatō wai ki te ringihia he hupa uaua mā roto i te tātari, ā, ka puta he… hupa iti ake. Ehara i te mea koinā te mahi a te hupa, engari kua mārama koe ki te whakaaro 🍲.

9) Te Mahi me te Whakatau: Te Rohe Pakanga Tuturu 🧯

Ka taea e koe te "arotau" i tētahi tauira, ā, kāore tonu e pai te mahi. Ko te mahi te wāhi e tino kitea ai te roa me te utu.

He mea nui te wikitoria i te mahi tuku

• te puranga
  i te tere o te mahi. Engari ka whakanui ake i te roa o te mahi mēnā ka nui rawa. Tauritehia. (Te puranga hihiri a Triton)

• Te Kete I te
  Kete i te Prompt me te whakamahi anō i te KV-cache ka nui te mahi mō ngā horopaki e whakahuatia ana. (He whakamārama mō te KV cache)

• Putanga rerema
  Ahakoa he rite te roa o te wā, e whakaaro ana ngā kaiwhakamahi he tere ake. He mea nui te tirohanga 🙂.

• Te whakaiti i te utu mō ia tohu
  Ka nui ake te mahi a ētahi puranga mō ia tohu. Whakaitihia taua utu, ā, ka nui tō wikitoria.

Kia tūpato ki te whakaroa o te hiku

Tērā pea ka ātaahua tō toharite i te wā e pōrearea ana tō p99. Engari, kei te hiku tonu ngā kaiwhakamahi. (Te"Tail latency" me te take e takoto ai ngā toharite)

10) Te Arotau i te Pūmārō: Whakatauritehia te Tauira ki te Mīhini 🧰🖥️

He rite te arotau i te tere o te waka reihi me te kore e tirotiro i ngā potae, ki te whakatikatika i te pūnaha me te kore e tirotiro i ngā potae. Ae, ka taea e koe, engari he ahua kuware.

Ngā whakaaro mō te GPU

• Ko te whānui o te mahara te mea e whakawhāiti ana i te nuinga o te wā, ehara i te rorohiko mata

• Ka taea e ngā rahinga puranga nui ake te āwhina, kia kore rā anō e āwhina

• He mea nui te whakakotahitanga o te kernel me te arotautanga mō ngā transformers (FlashAttention: IO-aware exact attention)

Ngā whakaaro mō te CPU

• He mea nui te miro, te whakapūtātanga, me te rohe mahara

• Ka taea e te utu nui mō te tokenization te whakahaere (🤗 Ngā tokenizers "Tere")

• Tērā pea me rerekē ngā rautaki ine ine i tō te GPU

Ngā whakaaro mō te taha/pūkoro

• Ko te tapuwae mahara te mea nui rawa atu

• He mea nui te rerekētanga o te whakaroa nā te mea he… pōuri ngā taputapu

• He maha ngā wā ka hinga ake ngā tauira iti, motuhake i ngā tauira whānui nunui

11) Ngā Ārai Kounga: Kaua e "Whakapaipai" i a Koe Hei Pūngāwerewere 🧪

Me haere tahi mai me te tirotiro kounga ia wikitoria tere. Ki te kore ka whakanui koe, ka tautoko, kātahi ka puta he karere pēnei i te "he aha te kaiawhina i kōrero ohorere ai pērā i te pāhihi?" 🏴☠️

Ngā ārai whaihua:

• Ngā tohu koura (huinga tohu pumau ka whakamatautauria tonutia e koe)

• Ngā inenga mahi (tika, F1, BLEU, te mea e tika ana)

• Ngā tirotiro ā-tangata (āe, tino pono)

• Ngā paepae whakamuri ("kāore e whakaaetia kia neke atu i te X% te hekenga")

Aroturukihia hoki ngā aratau rahunga:

• te nekeneke hōputu

• ngā huringa whanonga whakakorekore

• auau o te moemoeā

• te pikinga roa o te urupare

Ka taea e te arotautanga te whakarerekē i ngā whanonga i ngā huarahi whakamiharo. He mea motuhake. He mea whakapataritari. E ai ki te tirohanga whakamuri, e taea ana te matapae.

12) Rārangi Arowhai: Me pēhea te Arotau i ngā Tauira AI, Hipanga-i-te-Hipanga ✅🤖

Ki te hiahia koe ki tētahi raupapa mārama o ngā mahi mō te Me Pēhea te Arotau i ngā Tauira AI, anei te tukanga mahi hei pupuri i te tangata kia noho mārire:

1. Tautuhia te angitu
   Kōwhiria kia 1-2 ngā inenga matua (te whakaroa, te utu, te whakaputanga, te kounga).

2. Inehia te tūāpapa. Tātaihia
   ngā kawenga mahi tūturu, tuhia te p50/p95, te mahara, te utu. (PyTorch Profiler)

3. Whakatikahia ngā arai o ngā paipa
   Te uta raraunga, te tohu, te tukatuka i mua, te puranga.

4. Whakamahia ngā wikitoria rorohiko iti-mōrearea.
   He tika whakaranu, he arotautanga kernel, he puranga pai ake.

5. Whakamātauria ngā arotautanga o te kaikohikohi/wā whakahaere:
   te hopu kauwhata, ngā wā whakahaere whakatau, te whakakotahitanga kaiwhakahaere. (torch.compile akoranga, ngā tuhinga ONNX Runtime)

6. Whakaitihia te utu mō te tauira.
   Tātarihia marietia, whakaranua mēnā ka taea, tapatapahia mēnā e tika ana.

7. Te pupuri ratonga
   rangitahi, te whakamātau kawenga, ngā whakatikatika whakaroa hiku.

8. Whakamanahia te kounga
   Whakahaerehia ngā whakamātautau whakatauira me te whakatairite i ngā putanga taha-ki-te-taha.

9. Whakahouhia
   Ngā panonitanga iti, ngā tuhipoka mārama, whakahokia. Kāore e whakaatu - whai hua.

Āe, koinei tonu te Me pēhea te Arotau i ngā Tauira AI ahakoa he rite tonu ki te "Me pēhea te mutu te takahi i ngā rake". He rite tonu.

13) Ngā Hapa Noa (Kia Kaua e Whakahokia Pērā i a Tātou Katoa) 🙃

• Te arotau i mua i te ine
  Ka moumou taima koe. Kātahi ka arotau koe i te mea hē me te māia…

• Te whai i tētahi paearu kotahi
  Kei te ngaro ngā paearu. Ko tāu kawenga mahi te pono.

• Te kore aro ki te mahara
  Ka puta ngā whakaroa, ngā tukinga, me te wiri i ngā raruraru mahara. (Te Mārama ki te whakamahinga mahara CUDA i roto i te PyTorch)

• te ine nui rawa i te wā tino wawe.
  ine iti-moka, engari tīmata ki ngā mahi haumaru i te tuatahi.

• Kāore he mahere huri whakamuri.
  Ki te kore e taea te hoki wawe, ka noho ahotea ia whakatinanatanga. Ka puta he pepeke i te ahotea.

Ngā Tuhipoka Whakamutunga: Te Huarahi Tangata ki te Arotau 😌⚡

te Arotau i ngā Tauira AI i te mahi kotahi. He tukanga paparanga: ine, whakatika i te paipa, whakamahi i ngā kaikohikohi me ngā wā oma, whakahāngai i te tuku, kātahi ka whakaiti i te tauira mā te ine ine, te whakatōtō rānei mēnā e tika ana. Mahia i ia taahiraa, i ia taahiraa, kia mau ki ngā ārai kounga, kaua hoki e whakawhirinaki ki te "tere ake te ahua" hei ine (he ataahua ō kare ā-roto, ehara ō kare ā-roto i te kaitātari).

Ki te hiahia koe ki te kai poto rawa hei tango:

• Inehia i te tuatahi 🔍

• Whakapai ake i te paipa e whai ake nei 🧵

• Kātahi ka arotauhia te tauira 🧠

• Kātahi ka arotau i te tuku 🏗️

• Me tirotiro tonu i te kounga ✅

Ā, ki te āwhina, kia maumahara koe: ehara te whāinga i te "tauira tino tika". Ko te whāinga he tauira tere, he utu tika, he pono hoki e taea ai e koe te moe i te pō… i te nuinga o ngā pō 😴.

Tauira o te ao tūturu: Te arotau i tētahi whakarāpopototanga tīkiti tautoko 🎟️⚡

Horopaki

Whakaarohia he tīma SaaS iti e whakamahi ana i tētahi tauira AI hei whakarāpopoto i ngā tīkiti tautoko e tae mai ana i mua i te whakautu a tētahi kaihoko tangata. E mahi ana te tauira, engari he puhoi: he roa rawa te tatari a ngā kaihoko mō ngā whakarāpopototanga, ā, he nui ake te utu a te kamupene i te mea e tumanakohia ana mō te whakatau.

Ehara i te mea kia "pai ake" te tauira i ngā huarahi katoa. Kotahi te herenga matua i whiriwhiria e te tīma: te whakaiti i te whakaroa o te p95 me te pupuri tonu i te kounga whakarāpopototanga kia manakohia.

He mārama tā rātou whāinga:

Whakaitihia te whakaroa p95 mai i te 2.4 hēkona ki raro iho i te 1.2 hēkona, me te kore e neke atu i te kotahi hapa whakarāpopototanga nui i roto i tētahi huinga whakamātautau 50-tikiti.

Ngā mea e hiahiatia ana e te tukanga mahi

Hei whakatinana i tēnei, ka huihui te tīma:

He huinga whakamātautau koura 50-tikiti me ngā tikiti poto, waenga, me te paru

Te momo whakarāpopototanga e tumanakohia ana: 3 ngā matā, kāore he kōrero tito noa, tāpirihia te akiaki mēnā he mārama

Ngā inenga turanga: p50, p95, p99 whakaroa, ngā tohu i hangaia, te utu mō ia tīkiti, me te maha o ngā hapa

He rārangi arowhai arotake tangata māmā

Te urunga ki ngā rangitaki tauira, ngā tatau tohu, me ngā tautuhinga puranga/pūtanga ōrite

He kōwhiringa huri whakamuri mēnā ka heke te kounga

Ko te mea nui: kāore rātou e tīmata ki te ine ine. Tuatahi, ka tirohia e rātou mēnā kei te moumou taima te paipa.

Tauira tohutohu

Mō ia tīkiti tautoko, whakarāpopototia te raruraru a te kiritaki ki roto i ngā matā e toru.

Whakaurua:

• te raruraru matua

• tetahi wāhanga hua kua whakahuatia

• te āwangawanga, te pānga pakihi rānei, mēnā kua whakahuatia

Kaua e tito noa i ngā kōrero ngaro. Ki te kore e ranea ngā kōrero a te kiritaki, me kī atu "kāore i te tohua".

Kaua e neke atu i te 80 kupu te roa o te whakarāpopototanga.

Me pēhea te whakamātautau

Whakahaerehia ngā tīkiti 50 ōrite i roto i te tatūnga tawhito me te tatūnga hou.

Mō ia oma, tuhia:

te whakaroa o te p50, p95, me te p99

Ngā tohu putanga toharite

Te utu mō ia 1,000 tīkiti

Te maha o ngā whakarāpopototanga me ngā taipitopito i hangaia

Te maha o ngā whakarāpopototanga e hiahia ana kia tuhia anōtia e te tangata

Kātahi ka whakamātauria ētahi āhuatanga uaua:

He tīkiti me ngā putanga motuhake e toru

He karere riri nui a te kiritaki

He tīkiti koretake, kāore he taipitopito

He tīkiti kei roto ngā poro rākau kua whakapirihia

He tīkiti e kī ana te kiritaki mō te whakakore i tā rātou pūkete

Ka mau i tēnei te hapa noa, ina tere ake te tauira engari kāore i te tino tupato mā te arotautanga.

Hua

He hua tauira, i runga i te wā i whakaritea ai he 50 tauira tīkiti i mua atu, ā, i muri mai i ngā haerenga arotautanga e toru:

Pūtake:

whakaroa p95: 2.4 hēkona

whakaroa p99: 3.1 hēkona

Te roanga putanga toharite: 142 kupu

Ngā tuhinga anō a te tangata: 11 i roto i te 50

Ngā hapa nui o te hanga taipitopito: 3 i roto i te 50

I muri i te arotautanga:

whakaroa p95: 1.1 hēkona

whakaroa p99: 1.6 hēkona

Te roanga putanga toharite: 61 kupu

Ngā tuhinga anō a te tangata: 5 i roto i te 50

Ngā hapa nui o te hanga taipitopito: 1 i roto i te 50

Ngā mea i rerekē:

I whakawhāitihia e te tīma te roa o te tuhinga ki te 80 kupu

I kohikohia e rātou ngā tīkiti iti-matua ki roto i ngā rōpū tokowaru

I rongoatia e rātou ngā horopaki kaupapa here hua e whakahuatia ana

I whakakāhia e rātou te tika whakaranu i muri i te whakaū i te kounga e mau ana

I waihotia te ine ine mō muri ake nei nā te mea kua tutuki kē te whāinga whakaroa

I heke iho hoki te utu i tēnei tauira nā te mea he iti ake ngā tohu i puta mai i te tauira. Mena i puta mai i te tatūnga tawhito tata ki te 142 kupu mō ia tīkiti, ā, i puta mai i te tatūnga hou tata ki te 61 kupu, kua heke te roa o te putanga tata ki te 57%. He inenga tēnā ka taea e te tīma te manatoko tika mai i ngā rangitaki.

He aha te mea ka hē

Ko te hapa tino whakamatautau ko te arotau i te tere o te moni. Ehara i te whakapainga te whakarāpopototanga tere ake e hanga ana i tētahi kupu whakaari whakahoki moni.

Ētahi atu hapa ngāwari:

Te whakamātautau i ngā tīkiti ma anake

Te kore aro ki te whakaroa o te p99

Wareware ki te whakatairite i te roa o te putanga

Te whakarerekē i ngā tautuhinga puranga me ngā tautuhinga tauira i te wā kotahi

Te whakamahi i te whakaroa toharite hei utu mō te whakaroa hiku

Te kī "i noho tonu te kounga" me te kore he rārangi arowhai arotake

He māmā noa iho te ture arotake haumaru ake: ki te neke atu i te 2 i roto i te 50 ngā whakarāpopototanga e tito noa ana i ngā kōrero nui, hoki whakamuri ka tirotiro.

He kai tango-ā-ringa whai hua

Koinei te āhua o te arotautanga tauira AI pakari i roto i te mahi: whiriwhiria tētahi here, inehia te pūnaha o nāianei, tangohia te para i te tuatahi, whakamahia ngā huringa iti-mōrearea, kātahi ka tirohia te kounga me ngā whakamātautau hoha engari he mea nui. Ehara i te mea ko te tauira tere anake te wikitoria. He tauira tere ake ka taea tonu te whakawhirinaki.

Ngā Pātai Auau

Te tikanga o te arotau i tētahi tauira AI i roto i te mahi

Ko te tikanga o te "arotau" ko te whakapai ake i tētahi herenga matua: te whakaroa, te utu, te tapuwae mahara, te tika, te pumau, te kaha tuku ratonga rānei. Ko te wāhanga uaua ko ngā taupatupatu - mā te pana i tētahi wāhi ka pakaru tētahi atu. Ko tētahi huarahi mahi ko te whiriwhiri i tētahi whāinga mārama (pēnei i te whakaroa p95, te wā-ki-te-kounga rānei) me te arotau ki taua whāinga. Ki te kore he whāinga, he ngāwari te "whakapai" engari ka ngaro tonu.

Me pēhea te arotau i ngā tauira AI me te kore e whara i te kounga

Whakaarohia ia tere, ia huringa utu rānei he hokinga puku pea. Whakamahia ngā ārai pēnei i ngā tohu koura, ngā inenga mahi, me ngā tirotiro tere a te tangata. Whakatakotoria he paepae mārama mō te rerekētanga kounga e whakaaetia ana, ā, whakatairitea ngā putanga taha-ki-taha. Mā tēnei ka ārai i te hurihanga o te "tere ake" ki te "he aha i rerekē ai i te wā whakaputa?" i muri i tō tukunga.

Ngā mea hei ine i mua i te tīmatanga o te arotau

Tīmata ki ngā ōrau whakaroa (p50, p95, p99), te putanga (tohu/hēkona, tono/hēkona rānei), te whakamahinga GPU, me te VRAM/RAM tihi. Aroturukihia te utu mō ia whakatau, mō ia tohu 1k rānei mēnā he herenga te utu. Tātaihia he horopaki tūturu e tukuna ana e koe, kaua he tohu takaro. Mā te pupuri i tētahi "pukapuka mahi" iti ka āwhina i a koe ki te karo i te whakapae me te whakahoki anō i ngā hapa.

Ngā wikitoria tere, iti te mōrearea mō te mahi whakangungu

Ko te tino tika (FP16/BF16) te mea tere rawa atu i te nuinga o te wā, engari kia tūpato ki ngā āhuatanga tau. Mena he iti te rahi o te puranga, ka taea e te kohikohinga rōnaki te whakau i te arotautanga me te kore e pakaru te mahara. Ka whakawhitihia e te arowhai rōnaki te tatau taapiri mō te mahara iti, ka taea ai ngā horopaki nui ake. Kaua e warewarehia te tokenization me te whakatikatika uta raraunga - ka taea e rātou te whakangāwari i te GPU.

Āhea me whakamahi i te torch.compile, te ONNX Runtime, te TensorRT rānei

Ko te arotahi o ēnei taputapu ko te hopu kauwhata, te whakakotahitanga o te kernel, me te arotautanga o te kauwhata wā-oma. Ka taea e rātou te tuku tere ake i te whakatau, engari he rerekē ngā hua i runga i te āhua o te tauira me te taputapu. Ko ētahi o ngā tatūnga he rite ki te makutu; ko ētahi he iti noa te neke. Kia mataara ki te aro ki ngā huringa āhua me ngā hapa "gremlin" i ētahi wā - ine i mua me muri i runga i tō kawenga mahi tūturu.

He pai te ine ine, me pēhea te karo i te haere rawa

Ka taea e te ine rahinga te whakaiti i te mahara me te tere ake i te whakatau, inā koa me te INT8, engari ka heke te kounga i ngā take taha. Ka nui ake ngā penapena mā ngā kōwhiringa iti-moka (pēnei i te INT4/k-moka) me te mōrearea nui ake. Ko te tikanga haumaru ko te aromatawai i runga i tētahi huinga whakamātautau tūturu me te whakataurite i ngā putanga, kaua i te kare ā-roto. Tīmata ki ngā mahi haumaru ake i te tuatahi, kātahi ka iti ake te tika mēnā e tika ana.

Te rerekētanga i waenga i te kutikuti me te whakahoro hei whakaiti i te rahi o te tauira

Ka tangohia e te tapahi ngā tawhā "taumaha mate" ā, he maha ngā wā me whakangungu anō kia ora ai te kounga, inā koa ka mahia kia kaha. Ka whakangungua e te whakahoromata he tauira ākonga iti ake kia whai i ngā whanonga a te kaiako nui ake, ā, ka taea e ia te whai huatanga roa ake i te ine nui rawa atu. Mena kei te hiahia koe ki tētahi tauira iti ake e mahi ana i te āhua ōrite, ā, e mau tonu ana te pumau, ko te whakahoromata te huarahi ma.

Me pēhea te whakaiti i te utu whakatau me te whakaroa mā te whakapai ake i te ratonga

Ko te mahi ratonga te wāhi e kitea ai te arotautanga: ka whakanui ake te puranga i te tere o te tuku engari ka pā kino pea ki te roa o te wā mēnā ka nui rawa te mahi, nō reira kia āta whakatikatika. Ka nui te mahi pupuri (te pupuri tere me te whakamahi anō i te KV-cache) ina hoki anō ngā horopaki. Ka whakapai ake te putanga rere i te tere e kitea ana ahakoa he rite te wā katoa. Tirohia hoki te utu nui mō ia tohu i roto i tō puranga - ka tere te tāpiri o te mahi iti mō ia tohu.

He aha te hiranga o te whakaroa o te hiku i te wā e arotau ana i ngā tauira AI

He ātaahua te āhua o ngā toharite i te wā e raru ana te p99, ā, ko te nuinga o ngā kaiwhakamahi kei te noho i roto i te hiku. Ko te roa o te wā e raru ai te hiku ka puta mai i te wiri: te pakaru o te mahara, te pikinga o te tukatuka i mua i te CPU, te puhoi o te tohu, te whanonga puranga ngoikore rānei. Koia te take e aro nui ana te aratohu ki ngā ōrau me ngā kawenga mahi tūturu. Mena ka arotau noa koe i te p50, ka taea tonu e koe te tuku i tētahi wheako "he puhoi noa iho te ahua."

Ngā Tohutoro

1. Amazon Web Services (AWS) - Ngā ōrau o te AWS CloudWatch (ngā whakamāramatanga tatauranga) - docs.aws.amazon.com

2. Google - Te Hiku i te Tauine (te mahi pai mō te whakaroa hiku) - sre.google

3. Google - Ngā Whāinga Taumata Ratonga (Pukapuka SRE) - ngā ōrau whakaroa - sre.google

4. PyTorch - torch.compile - docs.pytorch.org

5. PyTorch - FullyShardedDataParallel (FSDP) - docs.pytorch.org

6. PyTorch - PyTorch Profiler - docs.pytorch.org

7. PyTorch - Ngā tikanga CUDA: te whakahaere mahara (ngā tuhipoka tohatoha mahara CUDA) - docs.pytorch.org

8. PyTorch - Te Tika Whakaranu Aunoa (torch.amp / AMP) - docs.pytorch.org

9. PyTorch - torch.utils.checkpoint - docs.pytorch.org

10. PyTorch - Aratohu Whakatūnga Mahi - docs.pytorch.org

11. PyTorch - Akoranga Tapahi - docs.pytorch.org

12. PyTorch - Te Mārama ki te Whakamahinga Mahara CUDA i roto i te PyTorch - docs.pytorch.org

13. PyTorch - akoranga / tirohanga whānui mō te torch.compile - docs.pytorch.org

14. Wā Whakahaere ONNX - Tuhinga Wā Whakahaere ONNX - onnxruntime.ai

15. NVIDIA - Ngā tuhinga mō TensorRT - docs.nvidia.com

16. NVIDIA - Ngā momo ine TensorRT - docs.nvidia.com

17. NVIDIA - Ngā Pūnaha Nsight - developer.nvidia.com

18. NVIDIA - Tūmau Whakatau Triton - puranga hihiri - docs.nvidia.com

19. DeepSpeed ​​- Ngā tuhinga mō te ZeRO Wāhanga 3 - deepspeed.readthedocs.io

20. bitsandbytes (turanga-bitsandbytes) - bitsandbytes - github.com

21. Te Awhi i te Kanohi - Whakateretere: Aratohu Whakaemi Rōrahi - huggingface.co

22. Te Awhi i te Kanohi - Ngā tuhinga mō ngā Tokenizers - huggingface.co

23. Kanohi Awhi - Transformers: Aratohu PEFT - huggingface.co

24. Kanohi Āwhi - Transformers: Whakamārama mō te KV cache - huggingface.co

25. Kanohi Āwhi - Ngā Whakawhiti: Ngā Kaiwhakamahi Tohu "Tere" (ngā akomanga kaiwhakamahi tohu) - huggingface.co

26. arXiv - Te Whakamaeneene i te Mātauranga i roto i te Whatunga Io (Hinton et al., 2015) - arxiv.org

27. arXiv - LoRA: Te Whakarerekētanga Iti-Pae o ngā Tauira Reo Nui - arxiv.org

28. arXiv - FlashAttention: Te Aroturuki Tika Tere me te Whai Hua i te Mahara me te Mōhiotanga IO - arxiv.org