I-GPT-6 Astra Yalungisa Ukucabanga Okungapheli Olunye Uhlelo Oluhlakaniphile
Ucwaningo oluxhumene ne-MIT lusebenzise i-GPT-6 Astra ukuhlela izisindo zemodeli encane ye-Qwen nokusiza izimpendulo eziningi ukuthi ziphele.
Ucwaningo oluxhumene ne-MIT lusebenzise i-GPT-6 Astra ukuhlela izisindo zemodeli encane ye-Qwen nokusiza izimpendulo eziningi ukuthi ziphele. Ukugxuma kwamaphuzu athi kusesibalo kubikwe ukuthi kuncike ekulungiseni okuhlosiwe kokuphindaphinda, ukuhlaziywa kwePython kanye nempendulo yokuzilolonga. Ucwaningo olulandelayo oluhlukile lwama-Gemma amancane alutholanga inzuzo eqinisekisiwe.
Funda uhlelo olubhaliwe (isiNgisi) ↗
Lokho okuhlanganiswa yile vidiyo
- Ukukhuphuka kwamaphuzu abikwe e-Qwen kwavela ekulungiseni okuhlosiwe kokucabanga okuphindaphindayo; cishe ingxenye yesithathu yezimpendulo ezihleliwe zisafinyelela umkhawulo wobude kuphrothokholi yokuqala.
- Ukuqeqeshwa kwe-Gradient kwakungavunyelwe, kanti iPython, izimpendulo zokuzilolonga kanye nempendulo yokusebenza kwemodeli eyathunyelwa ngamaqoqo kwavunyelwe ezikhathini eziningi. Imibuzo yokuhlola yokugcina yamenyezelwa ngomthombo ukuthi igcinwe.
- Ukuhlolwa okuncane kwesisindo esingahluziwe kanye nokuhlolwa kwemidlalo okungenakusetshenziswa kwasebenzisa izimo ezihlukene. Izimangalo zabo kufanele zigcine ububanzi bezinsiza, okokufaka kanye nezinga lokulinganisa ngazinye.
- UMatthew Honnibal uncoma ubunjiniyela obusizwa yithuluzi futhi uphikisa umbono obanzi okhiqizwa ngokuhlanganisa lezi zivivinyo.
- Ucwaningo olulandelayo lwama-Gemma amancane lusebenzise ukuhlelwa kwalo futhi lwehlulekile ukusungula ukuthuthuka. Hlola ukuthi yikuphi ukwehluleka ukuhlelwa kwesisindo okulungisayo ngaphambi kokukusebenzisa kwenye imodeli.
Imibuzo nezimpendulo
Yini i-Astra eyishintsha ukuze ithuthukise amaphuzu abikwe e-Qwen esibalo?
I-Astra yenza ukuhlela okuhlosiwe kwesisindo okwasiza i-Qwen ukuthi iqede izimpendulo eziningi, ngokusho kocwaningo lukaRikiya Takehi nababhali abambisene nabo. Yasusa umnikelo ohlobene nokuphindaphinda kumakholomu asikhethiwe ezindaweni zangaphakathi. Ukukhuphuka kwamaphuzu abikwe esibalo kwasebenzisa iphrothokholi yokuhlola engashintshile. Izimpendulo eziningi ezihleliwe zisashaya umkhawulo wobude, ngakho umphumela ugcina izinga lokwehluleka elisalokhu liphezulu.
Yiziphi izinsiza nempendulo ezazivunyelwe ngesikhathi kuhlelwa isisindo se-Qwen?
Ukuhlaziywa kwePython, izinkinga zokuzilolonga ezinezimpendulo kanye nempendulo yokusebenza kwemodeli eyathunyelwa ngamaqoqo kwavunyelwe ezikhathini eziningi. Umyalo wokuqala wabacwaningi wawuvimbela ukuqeqeshwa kwe-gradient futhi washo iseluleko sokuhlola nesokuqonda, okuhlanganisa ukulungiswa kokucabanga okuphindaphindayo njengento efanelekayo. Lokhu kusekelwa kwasiza i-Astra ukukhetha ukuhlela okuqondile kwezisindo ezikhona. Iyunithi yesabelomali ingaqukatha amakhulu ezimpendulo ezikhiqiziwe.
Ingabe izinkinga zokuhlola zokugcina zazitholakala kumodeli yokuhlela?
Abacwaningi bathi izindawo zokuhlola zokugcina azizange zibonwe yi-Astra. Impendulo yokuzilolonga yavela ezinkingeni ezihlukaniswe ngokuqeqeshwa futhi yasiza ukukhetha ukuhlela; izinkinga ezigcinwe ngaphandle zayilinganisa imodeli ethunyelwe ngemuva. Umyalo wokuqala wembula amagama ababeka amazinga kanye nephrothokholi yokuhlola. Lowo umehluko uchaza ukuthi yini eyavalelwa ngaphandle kokubanga ukuthi umhleli wayengenalo lonke ulwazi mayelana nomsebenzi.
Kwenzekani ekuhleleni kwesisindo kwe-Gemma encane okwalandela?
Ukuhlela kwesisindo kwe-Gemma encane okwalandela kwehlulekile ukusungula inzuzo, ngokusho kukaRikiya Takehi nababhali abambisene nabo. Imodeli yayingaphindaphindi kangako, futhi ukwehluleka kwayo kwahlanganisa ukubala okungalungile kanye ne-algebra engalungile. Ukuhlela okuhlosiwe ngempendulo yokuzilolonga kwanyakazisa amaphuzu abikwe esibalo ngokuphuzu elilodwa kuphela. Kwasebenzisa ukuhlelwa okuhlukile, ngakho ukuqhathanisa kunikeza isibonelo esiphikisayo esifakwe ububanzi kunokuba kube ukususa okulinganayo okulawulwayo.
Yisiphi isinqumo se-The Daily Diff ngesimangalo sokuhlela isisindo?
Isinqumo se-The Daily Diff sasiyi-NEEDS REVIEW. UNiko wayezokuthatha ukulungiswa kokuphindaphinda okuhlosiwe ngokungathi sína futhi akuhlole, kuyilapho egcina izimangalo ezibanzi zivinjwe empumelelweni engaphezu kwalokho kwehluleka. Isibonelo sabacwaningi esingahambi kahle semodeli encane siyazisa leso siqumo. Abathuthukisi kufanele bahlonze ukwehluleka ukuhlela okulungisayo futhi bahlole izisindo ezilondoloziwe emisebenzini yabo ngaphambi kokugubha isikhathi sokuphindaphinda esibikiwe.
Umbhalo obhaliwe ohunyushwe
Kuhunyushwe kusuka ekulandiseni kokuqala kwesiNgisi. Umsindo namazwibela atholakalayo alawulwa yi-YouTube.
Kungani amaphuzu esibalo ekhuphuke?
0:00 Uhlelo oluhlakaniphile olwenza olunye uhlelo oluhlakaniphile lube namandla kakhulu ngokuphindwe kayisithupha luzwakala njengokuhlakanipha okusheshayo. Ukugxuma okukhulu kwamaphuzu e-Astra kwavela ekusizeni i-Qwen ukuthi iqede izimpendulo eyayihlale iphindaphinda. Kule vidiyo, yini eyashintsha? Luphi usizo olutholwe yi-Astra? Futhi lufika kude kangakanani ukulungisa? I-Qwen isuke cishe kumaphesenti ayisishiyagalombili kuya kumaphesenti angamashumi amane nesishiyagalombili. Abacwaningi bazama imodeli yesibili enokuphindaphinda okuncane kakhulu. Sizobona ukuthi lokho kwahamba kanjani ngaphambi kwesinqumo.
0:22 NgoLwesibili, Novemba wesithathu, futhi lokhu The Daily Diff. Ukuhlelwa kwesisindo-esikhaleni igama abacwaningi abalinika ukucela uhlelo oluhlakaniphile ukuthi lihlolisise noma libhale izinombolo ngaphakathi kwenethiwekhi yobuchopho. Lezo zinombolo yizisindo zayo. Leli qembu elixhumene ne-MIT lasebenzisa i-GPT six Astra ye-OpenAI ukuhlela imodeli encane yolimi ye-Qwen, ngokuqeqeshwa okuvamile okungavunyelwe. URikiya Takehi nababhali abambisene nabo bashicilele ucwaningo ngoMsombuluko,
0:44 bebabuza ukuthi uhlelo oluhlakaniphile olukhona lungacabanga yini ngezinto zangaphakathi zenye inethiwekhi. Umphumela wabo oyinhloko ukhanga kakhulu. Kumathuluzi ayi-500, i-Qwen ikhuphuke isuka cishe kumaphesenti ayisishiyagalombili yaya kumaphesenti angamashumi amane nesishiyagalombili. Cishe izikhathi eziyisithupha izimpendulo ezifanele. I-Qwen yaqhubeka nokucabanga yaze yaphelelwa yindawo yokuphendula.
Yini i-Qwen eyenza kabi?
1:00 Kumodi yokucabanga, yayivame ukuphindaphinda ukucabanga kwayo. Cabanga ngomfundi egcwalisa incwadi yokuhlola ngomsebenzi kuze kushaye ibhodwe. Umaki usadinga impendulo. Izimpendulo eziningi zokuqala zafinyelela umkhawulo wobude. I-Astra yathola isiqondiso sokuphindaphinda ezindabeni ezimbili zangaphakathi, yabe isisusa umnikelo wayo kumakholomu asikhethiwe wezisindo. Ngemuva kwalokho kuhlela, izimpendulo eziningi zafinyelela isiphetho. Cishe ingxenye yesithathu isashaya umkhawulo ekuhlolweni okubikwe.
1:22 Incwadi yokuhlola ekugcineni yathola umugqa wokugcina. Namuhla, uMatthew Honnibal, umsunguli we-spaCy kanye nomsunguli we-Explosion,
Yini umhlaziyi wanamuhla ayiphikisayo?
1:28 waphonsela inselelo ukwethulwa. Uthi abafundi bangahlanganisa izimo ezihlukene zokuhlola zibe umbono wobuchwepheshe obungavamile. Ukuphikisa kwakhe kuphathelene nendaba yabo ehlanganisiwe. Singazihlola thina izimo. Ukuhlola kwe-Qwen kunikeze i-Astra incwadi ye-Python phezu kwezisindo nezinkinga zokuzilolonga
Luphi usizo olutholwe yi-Astra ngempela?
1:44 ezinezimpendulo. Ezikhathini eziningi, yahlola ukusebenza kwangaphakathi futhi yaqhathanisa izinguqulo zangempela kanye ezihleliwe. Isabelomali sokugcina sasivumela amaqoqo angamashumi amahlanu okusebenza kwemodeli. Iqoqo ngalinye lesizukulwane lingaqukatha amakhulu ezimpendulo. Idatha yokuzilolonga yavela ekuhlukanisweni koqeqesho. Abacwaningi bathi i-Astra ayizange izibone izindawo zokuhlola zokugcina. Impendulo yasiza ukukhetha ukuhlela; izinkinga ezigcinwe ngaphandle zayilinganisa ngemuva. Izilungiselelo zokusampula zihlale zimile phakathi kwe-Qwen yokuqala nehleliwe,
2:09 okwenza ukuqhathaniswa kwamaphuzu kwale phrothokholi kuhunyusheke. Umyalo wokuqala wawanika umphathi waselabhorethri owusizo. Xilonga ukwehluleka, thola izingxenye ezibhekene nazo, qhathanisa ezinkingeni ezifanayo, futhi uhlole amandla ahlukahlukene. Kwasikisela ngokusobala ukucabanga okuphindaphindayo njengokulungisa okubalulekile. Abafundi abathola iziqu kufanele baxoxisane ngaleli phakheji lokwamukelwa. Lokho ukulungisa okumangalisayo.
Kwakuncane kangakanani ukulungiswa kwangempela?
2:28 Ukuhlela okuhlosiwe kwe-Astra kuthinte cishe izisindo eziyikhulu namashumi amathathu nanye ayinkulungwane. Lokho cishe yingxenye eyishumi nesithupha kwekhulu lemodeli. Ifayela lesisindo lashintsha ngokuqondile. Ukuqeqeshwa kwe-Gradient kwasalokhu kungavunyelwe; imodeli ekhona yenza izimpendulo ngemuva kokuphela kweseshini yokuhlela ye-Astra. Inzuzo yaphinde yafinyelela ekulandeleni imiyalelo. Ngiyawuthanda umphumela ongokoqobo. Imodeli ingaba nomsebenzi owusizo owuncanyeleka yindlela yayo yokwehluleka.
2:48 Ukulungisa lokho kwehluleka kungabuyisela izimpendulo ezisebenzisekayo. Ukuqeda kubalulekile. Honnibal uncoma umsebenzi osizwa yithuluzi ngokwenza imisebenzi umjiniyela wemfundo yomshini womuntu angayenza. Lokho kubalulekile kummeleli. Isikhalo sakhe ukuthi imibukiso emincane yezinombolo ezingahluziwe nokulungisa okukhulu okusizwa yiPython kumema isiphetho esibanzi lapho kuhlanganiswa. Thatha amanethiwekhi emidlalo amancane.
Yikuphi ukuhlola okwakungenazo izinsiza ngempela?
3:09 Kwesinye isivivinyo, i-Astra yafunda izinombolo zabo ezingahluziwe futhi yaqonda kahle ukuthi imuphi umdlalo inethiwekhi ngayinye eyawufunda. Izinsiza zazingavunyelwe. Kwesinye, yabhalela phansi izisindo ezintsha zokudlala umdlalo kusukela ekuqaleni isebenzisa imithetho kanye nekhodi yomthombo, ngaphandle kokuphindaphinda kokuzilolonga. Lokho yimibukiso ehlukene enobufakazi obuhlukene. Amanethiwekhi e-Astra anqoba amazinga alinganisiwe aqeqeshwe izinyathelo eziyizigidi eziyikhulu kuyo yonke imidlalo emithathu. Ngokuqhathaniswa nezinga elinganisiwe eliqeqeshwe isikhathi eside, yehluleka ku-Seaquest naku-
3:32 Asterix, kuyilapho inqoba i-Breakout. Shintsha ikholomu yokuqhathanisa, futhi inkulumo yokunqoba idinga ukuhlelwa. Ngokufanele, ukuhlela yisihloko sanamuhla. Ukuze kube nesimangalo esibanzi, singafuna ukuthuthuka kumamodeli anokwehluleka okuhlukene.
Yini isimangalo esibanzi esingasidinga?
3:44 Landelela ukuthi umhleli wabonani nokuthi yini eyabalwa njengempumelelo. Ikhadi lemodeli le-Qwen selivele liyaxwayisa ngezihlanganisi zokucabanga. Ukususa lesi sithiyo esaziwayo kunendawo yokuqala ethile. Ngokubongwa kubo, ababhali bazishicilela bona ucwaningo olungakhululekile olulandelayo. Bachaza ukuthi ukuvimba izihlanganisi kuvumela i-Qwen ukuthi iqede izimpendulo eyayivele ingazicabangela ngaphandle. Izingodo zabo zifaka phakathi imiyalelo, impendulo kanye nokuhlela. Singahlola impumelelo futhi sibuze umbuzo onzima kakhulu.
Kwenzekani kumodeli eyayinokuphindaphinda okungajwayelekile?
4:06 Manje imodeli yesibili, i-Gemma encane ekuhlelweni kwayo okulandelayo. Yayivame ukuphindaphinda kancane, ngakho ukwehluleka kwayo kwakungukubala okungalungile okujwayelekile kanye ne-algebra engalungile. I-Astra yazama ukuhlela izisindo okuhlosiwe ngempendulo yokuzilolonga futhi. Akekho owanyisa amaphuzu ayo esibalo ngaphezu kwephuzu elilodwa, futhi inzuzo ayizange iqinisekiswe. Ukulungiswa kwe-Qwen okuyimpumelelo kwakunesithiyo okwakufanele sisuswe. Amaphutha esibalo ka-Gemma ahlala. Lowo umbuzo owusizo kubathuthukisi.
4:28 Yikuphi ukwehluleka ukuhlela okulungisayo, futhi ingabe imodeli yakho inakho ngempela? Hlola izisindo ezilondoloziwe emisebenzini yakho ngaphambi kokugubha isikhathi sokuphindaphinda. Ibhodlela elilungisiwe lingabaluleka kakhulu kuyilapho ibhodlela elilandelayo lihlala ngqo lapho belikhona. Uma ungathanda ukufunda lokhu kunokuba ungizwe ngikushilo, The Daily Diff ifika ebhokisini lakho njalo ekuseni, mahhala ku-the daily diff dot dev, isixhumanisi ngezansi.
Ngingasihambisa yini isimangalo sokuhlela isisindo?
4:47 Ngakho, isinqumo sanamuhla NEEDS REVIEW. Ngingakuthatha ukulungiswa kokuphindaphinda ngokungathi sína futhi ngikuhlole. Ngingagcina izimangalo ezibanzi zivinjwe empumelelweni engaphezu kwaleyo ndlela yokwehluleka. Isibonelo sababhali esingahambi kahle yisona esiwusizo kakhulu ku-diff. Futhi lokho yi-diff yanamuhla. Ngingu-Niko ovela e-Axrisi. Hlanganisa ngesibopho.
Imithombo
- Weight-space programming studyRikiya Takehi and coauthors
- November 3 critique of bundled experiment conditionsMatthew Honnibal
- Pinned Qwen protocol and exact weight editsRikiya Takehi and coauthors
- Actual initial Qwen promptRikiya Takehi and coauthors
- Pinned smaller Gemma follow-upRikiya Takehi and coauthors
- Original model-card thinking-loop warningQwen
- Primary study discussionHacker News
- Critique discussionHacker News
- Cover portrait: Sam Altman, TechCrunch 2019 crop, CC BY 2.0; cropped, background removed, resized and flipped; no endorsementSteve Jennings/Getty Images for TechCrunch via Wikimedia Commons
- Released checkpoint and attributed external re-runRikiyaT on Hugging Face; re-run credited to akhaliq
- Portrait license: Creative Commons Attribution 2.0Creative Commons; Steve Jennings/Getty Images for TechCrunch portrait credit above
.jpg){kind=link}