TAHAKOM
• LAMM-Dataset[20]: Adiversemulti-modaldatasetfeaturing186Kpairsoflanguage-imageinstructionsand
responses,alongside10Klanguage-3Dpairings,sourcedfrom8imageand4pointclouddatasets. Itincludes
fourdistincttypesofmulti-modalinstruction-responsepairings.
• function_calling_extended: AdatasetconsistingofEnglishcodepairs,designedtorefinetheAPIusage
capabilitiesofLLMs. Itisahuman-createddatasetfocusedonimprovingtheinterfacebetweenlanguage
modelsandcodingqueries.
• AmericanStories[23]:ThisdatasetisacomprehensivecorpusfromtheUSLibraryofCongressforpretraining
LLMs,enrichingthemwithawealthofAmericanculturalandhistoricalnarratives.
• dolma[24,25]: EmployedforOLMopretraining,thedolmadatasetencompassesa3trilliontokenrepository,
aimedatprovidingarobust,diversecorpusforfoundationallanguagemodeltraining.
• Platypus[26]: PartofthePlatypus2series,thisdatasetwith25,000Englishentriesistailoredforenhancing
STEMreasoningcapabilitieswithinLLMs,fosteringtheirabilitytoprocessandreasonwithscientificand
technicalinformation.
• Puffin: TheRedmond-PuffinSeriesisadialogdatasetwithabout3,000entries,characterizedbyitslong
conversationcontextsandmulti-turndialoguestructure,essentialfortrainingLLMsinsustainingcoherent,
context-awareinteractions.
• tiny_series: This dataset contains concise codes or texts in English and is devised to sharpen an LLM’s
problem-solvingandreasoningprocessesthroughtargeted,preciseexamples.
• LongBench[27]: Servingasanevaluativebenchmark,LongBenchcomprises17tasksinbothEnglishand
Chinese,testingthelong-contextunderstandingandretentioninLLMs.
• orca-chat[28]: Thisdataset,consistingof198,463Englishdialogentries,isanOrca-styledialogdataset
aimedatimprovingLLM’slong-contextconversationalability.
• DialogStudio[29]: AcollectionofdiversemultilingualdatasetsaimedatbuildingconversationalChatbots.
• chatbot_arena_conversations[30]:With33kmultilingualdialogconversations,thisdatasetcollectedcleaned
conversationswithpairwisehumanpreferencesonChatbotArena. ItisusedforReinforcementLearningwith
HumanFeedback(RLHF).
• WebGLM-qa[31]: UsedbyWebGLM,thisEnglishpairsdatasetcontains43.6kentries. ItisaQAsystem
basedonLLMandtheInternetwhereeachentrycomprisesaquestion,aresponse,andareference,withthe
responsebeinggroundedinthereference.
• phi-1[32]: AnEnglishdialogdatasetgeneratedusingthemethodoutlinedin"TextbooksAreAllYouNeed,"
focusingonmathandcomputerscienceproblems.
• Linly-pretraining-dataset: AChinesepretrainingdatasetusedbytheLinlyseriesmodel,itcomprises3.4GB
ofdatafromClueCorpusSmall,CSLnews-crawl,andmore.
• FineGrainedRLHF[33]: Containingapproximately5,000Englishexamples,thisrepositoryaimstodevelop
anewframeworktocollecthumanfeedbacktoimproveLLMsinaspectssuchasfactualcorrectnessandtopic
relevance.
• dolphin: With4.5millionEnglishpairentries,thisdatasetattemptstoreplicateMicrosoft’sOrcaandisbased
onFLANv2.
• openchat_sharegpt4_dataset[34]: Ahigh-qualityEnglishdialogdatasetwith6kdialogsgeneratedusing
GPT-4tocompleterefinedShareGPTprompts,usedbyOpenChat.
Eachdatasetisuniquelycuratedtoaddressspecifictrainingobjectives,significantlycontributingtotheprogressionof
languagemodelsinhandlingcomplextasksandbolsteringtheiroverallproficiency. Moredatasetscanbefoundin
Table3
6 LLMsandVLMs: challngesandfutureresearchdirections
7 LLMs
Designing Artificial General Intelligence (AGI) Benchmarks: Developing benchmarks for AGI necessitates a
deepexplorationofthedistinctionsbetweenhumanandmachineintelligence. Humanintelligence,withitsnuances,
8