DeepSeek-Coder When the Large Language Model Meets Programming - The Rise of Code Intelligence-deepseek编码器：当大型语言模型遇到编程时.pdf

DeepSeek-Coder:WhentheLargeLanguageModelMeets

Programming-TheRiseofCodeIntelligence

1∗1,21111

DayaGuo*,QihaoZhu,DejianYang,ZhendaXie,KaiDong,WentaoZhang

1111121

GuantingChen,XiaoBi,Y.Wu,Y.K.Li,FuliLuo,YingfeiXiong,WenfengLiang

1DeepSeek-AI

2KeyLabofHCST(PKU),MOE;SCS,PekingUniversity

{zhuqh,guodaya}@

/deepseek-ai/DeepSeek-Coder

4

2

0

2Abstract

n

a

J

Therapiddevelopmentoflargelanguagemodelshasrevolutionizedcodeintelligencein

6

2softwaredevelopment.However,thepredominanceofclosed-sourcemodelshasrestricted

extensiveresearchanddevelopment.Toaddressthis,weintroducetheDeepSeek-Coderseries,

]

Earangeofopen-sourcecodemodelswithsizesfrom1.3Bto33B,trainedfromscratchon2

Strilliontokens.Thesemodelsarepre-trainedonahigh-qualityproject-levelcodecorpusand

.

semployafill-in-the-blanktaskwitha16Kwindowtoenhancecodegenerationandinfilling.

cOurextensiveevaluationsdemonstratethatDeepSeek-Codernotonlyachievesstate-of-the-art

[

performanceamongopen-sourcecodemodelsacrossmultiplebenchmarksbutalsosurpasses

2

vexistingclosed-sourcemodelslikeCodexandGPT-3.5.Furthermore,DeepSeek-Codermodels

6areunderapermissivelicensethatallowsforbothresearchandunrestrictedcommercialuse.

9

1

4

1

.

1

0

4

2

:

v

i

X

r

a

Figure1|T