建站
咨詢
售后
建站咨詢
新聞中心
這里有您想知道的互聯(lián)網(wǎng)營(yíng)銷解決方案
深入理解Go-垃圾回收機(jī)制
Go的GC自打出生的時(shí)候就開(kāi)始被人詬病，但是在引入v1.5的三色標(biāo)記和v1.8的混合寫屏障后，正常的GC已經(jīng)縮短到10us左右，已經(jīng)變得非常優(yōu)秀，了不起了，我們接下來(lái)探索一下Go的GC的原理吧
網(wǎng)站建設(shè)哪家好，找成都創(chuàng)新互聯(lián)！專注于網(wǎng)頁(yè)設(shè)計(jì)、網(wǎng)站建設(shè)、微信開(kāi)發(fā)、微信小程序定制開(kāi)發(fā)、集團(tuán)企業(yè)網(wǎng)站建設(shè)等服務(wù)項(xiàng)目。為回饋新老客戶創(chuàng)新互聯(lián)還提供了江油免費(fèi)建站歡迎大家使用！
三色標(biāo)記原理
我們首先看一張圖，大概就會(huì)對(duì) 三色標(biāo)記法 有一個(gè)大致的了解：
原理：
- 首先把所有的對(duì)象都放到白色的集合中
- 從根節(jié)點(diǎn)開(kāi)始遍歷對(duì)象，遍歷到的白色對(duì)象從白色集合中放到灰色集合中
- 遍歷灰色集合中的對(duì)象，把灰色對(duì)象引用的白色集合的對(duì)象放入到灰色集合中，同時(shí)把遍歷過(guò)的灰色集合中的對(duì)象放到黑色的集合中
- 循環(huán)步驟3，知道灰色集合中沒(méi)有對(duì)象
- 步驟4結(jié)束后，白色集合中的對(duì)象就是不可達(dá)對(duì)象，也就是垃圾，進(jìn)行回收
寫屏障
Go在進(jìn)行三色標(biāo)記的時(shí)候并沒(méi)有STW，也就是說(shuō)，此時(shí)的對(duì)象還是可以進(jìn)行修改
那么我們考慮一下，下面的情況
我們?cè)谶M(jìn)行三色標(biāo)記中掃描灰色集合中，掃描到了對(duì)象A，并標(biāo)記了對(duì)象A的所有引用，這時(shí)候，開(kāi)始掃描對(duì)象D的引用，而此時(shí)，另一個(gè)goroutine修改了D->E的引用，變成了如下圖所示
這樣會(huì)不會(huì)導(dǎo)致E對(duì)象就掃描不到了，而被誤認(rèn)為 為白色對(duì)象，也就是垃圾
寫屏障就是為了解決這樣的問(wèn)題，引入寫屏障后，在上述步驟后，E會(huì)被認(rèn)為是存活的，即使后面E被A對(duì)象拋棄，E會(huì)被在下一輪的GC中進(jìn)行回收，這一輪GC中是不會(huì)對(duì)對(duì)象E進(jìn)行回收的
Go1.9中開(kāi)始啟用了混合寫屏障，偽代碼如下
- writePointer(slot, ptr):
- shade(*slot)
- if any stack is grey:
- shade(ptr)
- *slot = ptr
混合寫屏障會(huì)同時(shí)標(biāo)記指針寫入目標(biāo)的"原指針"和“新指針".
標(biāo)記原指針的原因是, 其他運(yùn)行中的線程有可能會(huì)同時(shí)把這個(gè)指針的值復(fù)制到寄存器或者棧上的本地變量
因?yàn)閺?fù)制指針到寄存器或者棧上的本地變量不會(huì)經(jīng)過(guò)寫屏障, 所以有可能會(huì)導(dǎo)致指針不被標(biāo)記, 試想下面的情況：
- [go] b = obj
- [go] oldx = nil
- [gc] scan oldx...
- [go] oldx = b.x // 復(fù)制b.x到本地變量, 不進(jìn)過(guò)寫屏障
- [go] b.x = ptr // 寫屏障應(yīng)該標(biāo)記b.x的原值
- [gc] scan b...
- 如果寫屏障不標(biāo)記原值, 那么oldx就不會(huì)被掃描到.
標(biāo)記新指針的原因是, 其他運(yùn)行中的線程有可能會(huì)轉(zhuǎn)移指針的位置, 試想下面的情況:
- [go] a = ptr
- [go] b = obj
- [gc] scan b...
- [go] b.x = a // 寫屏障應(yīng)該標(biāo)記b.x的新值
- [go] a = nil
- [gc] scan a...
- 如果寫屏障不標(biāo)記新值, 那么ptr就不會(huì)被掃描到.
混合寫屏障可以讓GC在并行標(biāo)記結(jié)束后不需要重新掃描各個(gè)G的堆棧, 可以減少M(fèi)ark Termination中的STW時(shí)間
除了寫屏障外, 在GC的過(guò)程中所有新分配的對(duì)象都會(huì)立刻變?yōu)楹谏? 在上面的mallocgc函數(shù)中可以看到
回收流程
GO的GC是并行GC, 也就是GC的大部分處理和普通的go代碼是同時(shí)運(yùn)行的, 這讓GO的GC流程比較復(fù)雜.
首先GC有四個(gè)階段, 它們分別是:
- Sweep Termination: 對(duì)未清掃的span進(jìn)行清掃, 只有上一輪的GC的清掃工作完成才可以開(kāi)始新一輪的GC
- Mark: 掃描所有根對(duì)象, 和根對(duì)象可以到達(dá)的所有對(duì)象, 標(biāo)記它們不被回收
- Mark Termination: 完成標(biāo)記工作, 重新掃描部分根對(duì)象(要求STW)
- Sweep: 按標(biāo)記結(jié)果清掃span
下圖是比較完整的GC流程, 并按顏色對(duì)這四個(gè)階段進(jìn)行了分類:
在GC過(guò)程中會(huì)有兩種后臺(tái)任務(wù)(G), 一種是標(biāo)記用的后臺(tái)任務(wù), 一種是清掃用的后臺(tái)任務(wù).
標(biāo)記用的后臺(tái)任務(wù)會(huì)在需要時(shí)啟動(dòng), 可以同時(shí)工作的后臺(tái)任務(wù)數(shù)量大約是P的數(shù)量的25%, 也就是go所講的讓25%的cpu用在GC上的根據(jù).
清掃用的后臺(tái)任務(wù)在程序啟動(dòng)時(shí)會(huì)啟動(dòng)一個(gè), 進(jìn)入清掃階段時(shí)喚醒.
目前整個(gè)GC流程會(huì)進(jìn)行兩次STW(Stop The World), 第一次是Mark階段的開(kāi)始, 第二次是Mark Termination階段.
第一次STW會(huì)準(zhǔn)備根對(duì)象的掃描, 啟動(dòng)寫屏障(Write Barrier)和輔助GC(mutator assist).
第二次STW會(huì)重新掃描部分根對(duì)象, 禁用寫屏障(Write Barrier)和輔助GC(mutator assist).
需要注意的是, 不是所有根對(duì)象的掃描都需要STW, 例如掃描棧上的對(duì)象只需要停止擁有該棧的G.
寫屏障的實(shí)現(xiàn)使用了Hybrid Write Barrier, 大幅減少了第二次STW的時(shí)間.
源碼分析
gcStart
- func gcStart(mode gcMode, trigger gcTrigger) {
- // Since this is called from malloc and malloc is called in
- // the guts of a number of libraries that might be holding
- // locks, don't attempt to start GC in non-preemptible or
- // potentially unstable situations.
- // 判斷當(dāng)前g是否可以搶占，不可搶占時(shí)不觸發(fā)GC
- mp := acquirem()
- if gp := getg(); gp == mp.g0 || mp.locks > 1 || mp.preemptoff != "" {
- releasem(mp)
- return
- }
- releasem(mp)
- mp = nil
- // Pick up the remaining unswept/not being swept spans concurrently
- //
- // This shouldn't happen if we're being invoked in background
- // mode since proportional sweep should have just finished
- // sweeping everything, but rounding errors, etc, may leave a
- // few spans unswept. In forced mode, this is necessary since
- // GC can be forced at any point in the sweeping cycle.
- //
- // We check the transition condition continuously here in case
- // this G gets delayed in to the next GC cycle.
- // 清掃 殘留的未清掃的垃圾
- for trigger.test() && gosweepone() != ^uintptr(0) {
- sweep.nbgsweep++
- }
- // Perform GC initialization and the sweep termination
- // transition.
- semacquire(&work.startSema)
- // Re-check transition condition under transition lock.
- // 判斷gcTrriger的條件是否成立
- if !trigger.test() {
- semrelease(&work.startSema)
- return
- }
- // For stats, check if this GC was forced by the user
- // 判斷并記錄GC是否被強(qiáng)制執(zhí)行的，runtime.GC()可以被用戶調(diào)用并強(qiáng)制執(zhí)行
- work.userForced = trigger.kind == gcTriggerAlways || trigger.kind == gcTriggerCycle
- // In gcstoptheworld debug mode, upgrade the mode accordingly.
- // We do this after re-checking the transition condition so
- // that multiple goroutines that detect the heap trigger don't
- // start multiple STW GCs.
- // 設(shè)置gc的mode
- if mode == gcBackgroundMode {
- if debug.gcstoptheworld == 1 {
- mode = gcForceMode
- } else if debug.gcstoptheworld == 2 {
- mode = gcForceBlockMode
- }
- }
- // Ok, we're doing it! Stop everybody else
- semacquire(&worldsema)
- if trace.enabled {
- traceGCStart()
- }
- // 啟動(dòng)后臺(tái)標(biāo)記任務(wù)
- if mode == gcBackgroundMode {
- gcBgMarkStartWorkers()
- }
- // 重置gc 標(biāo)記相關(guān)的狀態(tài)
- gcResetMarkState()
- work.stwprocs, work.maxprocs = gomaxprocs, gomaxprocs
- if work.stwprocs > ncpu {
- // This is used to compute CPU time of the STW phases,
- // so it can't be more than ncpu, even if GOMAXPROCS is.
- work.stwprocs = ncpu
- }
- work.heap0 = atomic.Load64(&memstats.heap_live)
- work.pauseNS = 0
- work.mode = mode
- now := nanotime()
- work.tSweepTerm = now
- work.pauseStart = now
- if trace.enabled {
- traceGCSTWStart(1)
- }
- // STW,停止世界
- systemstack(stopTheWorldWithSema)
- // Finish sweep before we start concurrent scan.
- // 先清掃上一輪的垃圾，確保上輪GC完成
- systemstack(func() {
- finishsweep_m()
- })
- // clearpools before we start the GC. If we wait they memory will not be
- // reclaimed until the next GC cycle.
- // 清理 sync.pool sched.sudogcache、sched.deferpool，這里不展開(kāi)，sync.pool已經(jīng)說(shuō)了，剩余的后面的文章會(huì)涉及
- clearpools()
- // 增加GC技術(shù)
- work.cycles++
- if mode == gcBackgroundMode { // Do as much work concurrently as possible
- gcController.startCycle()
- work.heapGoal = memstats.next_gc
- // Enter concurrent mark phase and enable
- // write barriers.
- //
- // Because the world is stopped, all Ps will
- // observe that write barriers are enabled by
- // the time we start the world and begin
- // scanning.
- //
- // Write barriers must be enabled before assists are
- // enabled because they must be enabled before
- // any non-leaf heap objects are marked. Since
- // allocations are blocked until assists can
- // happen, we want enable assists as early as
- // possible.
- // 設(shè)置GC的狀態(tài)為 gcMark
- setGCPhase(_GCmark)
- // 更新 bgmark 的狀態(tài)
- gcBgMarkPrepare() // Must happen before assist enable.
- // 計(jì)算并排隊(duì)root 掃描任務(wù)，并初始化相關(guān)掃描任務(wù)狀態(tài)
- gcMarkRootPrepare()
- // Mark all active tinyalloc blocks. Since we're
- // allocating from these, they need to be black like
- // other allocations. The alternative is to blacken
- // the tiny block on every allocation from it, which
- // would slow down the tiny allocator.
- // 標(biāo)記 tiny 對(duì)象
- gcMarkTinyAllocs()
- // At this point all Ps have enabled the write
- // barrier, thus maintaining the no white to
- // black invariant. Enable mutator assists to
- // put back-pressure on fast allocating
- // mutators.
- // 設(shè)置 gcBlackenEnabled 為 1，啟用寫屏障
- atomic.Store(&gcBlackenEnabled, 1)
- // Assists and workers can start the moment we start
- // the world.
- gcController.markStartTime = now
- // Concurrent mark.
- systemstack(func() {
- now = startTheWorldWithSema(trace.enabled)
- })
- work.pauseNS += now - work.pauseStart
- work.tMark = now
- } else {
- // 非并行模式
- // 記錄完成標(biāo)記階段的開(kāi)始時(shí)間
- if trace.enabled {
- // Switch to mark termination STW.
- traceGCSTWDone()
- traceGCSTWStart(0)
- }
- t := nanotime()
- work.tMark, work.tMarkTerm = t, t
- workwork.heapGoal = work.heap0
- // Perform mark termination. This will restart the world.
- // stw,進(jìn)行標(biāo)記，清掃并start the world
- gcMarkTermination(memstats.triggerRatio)
- }
- semrelease(&work.startSema)
- }
gcBgMarkStartWorkers
這個(gè)函數(shù)準(zhǔn)備一些 執(zhí)行bg mark工作的goroutine，但是這些goroutine并不是立即工作的，而是到等到GC的狀態(tài)被標(biāo)記為gcMark 才開(kāi)始工作，見(jiàn)上個(gè)函數(shù)的119行
- func gcBgMarkStartWorkers() {
- // Background marking is performed by per-P G's. Ensure that
- // each P has a background GC G.
- for _, p := range allp {
- if p.gcBgMarkWorker == 0 {
- go gcBgMarkWorker(p)
- // 等待gcBgMarkWorker goroutine 的 bgMarkReady信號(hào)再繼續(xù)
- notetsleepg(&work.bgMarkReady, -1)
- noteclear(&work.bgMarkReady)
- }
- }
- }
gcBgMarkWorker
后臺(tái)標(biāo)記任務(wù)的函數(shù)
- func gcBgMarkWorker(_p_ *p) {
- gp := getg()
- // 用于休眠結(jié)束后重新獲取p和m
- type parkInfo struct {
- m muintptr // Release this m on park.
- attach puintptr // If non-nil, attach to this p on park.
- }
- // We pass park to a gopark unlock function, so it can't be on
- // the stack (see gopark). Prevent deadlock from recursively
- // starting GC by disabling preemption.
- gp.m.preemptoff = "GC worker init"
- park := new(parkInfo)
- gp.m.preemptoff = ""
- // 設(shè)置park的m和p的信息，留著后面?zhèn)鹘ogopark，在被gcController.findRunnable喚醒的時(shí)候，便于找回
- park.m.set(acquirem())
- park.attach.set(_p_)
- // Inform gcBgMarkStartWorkers that this worker is ready.
- // After this point, the background mark worker is scheduled
- // cooperatively by gcController.findRunnable. Hence, it must
- // never be preempted, as this would put it into _Grunnable
- // and put it on a run queue. Instead, when the preempt flag
- // is set, this puts itself into _Gwaiting to be woken up by
- // gcController.findRunnable at the appropriate time.
- // 讓gcBgMarkStartWorkers notetsleepg停止等待并繼續(xù)及退出
- notewakeup(&work.bgMarkReady)
- for {
- // Go to sleep until woken by gcController.findRunnable.
- // We can't releasem yet since even the call to gopark
- // may be preempted.
- // 讓g進(jìn)入休眠
- gopark(func(g *g, parkp unsafe.Pointer) bool {
- park := (*parkInfo)(parkp)
- // The worker G is no longer running, so it's
- // now safe to allow preemption.
- // 釋放當(dāng)前搶占的m
- releasem(park.m.ptr())
- // If the worker isn't attached to its P,
- // attach now. During initialization and after
- // a phase change, the worker may have been
- // running on a different P. As soon as we
- // attach, the owner P may schedule the
- // worker, so this must be done after the G is
- // stopped.
- // 設(shè)置關(guān)聯(lián)p，上面已經(jīng)設(shè)置過(guò)了
- if park.attach != 0 {
- p := park.attach.ptr()
- park.attach.set(nil)
- // cas the worker because we may be
- // racing with a new worker starting
- // on this P.
- if !p.gcBgMarkWorker.cas(0, guintptr(unsafe.Pointer(g))) {
- // The P got a new worker.
- // Exit this worker.
- return false
- }
- }
- return true
- }, unsafe.Pointer(park), waitReasonGCWorkerIdle, traceEvGoBlock, 0)
- // Loop until the P dies and disassociates this
- // worker (the P may later be reused, in which case
- // it will get a new worker) or we failed to associate.
- // 檢查P的gcBgMarkWorker是否和當(dāng)前的G一致, 不一致時(shí)結(jié)束當(dāng)前的任務(wù)
- if _p_.gcBgMarkWorker.ptr() != gp {
- break
- }
- // Disable preemption so we can use the gcw. If the
- // scheduler wants to preempt us, we'll stop draining,
- // dispose the gcw, and then preempt.
- // gopark第一個(gè)函數(shù)中釋放了m，這里再搶占回來(lái)
- park.m.set(acquirem())
- if gcBlackenEnabled == 0 {
- throw("gcBgMarkWorker: blackening not enabled")
- }
- startTime := nanotime()
- // 設(shè)置gcmark的開(kāi)始時(shí)間
- _p_.gcMarkWorkerStartTime = startTime
- decnwait := atomic.Xadd(&work.nwait, -1)
- if decnwait == work.nproc {
- println("runtime: workwork.nwait=", decnwait, "work.nproc=", work.nproc)
- throw("work.nwait was > work.nproc")
- }
- // 切換到g0工作
- systemstack(func() {
- // Mark our goroutine preemptible so its stack
- // can be scanned. This lets two mark workers
- // scan each other (otherwise, they would
- // deadlock). We must not modify anything on
- // the G stack. However, stack shrinking is
- // disabled for mark workers, so it is safe to
- // read from the G stack.
- // 設(shè)置G的狀態(tài)為waiting，以便于另一個(gè)g掃描它的棧(兩個(gè)g可以互相掃描對(duì)方的棧)
- casgstatus(gp, _Grunning, _Gwaiting)
- switch _p_.gcMarkWorkerMode {
- default:
- throw("gcBgMarkWorker: unexpected gcMarkWorkerMode")
- case gcMarkWorkerDedicatedMode:
- // 專心執(zhí)行標(biāo)記工作的模式
- gcDrain(&_p_.gcw, gcDrainUntilPreempt|gcDrainFlushBgCredit)
- if gp.preempt {
- // 被搶占了，把所有本地運(yùn)行隊(duì)列中的G放到全局運(yùn)行隊(duì)列中
- // We were preempted. This is
- // a useful signal to kick
- // everything out of the run
- // queue so it can run
- // somewhere else.
- lock(&sched.lock)
- for {
- gp, _ := runqget(_p_)
- if gp == nil {
- break
- }
- globrunqput(gp)
- }
- unlock(&sched.lock)
- }
- // Go back to draining, this time
- // without preemption.
- // 繼續(xù)執(zhí)行標(biāo)記工作
- gcDrain(&_p_.gcw, gcDrainNoBlock|gcDrainFlushBgCredit)
- case gcMarkWorkerFractionalMode:
- // 執(zhí)行標(biāo)記工作，知道被搶占
- gcDrain(&_p_.gcw, gcDrainFractional|gcDrainUntilPreempt|gcDrainFlushBgCredit)
- case gcMarkWorkerIdleMode:
- // 空閑的時(shí)候執(zhí)行標(biāo)記工作
- gcDrain(&_p_.gcw, gcDrainIdle|gcDrainUntilPreempt|gcDrainFlushBgCredit)
- }
- // 把G的waiting狀態(tài)轉(zhuǎn)換到runing狀態(tài)
- casgstatus(gp, _Gwaiting, _Grunning)
- })
- // If we are nearing the end of mark, dispose
- // of the cache promptly. We must do this
- // before signaling that we're no longer
- // working so that other workers can't observe
- // no workers and no work while we have this
- // cached, and before we compute done.
- // 及時(shí)處理本地緩存，上交到全局的隊(duì)列中
- if gcBlackenPromptly {
- _p_.gcw.dispose()
- }
- // Account for time.
- // 累加耗時(shí)
- duration := nanotime() - startTime
- switch _p_.gcMarkWorkerMode {
- case gcMarkWorkerDedicatedMode:
- atomic.Xaddint64(&gcController.dedicatedMarkTime, duration)
- atomic.Xaddint64(&gcController.dedicatedMarkWorkersNeeded, 1)
- case gcMarkWorkerFractionalMode:
- atomic.Xaddint64(&gcController.fractionalMarkTime, duration)
- atomic.Xaddint64(&_p_.gcFractionalMarkTime, duration)
- case gcMarkWorkerIdleMode:
- atomic.Xaddint64(&gcController.idleMarkTime, duration)
- }
- // Was this the last worker and did we run out
- // of work?
- incnwait := atomic.Xadd(&work.nwait, +1)
- if incnwait > work.nproc {
- println("runtime: p.gcMarkWorkerMode=", _p_.gcMarkWorkerMode,
- "workwork.nwait=", incnwait, "work.nproc=", work.nproc)
- throw("work.nwait > work.nproc")
- }
- // If this worker reached a background mark completion
- // point, signal the main GC goroutine.
- if incnwait == work.nproc && !gcMarkWorkAvailable(nil) {
- // Make this G preemptible and disassociate it
- // as the worker for this P so
- // findRunnableGCWorker doesn't try to
- // schedule it.
- // 取消p m的關(guān)聯(lián)
- _p_.gcBgMarkWorker.set(nil)
- releasem(park.m.ptr())
- gcMarkDone()
- // Disable preemption and prepare to reattach
- // to the P.
- //
- // We may be running on a different P at this
- // point, so we can't reattach until this G is
- // parked.
- park.m.set(acquirem())
- park.attach.set(_p_)
- }
新聞名稱：深入理解Go-垃圾回收機(jī)制
標(biāo)題鏈接：http://fisionsoft.com.cn/article/dhcepdc.html
