Exploring Real-World Concurrency Bugs in Go Programming Language

Slide Note
Embed
Share

Golang, a programming language designed for efficient concurrency, utilizes lightweight threads called goroutines. This study delves into 171 Go concurrency bugs from various sources, analyzing root causes and fixing strategies. Highlighted results reveal insights for developers, pointing out that message passing in Go may introduce more bugs than shared memory usage. The study explores blocking and non-blocking bugs, providing valuable observations for future research in Go.


Uploaded on Sep 12, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Understanding Real-World Concurrency Bugs in Go Tengfei Tu1, Xiaoyu Liu2, Linhai Song1, and Yiying Zhang2 1Pennsylvania State University 2Purdue University 1

  2. Golang A young but widely-used programming lang. Designed for efficient and reliable concurrency Provide lightweight threads, called goroutines Support both message passing and shared memory message Lightweight threads (goroutines) Memory 2

  3. Massage Passing vs. Shared Memory Thread 1 Thread 2 Thread 1 Thread 2 Memory Message Passing Shared Memory Concurrency Bug 3

  4. Does Go Do Better? Message passing better than shared memory? How well does Go prevent concurrency bugs? 4

  5. The 1st Empirical Study Collect 171 Go concurrency bugs from 6 apps through manually inspecting GitHub commit log How we conduct the study? Taxonomy based on two orthogonal dimensions Root causes and fixing strategies Evaluate two built-in concurrency bug detectors Cause BlotDB shared memory message passing 171 Real-World Go Concurrency Bugs Behavior 5 blocking non-blocking

  6. Highlighted Results Message passing can make a lot of bugs sometimes even more than shared memory 9 observations for developers references 8 insights to guide future research in Go 6

  7. Outline Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Conclusions 7

  8. Outline Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Conclusions Conclusions Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug 8

  9. Message Passing in Go How to pass messages across goroutines? Channel: unbuffered channel vs. buffered channel Select: waiting for multiple channel operations Goroutine 1 Goroutine 2 Goroutine 1 Goroutine 2 select { case <- ch1: case <- ch2: } Non- deterministic ch <- m ch <- m m <- ch m <- ch select unbuffered channel buffered channel 9

  10. An Example of Go Concurrency Bug Parent Goroutine func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 10

  11. An Example of Go Concurrency Bug Parent Goroutine func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 11

  12. An Example of Go Concurrency Bug Child Goroutine Parent Goroutine func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 gofunc() result := fn() ch <- result }() 12

  13. An Example of Go Concurrency Bug Child Goroutine Parent Goroutine func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 blocking and goroutine leak gofunc() result := fn() ch <- result }() timeout signal 13

  14. An Example of Go Concurrency Bug Child Goroutine Parent Goroutine func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 , 1) gofunc() result := fn() ch <- result }() not blocking any more 14

  15. New Concurrency Features in Go func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 buffered channel vs. unbuffered channel anonymous function use select to wait for multiple channels 15

  16. New Concurrency Features in Go func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 buffered channel vs. unbuffered channel anonymous function use select to wait for multiple channels 16

  17. Outline Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Conclusions Conclusions 17

  18. Bug Taxonomy Categorize bugs based on two dimensions Root cause: shared memory vs. message passing func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 Cause shared memory channel message passing 18

  19. Bug Taxonomy Categorize bugs based on two dimensions Root cause: shared memory vs. message passing Behavior: blocking vs. non-blocking func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 Cause shared memory blocking message passing Behavior blocking non-blocking 19

  20. Bug Taxonomy Categorize bugs based on two dimensions Root cause: shared memory vs. message passing Behavior: blocking vs. non-blocking Behavior Cause Cause non- blocking blocking shared memory shared memory 105 36 69 message passing message passing 49 17 66 Behavior 86 85 blocking non-blocking 20

  21. Concurrency Usage Study Observation: Share memory synchronizations are used more often in Go applications. Shared Memory Message Passing 100% 80% 60% 40% 20% 0% Docker Kubernetes etcd CockroachDB gRPC BoltDB 21

  22. Outline Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Conclusions Conclusions 22

  23. Root Causes Conducting blocking operations to protect shared memory accesses to pass message across goroutines Message Passing Shared Memory 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 23

  24. (mis)Protecting Shared Memory Observation: Most blocking bugs caused by shared memory synchronizations have the same causes as traditional languages. 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 24

  25. Misuse of Channel Goroutine 1 Goroutine 2 ch <- m m <- ch blocking 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 25

  26. Misuse of Channel with Lock func goroutine1() { m.Lock() ch <- request func goroutine2() { for { m.Lock() m.Unlock() request <- ch } } m.Unlock() } 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 26

  27. Observation Observation: more blocking bugs in our studied Go applications are caused by wrong message passing. 20% More 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 27

  28. Implication Implication: we call for attention to the potential danger in programming with message passing. 20% More 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 28

  29. Outline Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Conclusions Conclusions 29

  30. Root Causes Failing to protect shared memory Errors during message passing Shared Memory Message Passing 50 40 BoltDB gRPC 30 CockroachDB 20 etcd Kubernetes 10 Docker 0 traditional anon. waitgroup lib chan misc 30

  31. Traditional Bugs > 50% 50 40 BoltDB gRPC 30 CockroachDB 20 etcd Kubernetes 10 Docker 0 traditional anon. waitgroup lib chan misc 31

  32. Misusing Channel Thread 1 Thread 2 Thread 1 Thread 2 close(ch) close(ch) close(ch) ch <- m panic! panic! 50 40 BoltDB gRPC 30 CockroachDB 20 etcd Kubernetes 10 Docker 0 traditional anon. waitgroup lib chan misc 32

  33. Implication Implication: new concurrency mechanisms Go introduced can themselves be the reasons of more concurrency bugs. 50 40 BoltDB gRPC 30 CockroachDB 20 etcd Kubernetes 10 Docker 0 traditional anon. waitgroup lib chan misc 33

  34. Conclusions 1st empirical study on go concurrency bugs shared memory vs. message passing blocking bugs vs. non-blocking bugs paper contains more details (contact us for more) Future works Statically detecting go concurrency bugs checkers built based on identified buggy patterns Already found concurrency bugs in real applications 34

  35. Thanks a lot! 35

  36. Questions? Cause shared memory BlotDB message passing Behavior 171 Real-World Go Concurrency Bugs blocking non-blocking Data Set: https://github.com/system-pclub/go-concurrency-bugs 36

Related


More Related Content