Exploring Real-World Concurrency Bugs in Go Programming Language

Golang, a programming language designed for efficient concurrency, utilizes lightweight threads called goroutines. This study delves into 171 Go concurrency bugs from various sources, analyzing root causes and fixing strategies. Highlighted results reveal insights for developers, pointing out that message passing in Go may introduce more bugs than shared memory usage. The study explores blocking and non-blocking bugs, providing valuable observations for future research in Go.

• Go programming language
• Concurrency bugs
• Goroutines
• Message passing
• Real-world study

esters_j Follow

Uploaded on Sep 12, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Understanding Real-World Concurrency Bugs in Go Tengfei Tu1, Xiaoyu Liu2, Linhai Song1, and Yiying Zhang2 1Pennsylvania State University 2Purdue University 1

2. Golang A young but widely-used programming lang. Designed for efficient and reliable concurrency Provide lightweight threads, called goroutines Support both message passing and shared memory message Lightweight threads (goroutines) Memory 2

3. Massage Passing vs. Shared Memory Thread 1 Thread 2 Thread 1 Thread 2 Memory Message Passing Shared Memory Concurrency Bug 3

4. Does Go Do Better? Message passing better than shared memory? How well does Go prevent concurrency bugs? 4

5. The 1st Empirical Study Collect 171 Go concurrency bugs from 6 apps through manually inspecting GitHub commit log How we conduct the study? Taxonomy based on two orthogonal dimensions Root causes and fixing strategies Evaluate two built-in concurrency bug detectors Cause BlotDB shared memory message passing 171 Real-World Go Concurrency Bugs Behavior 5 blocking non-blocking

6. Highlighted Results Message passing can make a lot of bugs sometimes even more than shared memory 9 observations for developers references 8 insights to guide future research in Go 6

7. Outline Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Conclusions 7

8. Outline Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Conclusions Conclusions Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug 8

9. Message Passing in Go How to pass messages across goroutines? Channel: unbuffered channel vs. buffered channel Select: waiting for multiple channel operations Goroutine 1 Goroutine 2 Goroutine 1 Goroutine 2 select { case <- ch1: case <- ch2: } Non- deterministic ch <- m ch <- m m <- ch m <- ch select unbuffered channel buffered channel 9

10. An Example of Go Concurrency Bug Parent Goroutine func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 10

11. An Example of Go Concurrency Bug Parent Goroutine func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 11

12. An Example of Go Concurrency Bug Child Goroutine Parent Goroutine func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 gofunc() result := fn() ch <- result }() 12

13. An Example of Go Concurrency Bug Child Goroutine Parent Goroutine func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 blocking and goroutine leak gofunc() result := fn() ch <- result }() timeout signal 13

14. An Example of Go Concurrency Bug Child Goroutine Parent Goroutine func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 , 1) gofunc() result := fn() ch <- result }() not blocking any more 14

15. New Concurrency Features in Go func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 buffered channel vs. unbuffered channel anonymous function use select to wait for multiple channels 15

16. New Concurrency Features in Go func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 buffered channel vs. unbuffered channel anonymous function use select to wait for multiple channels 16

17. Outline Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Conclusions Conclusions 17

18. Bug Taxonomy Categorize bugs based on two dimensions Root cause: shared memory vs. message passing func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 Cause shared memory channel message passing 18

19. Bug Taxonomy Categorize bugs based on two dimensions Root cause: shared memory vs. message passing Behavior: blocking vs. non-blocking func finishRequest(t sec) r object { ch := make(chan object) gofunc() result := fn() ch <- result }() select { case result = <- ch: return result case <- time.timeout(t): return nil } } //Kubernetes#5316 Cause shared memory blocking message passing Behavior blocking non-blocking 19

20. Bug Taxonomy Categorize bugs based on two dimensions Root cause: shared memory vs. message passing Behavior: blocking vs. non-blocking Behavior Cause Cause non- blocking blocking shared memory shared memory 105 36 69 message passing message passing 49 17 66 Behavior 86 85 blocking non-blocking 20

21. Concurrency Usage Study Observation: Share memory synchronizations are used more often in Go applications. Shared Memory Message Passing 100% 80% 60% 40% 20% 0% Docker Kubernetes etcd CockroachDB gRPC BoltDB 21

22. Outline Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Conclusions Conclusions 22

23. Root Causes Conducting blocking operations to protect shared memory accesses to pass message across goroutines Message Passing Shared Memory 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 23

24. (mis)Protecting Shared Memory Observation: Most blocking bugs caused by shared memory synchronizations have the same causes as traditional languages. 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 24

25. Misuse of Channel Goroutine 1 Goroutine 2 ch <- m m <- ch blocking 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 25

26. Misuse of Channel with Lock func goroutine1() { m.Lock() ch <- request func goroutine2() { for { m.Lock() m.Unlock() request <- ch } } m.Unlock() } 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 26

27. Observation Observation: more blocking bugs in our studied Go applications are caused by wrong message passing. 20% More 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 27

28. Implication Implication: we call for attention to the potential danger in programming with message passing. 20% More 30 25 BoltDB 20 gRPC CockroachDB 15 etcd 10 Kubernetes 5 Docker 0 Mutex Wait RWMutex Chan Chan w/ Lib 28

29. Outline Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Introduction A real bug example Go concurrency bug study Taxonomy Blocking Bug Non-blocking Bug Conclusions Conclusions 29

30. Root Causes Failing to protect shared memory Errors during message passing Shared Memory Message Passing 50 40 BoltDB gRPC 30 CockroachDB 20 etcd Kubernetes 10 Docker 0 traditional anon. waitgroup lib chan misc 30

31. Traditional Bugs > 50% 50 40 BoltDB gRPC 30 CockroachDB 20 etcd Kubernetes 10 Docker 0 traditional anon. waitgroup lib chan misc 31

32. Misusing Channel Thread 1 Thread 2 Thread 1 Thread 2 close(ch) close(ch) close(ch) ch <- m panic! panic! 50 40 BoltDB gRPC 30 CockroachDB 20 etcd Kubernetes 10 Docker 0 traditional anon. waitgroup lib chan misc 32

33. Implication Implication: new concurrency mechanisms Go introduced can themselves be the reasons of more concurrency bugs. 50 40 BoltDB gRPC 30 CockroachDB 20 etcd Kubernetes 10 Docker 0 traditional anon. waitgroup lib chan misc 33

34. Conclusions 1st empirical study on go concurrency bugs shared memory vs. message passing blocking bugs vs. non-blocking bugs paper contains more details (contact us for more) Future works Statically detecting go concurrency bugs checkers built based on identified buggy patterns Already found concurrency bugs in real applications 34

35. Thanks a lot! 35

36. Questions? Cause shared memory BlotDB message passing Behavior 171 Real-World Go Concurrency Bugs blocking non-blocking Data Set: https://github.com/system-pclub/go-concurrency-bugs 36