一、前提

1 nginx编译安装

假设已经安装好了nginx服务器，切能正常启动访问，个人版本为 `nginx version: nginx/1.14.0`，使用strace/pstack进行调试（或者gdb，没调试过）。strace 和ltrace 可以查看一个应用程序在运行过程中所发起的系统调用

2 strace命令

strace - trace system calls and signals 跟踪系统调用和信号，最重要的参数 -p，

strace -p pid

Attach to the process with the process ID pid and begin tracing. The trace may be terminated at any time by a keyboard interrupt signal (CTRL-C). strace will respond by detaching itself from the traced process(es) leaving it (them) to continue running. Multiple -p options can be used to attach to many processes in addition to command (which is optional if at least one -p option is given). -p "`pidof PROG`" syntax is supported

详细参数，自行查看 man strace（yum install strace）

3 nginx配置

nginx采用master-worker模式，且只有一个worker进程，方便调试

<code>worker_processes  1;
events {
    worker_connections  1024;
}/<code>

二、gstack 调试

<code>ps aux | grep nginx | grep -v grep
www      13366  0.0  0.1  21784  2164 ?        S    Mar23  0:00 nginx: worker process
root    15221  0.0  0.1  21784  2132 ?        ts  Mar13  0:00 nginx: master process /usr/local/nginx/sbin/nginx
/<code>

<code>
[@bx16-77-246 /usr/local/nginx]# gstack 13366
#0  0x00007fe98da830f3 in __epoll_wait_nocancel () from /lib64/libc.so.6
#1  0x00000000004320b8 in ngx_epoll_process_events (cycle=0xd7e1b0, timer=18446744073709551615, flags=1) at src/event/modules/ngx_epoll_module.c:800
#2  0x0000000000429c48 in ngx_process_events_and_timers (cycle=cycle@entry=0xd7e1b0) at src/event/ngx_event.c:242
#3  0x0000000000430644 in ngx_worker_process_cycle (cycle=0xd7e1b0, data=<optimized>) at src/os/unix/ngx_process_cycle.c:750
#4  0x000000000042ed72 in ngx_spawn_process (cycle=cycle@entry=0xd7e1b0, proc=proc@entry=0x4305d3 , data=data@entry=0x0, name=name@entry=0x480ead "worker process", respawn=respawn@entry=-4) at src/os/unix/ngx_process.c:199
#5  0x000000000042f90b in ngx_start_worker_processes (cycle=cycle@entry=0xd7e1b0, n=1, type=type@entry=-4) at src/os/unix/ngx_process_cycle.c:359
#6  0x0000000000431358 in ngx_master_process_cycle (cycle=cycle@entry=0xd7e1b0) at src/os/unix/ngx_process_cycle.c:244
#7  0x000000000040c84b in main (argc=<optimized>, argv=<optimized>) at src/core/nginx.c:382/<optimized>/<optimized>/<optimized>/<code>

从main()函数到epoll_wait()函数的调用关系一目了然，说明：（调用栈从下自上）

1 入口是nginx.c文件里的main函数，解析命令行参数，初始化

2 根据ngx_process变量，进入分支ngx_master_process_cycle（另一个分支是ngx_single_process_cycle）

3 进入master后，就需要启动worker进程，调用ngx_start_worker_processes

4 调用ngx_spawn_process启动worker进行，并执行回调ngx_worker_process_cycle

5 进入worker主循环，其中最重要的逻辑是ngx_process_events_and_timers

6 调用ngx_worker_process_init，执行各个模块的init钩子，并设置action为ngx_epoll_process_events

7 epoll开始进入事件等待 __epoll_wait_nocancel

三、strace 调试

<code>#参数1：9是epoll_create创建文件描述符，等待事件触发，1代表rdlist就绪队列有事件要处理
epoll_wait(9, [{EPOLLIN, {u32=14344048, u64=14344048}}], 512, -1) = 1
#接收client请求，3是socket的文件描述符
accept4(17, {sa_family=AF_INET, sin_port=htons(57695), sin_addr=inet_addr("10.2.82.135")}, [16], SOCK_NONBLOCK) = 3
#将3 （accept4 产生）加入到epoll的事件监听机制
epoll_ctl(9, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLRDHUP|EPOLLET, {u32=14344720, u64=14344720}}) = 0
#等待就绪事件，1表示有事件要处理
epoll_wait(9, [{EPOLLIN, {u32=14344720, u64=14344720}}], 512, 60000) = 1
#从client接收数据，长度484字节
recvfrom(3, "GET /ae/test HTTP/1.1\\r\\nHost: tem"..., 16384, 0, NULL, NULL) = 484
#判断_not_exists_是否存在，-1表示不存在
stat("/usr/local/nginx/html/_not_exists_", 0x7fff9e42c1c0) = -1 ENOENT (No such file or directory)
#更新事件注册
epoll_ctl(9, EPOLL_CTL_MOD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=14344720, u64=14344720}}) = 0
#初始化新的socket 4
socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 4
#对设备的I/O通道进行管理，0表示成功
ioctl(4, FIONBIO, [1])                  = 0
#将socket 4 加入epoll的事件监听机制
epoll_ctl(9, EPOLL_CTL_ADD, 4, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=14344944, u64=14344944}}) = 0
# socket 4 建立连接
connect(4, {sa_family=AF_INET, sin_port=htons(8080), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
#rdlist就绪队列里有两个事件待处理
epoll_wait(9, [{EPOLLOUT, {u32=14344720, u64=14344720}}, {EPOLLOUT, {u32=14344944, u64=14344944}}], 512, 60000) = 2
#获取套接字的选项信息
# 第一个参数：socket套接字
# 第二个参数：套接字层次
# 第三个参数：要获取的信息项 

getsockopt(4, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
#响应头509字节内容写入socket4缓冲区
writev(4, [{"GET /ae/test HTTP/1.0\\r\\nX-Forward"..., 509}], 1) = 509
#rdlist就绪队列里有一个事件待处理，即socket 3
epoll_wait(9, [{EPOLLIN|EPOLLOUT|EPOLLRDHUP, {u32=14344944, u64=14344944}}], 512, 60000) = 1
#从socket 4 读取响应头及内容
recvfrom(4, "HTTP/1.0 200 OK\\r\\nContent-Length:"..., 4096, 0, NULL, NULL) = 4096
readv(4, [{"m' => 'input',\\n  "..., 4096}], 1) = 4096
readv(4, [{"  'file_size' => 0,\\n "..., 4096}], 1) = 4096
readv(4, [{"put',\\r\\n      'rule' "..., 4096}], 1) = 4096
readv(4, [{"#39;2' => array(\\r\\n      "..., 4096}], 1) = 4096
readv(4, [{"  'file_size' => arr"..., 4096}], 1) = 4096
readv(4, [{"\\r\\n        'file_size' =&"..., 4096}], 1) = 4096
readv(4, [{"            'predownload'"..., 4096}], 1) = 4096
readv(4, [{"y (\\n      'label' => "..., 4096}], 1) = 4096
#向socket 3 写入数据
writev(3, [{"HTTP/1.1 200 OK\\r\\nServer: nginx/1"..., 165}, {"\\n\\n  \\n"..., 3949}, {"m' => 'input',\\n  "..., 4096}], 3) = 8210
writev(3, [{"  'file_size' => 0,\\n "..., 4096}, {"put',\\r\\n      'rule' "..., 4096}], 2) = 8192
writev(3, [{"#39;2' => array(\\r\\n      "..., 4096}, {"  'file_size' => arr"..., 4096}], 2) = 8192
writev(3, [{"\\r\\n        'file_size' =&"..., 4096}, {"            'predownload'"..., 4096}], 2) = 8192
writev(3, [{"y (\\n      'label' => "..., 4096}], 1) = 4096
readv(4, [{"  array(\\n                    \u0003"..., 4096}, {"9;file_ext' => 'mp4,g"..., 4096}, {"\\r\\n    
9 array(\\r\\n    'gotourl"..., 4096}, {"tle\" id=\"exampleModalScrollableT"..., 4096}, {"", 4096}, {"", 4096}, {"", 4096}, {"", 4096}], 9) = 17810
readv(4, [{"", 2670}, {"", 4096}, {"", 4096}, {"", 4096}, {"", 4096}], 5) = 0
writev(3, [{"  array(\\n                    \u0003"..., 4096}, {"9;file_ext' => 'mp4,g"..., 4096}, {"\\r\\n    
9 array(\\r\\n    'gotourl"..., 4096}, {"tle\" id=\"exampleModalScrollableT"..., 1426}], 5) = 17810
#关闭socket 4
close(4)                                = 0
write(28, "10.2.82.135 - - [24/Mar/2020:12:"..., 220) = 220
setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
#查看epoll的rdlist就绪队列是否有事件，0表示无
epoll_wait(9, [], 512, 65000)          = 0
#关闭socket 3
close(3)                                = 0
#继续等待事件触发
epoll_wait(9,

整个过程，你应该清晰的知道epoll对象上的rdlist就绪队列和rbtree的变化情况 

延伸阅读
[nginx 源码注释项目](https://blog.csdn.net/cjqh_hao/article/details/104076693)
参考文档
1 [epoll_create](http://www.man7.org/linux/man-pages/man2/epoll_create.2.html#NAME)
2 [strace工具](https://people.gnome.org/~newren/tutorials/developing-with-gnome/html/ch03s02.html)
3 [nginx 源码注释](https://github.com/HelloMrShu/nginx_comment_1.17.x)
"
/<code>

閱讀更多 martianboy 的文章

關鍵字: Nginx 深度 源码