使用 pycharm 对 Python 项目调试遇到非常诡异的现象

我在公司开发机上面 pipenv 搭建使用的是 py3.7，一开始我是用的 pycharm 远程调试，Run 没有任何问题，但是 Debug 会在进程池代码 multiprocessing.Pool()附近报错 KeyboardInterrupt 吗，实际上我什么也没有做，Run 没事儿，Debug 点一下就会自动中断。

ssh://[email protected]:22/root/.local/share/virtualenvs/cobra-CV_8c13e/bin/python -u /root/.pycharm_helpers/pydev/pydevd.py --multiproc --qt-support --client '0.0.0.0' --port 40819 --file /root/python/cobra/cobra.py -t /root/php/dvwa/vulnerabilities/sqli_blind/ -d warning: Debugger speedups using cython not found. Run '"/root/.local/share/virtualenvs/cobra-CV_8c13e/bin/python" "/root/.pycharm_helpers/pydev/setup_cython.py" build_ext --inplace' to build. pydev debugger: process 26949 is connecting Connected to pydev debugger (build 171.4694.94) [21:37:52] [DEBUG] [INIT] set logging level: debug [21:37:52] [DEBUG] [INIT] start scanning... [21:37:52] [INFO] [REPORT] Report URL: ?sid=a7df1eil4boi [21:37:52] [DEBUG] [PARSE-ARGS] Target Mode: folder [21:37:52] [DEBUG] [PARSE-ARGS] Output Mode: stream [21:37:52] [DEBUG] [PARSE-ARGS] target directory: /root/php/dvwa/vulnerabilities/sqli_blind/ [21:37:52] [INFO] [CLI] Target directory: /root/php/dvwa/vulnerabilities/sqli_blind [21:37:52] [DEBUG] [PICKUP] /root/php/dvwa/vulnerabilities/sqli_blind [21:37:52] [DEBUG] [PICKUP] [FILES] |--index.php [21:37:52] [DEBUG] [PICKUP] [FILES] |--source [21:37:52] [DEBUG] [PICKUP] [FILES] | |--medium.php [21:37:52] [DEBUG] [PICKUP] [FILES] | |--impossible.php [21:37:52] [DEBUG] [PICKUP] [FILES] | |--low.php [21:37:52] [DEBUG] [PICKUP] [FILES] | |--high.php [21:37:52] [DEBUG] [PICKUP] [FILES] |--help [21:37:52] [DEBUG] [PICKUP] [FILES] | |--help.php [21:37:52] [DEBUG] [PICKUP] [FILES] |--cookie-input.php [21:37:52] [DEBUG] [PICKUP] [EXTENSION-COUNT] .php : 7 [21:37:52] [DEBUG] [DETECTION] [LANGUAGE] .php 7 [21:37:52] [DEBUG] [DETECTION] [LANGUAGE] found the chiefly language(php), maybe have largest, continue... [21:37:52] [DEBUG] [DETECTION] [LANGUAGE] main language(php), tmp language(None) [21:37:52] [DEBUG] Dependency analysis cannot be done without finding dependency files [21:37:52] [INFO] [DETECTION] [FRAMEWORK] Unknown Framework [21:37:52] [INFO] [CLI] [STATISTIC] Language: php Framework: Unknown Framework [21:37:52] [INFO] [CLI] [STATISTIC] Files: 7, Extensions:1, Consume: 0.0 [21:37:52] [DEBUG] filter dependency rules [21:37:52] [DEBUG] filter dependency rules [21:37:53] [DEBUG] filter dependency rules [21:37:53] [DEBUG] filter dependency rules [21:37:53] [DEBUG] filter dependency rules [21:37:53] [DEBUG] filter dependency rules [21:37:53] [DEBUG] filter dependency rules [21:37:53] [DEBUG] filter dependency rules [21:37:53] [DEBUG] filter dependency rules [21:37:53] [DEBUG] filter dependency rules [21:37:53] [DEBUG] filter dependency rules [21:37:53] [DEBUG] filter dependency rules [21:37:54] [DEBUG] filter dependency rules [21:37:54] [DEBUG] filter dependency rules [21:37:54] [DEBUG] filter dependency rules [21:37:54] [DEBUG] filter dependency rules [21:37:54] [DEBUG] filter dependency rules [21:37:54] [DEBUG] filter dependency rules [21:37:54] [INFO] [PUSH] 18 CVE Rules [21:37:55] [DEBUG] Dependency analysis cannot be done without finding dependency files [21:37:55] [DEBUG] [SCAN] [STORE] Not found vulnerabilities on this rule! ^CTraceback (most recent call last): File "/root/.pycharm_helpers/pydev/pydevd.py", line 1591, in <module> globals = debugger.run(setup['file'], None, None, is_module) File "/root/.pycharm_helpers/pydev/pydevd.py", line 1018, in run pydev_imports.execfile(file, globals, locals) # execute the script File "/root/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "/root/python/cobra/cobra.py", line 22, in <module> sys.exit(main()) File "/root/python/cobra/cobra/__init__.py", line 110, in main cli.start(args.target, args.format, args.output, args.special_rules, a_sid, args.dels) File "/root/python/cobra/cobra/cli.py", line 94, in start language=main_language, framework=main_framework, file_count=file_count, xtension_count=len(files)) File "/root/python/cobra/cobra/engine.py", line 160, in scan cve_vuls = scan_cve(target_directory) File "/root/python/cobra/cobra/cve.py", line 406, in scan_cve pool.join() File "/usr/local/python3/lib/python3.7/multiprocessing/pool.py", line 560, in join p.join() File "/usr/local/python3/lib/python3.7/multiprocessing/process.py", line 140, in join res = self._popen.wait(timeout) File "/usr/local/python3/lib/python3.7/multiprocessing/popen_fork.py", line 48, in wait return self.poll(os.WNOHANG if timeout == 0.0 else 0) File "/usr/local/python3/lib/python3.7/multiprocessing/popen_fork.py", line 28, in poll pid, sts = os.waitpid(self.pid, flag) KeyboardInterrupt Process finished with exit code 1

因为看不懂报错代码，所以我就使用了我搭建在 vmware 中的虚拟机 centos7，然后同样使用 pipenv 安装了 py3.7 的虚拟环境，但是诡异得是 Run 都会产生问题了，而且报错也和之前远程调试 Debug 不一样，

/root/.local/share/virtualenvs/cobra-CV_8c13e/bin/python /root/python/cobra/cobra.py -t /root/python/cobra/tests/vulnerabilities/ --- Logging error --- Traceback (most recent call last): File "/root/python/cobra/cobra/log.py", line 123, in emit stream.write(message) TypeError: write() argument must be str, not bytes Call stack: File "/root/python/cobra/cobra.py", line 22, in <module> sys.exit(main()) File "/root/python/cobra/cobra/__init__.py", line 110, in main cli.start(args.target, args.format, args.output, args.special_rules, a_sid, args.dels) File "/root/python/cobra/cobra/cli.py", line 64, in start logger.info('[REPORT] Report URL: {u}'.format(u=report)) File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1332, in info self._log(INFO, msg, args, **kwargs) File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1468, in _log self.handle(record) File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1478, in handle self.callHandlers(record) File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1540, in callHandlers hdlr.handle(record) File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 854, in handle self.emit(record) File "/root/python/cobra/cobra/log.py", line 134, in emit self.handleError(record) Message: '[REPORT] Report URL: ?sid=a9d766qddopy' Arguments: () --- Logging error --- Traceback (most recent call last): File "/root/python/cobra/cobra/log.py", line 123, in emit stream.write(message) TypeError: write() argument must be str, not bytes Call stack: File "/root/python/cobra/cobra.py", line 22, in <module> sys.exit(main()) File "/root/python/cobra/cobra/__init__.py", line 110, in main cli.start(args.target, args.format, args.output, args.special_rules, a_sid, args.dels) File "/root/python/cobra/cobra/cli.py", line 75, in start logger.info('[CLI] Target directory: {d}'.format(d=target_directory)) File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1332, in info self._log(INFO, msg, args, **kwargs) File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1468, in _log self.handle(record) File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1478, in handle self.callHandlers(record) File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1540, in callHandlers hdlr.handle(record) File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 854, in handle self.emit(record) File "/root/python/cobra/cobra/log.py", line 134, in emit self.handleError(record) Message: '[CLI] Target directory: /root/python/cobra/tests/vulnerabilities' Arguments: () --- Logging error ---

但是我直接在 centos 命令行下运行 pycharm 运行的命令是没有问题的，我是真不知道咋办了，只能打扰求助各位前辈，愿意花 20 块钱求教到底为何，聊表心意，困扰了一晚上。

[root@xiaopo-centos cobra]# /root/.local/share/virtualenvs/cobra-CV_8c13e/bin/python /root/python/cobra/cobra.py -t /root/python/cobra/tests/vulnerabilities/ [21:05:09] [INFO] [REPORT] Report URL: ?sid=a9d7661ujlcb [21:05:09] [INFO] [CLI] Target directory: /root/python/cobra/tests/vulnerabilities [21:05:09] [INFO] [CLI] [STATISTIC] Language: php Framework: Spring [21:05:09] [INFO] [CLI] [STATISTIC] Files: 15, Extensions:14, Consume: 0.0 [21:05:13] [INFO] [PUSH] 18 CVE Rules [21:05:15] [INFO] [PUSH] 95 Rules [21:05:18] [INFO] [SCAN] Trigger Rules/Not Trigger Rules/Off Rules: 65/2/29 Vulnerabilities (69)

8 条回复 2019-03-15 15:29:06 +08:00

hanssx

2019-03-13 22:28:03 +08:00

pycon 群里刘师傅说了一个问题，“你用的是虚拟环境下的 python，为啥报错目录是 /usr/local/python3 呢”
我也感觉好奇怪，按道理来说，报错应该是在虚拟环境下的吧？！

simple2025

2019-03-13 22:31:23 +08:00

感觉是 log.py 文件的问题
```
def emit(self, record):
try:
message = stdout_encode(self.format(record))
stream = self.stream

if not self.is_tty:
if message and message[0] == "\r":
message = message[1:]
stream.write(message)
else:
self.output_colorized(message)
stream.write(getattr(self, 'terminator', '\n'))

self.flush()
except (KeyboardInterrupt, SystemExit):
raise
except IOError:
pass
except Exception as e:
self.handleError(record)
```
这个改成
```
def emt(self, record):
try:
message = stdout_encode(self.format(record))
stream = self.stream

if not self.is_tty:
if message and message[0] == "\r":
message = message[1:]
stream.write(message.decode(sys.stdout.encoding))
else:
self.output_colorized(message)
stream.write(getattr(self, 'terminator', '\n'))

self.flush()
except (KeyboardInterrupt, SystemExit):
raise
except IOError:
pass
except Exception as e:
self.handleError(record)
```
感觉是 pycharm log 走的
```
if not self.is_tty:
if message and message[0] == "\r":
message = message[1:]
stream.write(message.decode(sys.stdout.encoding))
```
而 terminator 走的是
```
else:
self.output_colorized(message)
```

lniwn

2019-03-13 22:36:15 +08:00 via iPhone

以前遇到过 pycharm 调试 asyncio，调试时下断点，pycharm 偶发退出，但是提前下好断点，再启动调试就 OK。

hanssx

2019-03-13 23:03:17 +08:00

@chenqh 多谢师傅回复，没想到确实是这样，stream.write(message)改成 stream.write(message.decode('utf-8'))就可以了。

hanssx

2019-03-13 23:09:55 +08:00

@chenqh 但是我发现在虚拟机中使用 pycharm 调试，在使用了进程池的附近也会报一样的 KeyboardInterrupt 错误，看来不是远程调试的问题，不过师傅已经帮我解决了一个问题，师傅请问微信或者 qq 私聊或者发我吧，我给师傅转钱，我的 qq 是 9614 六 2392
@lniwn 提前下好，下到出错的后面了，也不行呢，感谢师傅。

hanssx

2019-03-14 11:45:51 +08:00

最后发现好像是 py3.7 的问题，换成 py2.7 可以了。
参考： https://forums.fast.ai/t/pycharm-ide-debugger-exception/28909/4

hanssx

2019-03-14 13:29:17 +08:00

经测试，py3.6.8 亦可，大概率真是 py3.7 的问题。。

EdwardChu

2019-03-15 15:29:06 +08:00

py3.7 好像真的有些问题，我 pyltp 都装不了，只能把 python 版本改成 3.6 的才装上去了。