This article brings you a detailed introduction to positioning and destruction in Python threads (with examples). It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.
I felt something was not right before work started, and I felt like I was going to take the blame. No, it’s the third day of work that makes me feel guilty.
We have an amazing background program that can dynamically load modules and run them in threads. In this way, the functions of the plug-in can be realized. When the module is updated, the background program itself will not exit. It will only close the thread corresponding to the module, update the code and then start it. 6 is not possible.
So I wrote a module to show off my skills, but I forgot to write the exit function, which resulted in a new thread being created every time the module was updated. Unless the program was restarted, those threads would remain alive.
This is not possible. We have to find a way to clean it up, otherwise I am afraid it will explode.
So how to clean it? All I can think of is two steps:
Find out the thread numbers tid that need to be cleaned up;
Destroy them;
Find out the thread ID
Similar to the usual troubleshooting, first check the thread status of the target process through the ps command. Because the thread name has been set by setName, normally it should I saw the corresponding thread. Directly use the following code to simulate this thread:
Python version of multi-threading
#coding: utf8
import threading
import os
import time
def tt():
info = threading.currentThread()
while True:
print 'pid: ', os.getpid()
print info.name, info.ident
time.sleep(3)
t1 = threading.Thread(target=tt)
t1.setName('OOOOOPPPPP')
t1.setDaemon(True)
t1.start()
t2 = threading.Thread(target=tt)
t2.setName('EEEEEEEEE')
t2.setDaemon(True)
t2.start()
t1.join()
t2.join()Copy after login
Copy after login
Output:
root@10-46-33-56:~# python t.py
pid: 5613
OOOOOPPPPP 139693508122368
pid: 5613
EEEEEEEEE 139693497632512
...
Copy after login
Copy after login
You can see that the thread name output in Python is what we set That, but the result of Ps made me doubt life:
root@10-46-33-56:~# ps -Tp 5613
PID SPID TTY TIME CMD
5613 5613 pts/2 00:00:00 python
5613 5614 pts/2 00:00:00 python
5613 5615 pts/2 00:00:00 python
Copy after login
Copy after login
Normally it shouldn’t be like this. I’m a little confused. Did I remember it wrong all along? Use another language version of multi-threading to test:
C version of multi-threading
#include<stdio.h>
#include<sys>
#include<sys>
#include<pthread.h>
void *test(void *name)
{
pid_t pid, tid;
pid = getpid();
tid = syscall(__NR_gettid);
char *tname = (char *)name;
// 设置线程名字
prctl(PR_SET_NAME, tname);
while(1)
{
printf("pid: %d, thread_id: %u, t_name: %s\n", pid, tid, tname);
sleep(3);
}
}
int main()
{
pthread_t t1, t2;
void *ret;
pthread_create(&t1, NULL, test, (void *)"Love_test_1");
pthread_create(&t2, NULL, test, (void *)"Love_test_2");
pthread_join(t1, &ret);
pthread_join(t2, &ret);
}</pthread.h></sys></sys></stdio.h>Copy after login
Copy after login
Output:
root@10-46-33-56:~# gcc t.c -lpthread && ./a.out
pid: 5575, thread_id: 5577, t_name: Love_test_2
pid: 5575, thread_id: 5576, t_name: Love_test_1
pid: 5575, thread_id: 5577, t_name: Love_test_2
pid: 5575, thread_id: 5576, t_name: Love_test_1
...
Copy after login
Copy after login
Use the PS command to verify again:
root@10-46-33-56:~# ps -Tp 5575
PID SPID TTY TIME CMD
5575 5575 pts/2 00:00:00 a.out
5575 5576 pts/2 00:00:00 Love_test_1
5575 5577 pts/2 00:00:00 Love_test_2
Copy after login
Copy after login
This is correct. The thread name can indeed be seen through Ps!
But why can’t I see the Python one? Since the thread name is set through setName, let’s take a look at the definition:
[threading.py]
class Thread(_Verbose):
...
@property
def name(self):
"""A string used for identification purposes only.
It has no semantics. Multiple threads may be given the same name. The
initial name is set by the constructor.
"""
assert self.__initialized, "Thread.__init__() not called"
return self.__name
@name.setter
def name(self, name):
assert self.__initialized, "Thread.__init__() not called"
self.__name = str(name)
def setName(self, name):
self.name = name
...
Copy after login
Copy after login
Seeing here is actually just setting the properties of the Thread object, and If it has not been touched to the root, it must be invisible~
It seems that we can no longer use ps or /proc/ to externally The python thread name is searched, so we can only solve it within Python.
So the question becomes, how to get all running threads inside Python?
threading.enumerate can perfectly solve this problem! Why?
Because it is clearly stated in the doc of the following function, all active thread objects are returned, excluding terminated and unstarted ones.
[threading.py]
def enumerate():
"""Return a list of all Thread objects currently alive.
The list includes daemonic threads, dummy thread objects created by
current_thread(), and the main thread. It excludes terminated threads and
threads that have not yet been started.
"""
with _active_limbo_lock:
return _active.values() + _limbo.values()
Copy after login
Copy after login
Because we get the Thread object, we can get the information related to the thread through this!
Please see the complete code example:
<div class="code" style="position:relative; padding:0px; margin:0px;"><div class="code" style="position:relative; padding:0px; margin:0px;"><pre class="brush:php;toolbar:false">#coding: utf8
import threading
import os
import time
def get_thread():
pid = os.getpid()
while True:
ts = threading.enumerate()
print '------- Running threads On Pid: %d -------' % pid
for t in ts:
print t.name, t.ident
print
time.sleep(1)
def tt():
info = threading.currentThread()
pid = os.getpid()
while True:
print 'pid: {}, tid: {}, tname: {}'.format(pid, info.name, info.ident)
time.sleep(3)
return
t1 = threading.Thread(target=tt)
t1.setName('Thread-test1')
t1.setDaemon(True)
t1.start()
t2 = threading.Thread(target=tt)
t2.setName('Thread-test2')
t2.setDaemon(True)
t2.start()
t3 = threading.Thread(target=get_thread)
t3.setName('Checker')
t3.setDaemon(True)
t3.start()
t1.join()
t2.join()
t3.join()</pre><div class="contentsignin">Copy after login</div></div><div class="contentsignin">Copy after login</div></div>
Output:
root@10-46-33-56:~# python t_show.py
pid: 6258, tid: Thread-test1, tname: 139907597162240
pid: 6258, tid: Thread-test2, tname: 139907586672384
------- Running threads On Pid: 6258 -------
MainThread 139907616806656
Thread-test1 139907597162240
Checker 139907576182528
Thread-test2 139907586672384
------- Running threads On Pid: 6258 -------
MainThread 139907616806656
Thread-test1 139907597162240
Checker 139907576182528
Thread-test2 139907586672384
------- Running threads On Pid: 6258 -------
MainThread 139907616806656
Thread-test1 139907597162240
Checker 139907576182528
Thread-test2 139907586672384
------- Running threads On Pid: 6258 -------
MainThread 139907616806656
Checker 139907576182528
...
Copy after login
Copy after login
The code looks a bit long, but the logic is quite simple, Thread-test1 and Thread-test2 prints out the current pid, thread id and thread name, and then exits after 3 seconds. This is to simulate the normal exit of the thread.
The Checker thread outputs all active threads in the current process through threading.enumerate every second.
It can be clearly seen that the information of Thread-test1 and Thread-test2 can be seen at the beginning. After they exit, only is left. MainThread and Checker are just themselves.
Destroy the specified thread
Since we can get the name and thread id, we can also kill the specified thread!
Suppose now Thread-test2 has gone black and gone crazy, and we need to stop it, then we can solve it in this way:
In the above code Basically, add and add the following code:
def _async_raise(tid, exctype):
"""raises the exception, performs cleanup if needed"""
tid = ctypes.c_long(tid)
if not inspect.isclass(exctype):
exctype = type(exctype)
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(exctype))
if res == 0:
raise ValueError("invalid thread id")
elif res != 1:
ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None)
raise SystemError("PyThreadState_SetAsyncExc failed")
def stop_thread(thread):
_async_raise(thread.ident, SystemExit)
def get_thread():
pid = os.getpid()
while True:
ts = threading.enumerate()
print '------- Running threads On Pid: %d -------' % pid
for t in ts:
print t.name, t.ident, t.is_alive()
if t.name == 'Thread-test2':
print 'I am go dying! Please take care of yourself and drink more hot water!'
stop_thread(t)
print
time.sleep(1)Copy after login
Copy after login
Output
root@10-46-33-56:~# python t_show.py
pid: 6362, tid: 139901682108160, tname: Thread-test1
pid: 6362, tid: 139901671618304, tname: Thread-test2
------- Running threads On Pid: 6362 -------
MainThread 139901706389248 True
Thread-test1 139901682108160 True
Checker 139901661128448 True
Thread-test2 139901671618304 True
Thread-test2: I am go dying. Please take care of yourself and drink more hot water!
------- Running threads On Pid: 6362 -------
MainThread 139901706389248 True
Thread-test1 139901682108160 True
Checker 139901661128448 True
Thread-test2 139901671618304 True
Thread-test2: I am go dying. Please take care of yourself and drink more hot water!
pid: 6362, tid: 139901682108160, tname: Thread-test1
------- Running threads On Pid: 6362 -------
MainThread 139901706389248 True
Thread-test1 139901682108160 True
Checker 139901661128448 True
// Thread-test2 已经不在了
Copy after login
Copy after login
After a while, although we treat Thread-test2 like this, it still cares about us :Drink more hot water,
PS: Although hot water is good, eight cups is enough, so don’t be greedy.
Back to the main story, the above method is extremely crude. Why do you say that?
Because its principle is: use Python's built-in API to trigger an exception on the specified thread so that it can automatically exit;
# #Don't use this method as a last resort, as there is a certain probability of triggering indescribable problems. Remember! Don't ask me why I know...
Why is it so difficult to stop threads
Multi-threading itself is designed for collaborative concurrency under the process. It is the smallest unit of scheduling, and the threads share the process. resources, so there will be many locking mechanisms and state controls.
If you use force to kill threads, there is a high chance that unexpected bugs will occur. And the most important lock resource release may also cause unexpected problems.
We can't even kill threads directly like killing processes through signals, because kill can only achieve our expectations by dealing with processes, but it is obviously not possible to deal with threads. No matter which thread is killed, the entire process will exit!
And because of the GIL, many children think that Python's threads are implemented by Python itself and do not actually exist. Python should be able to destroy them directly, right?
However, in fact, Python’s threads are genuine threads!
What does that mean? Python threads are native threads created by the operating system through pthread. Python only uses GIL to constrain these threads to decide when to start scheduling. For example, after running a number of instructions, the GIL will be handed over. As for who wins the oiran, it depends on the operating system.
If it is a simple thread, the system actually has ways to terminate it, such as: pthread_exit,pthread_kill or pthread_cancel, for details, see: https://www.cnblogs.com/Creat...
What a pity: There is no encapsulation of these methods at the Python level! Oh my god, so angry! Maybe people think that threads should be treated gently.
How to exit a thread gently
If you want to exit a thread gently, it is almost nonsense~
Either exit after running, or set the flag bit, and check the flag bit frequently. If you need to quit, just quit.
Extension
"How to correctly terminate a running child thread": https://www.cnblogs.com/Creat...
"Don't destroy python threads roughly" ：http://xiaorui.cc/2017/02/22/...
Welcome all experts to give advice and exchange, QQ discussion group: 258498217
Please note for reprinting Source: https://segmentfault.com/a/11...
Column
Technology Stack Hodgepodge
Article details
#
Lin_R
posted on
Posted 3 days ago
Python: Thread positioning and destruction
c
linux
Python
-
# 266 Reading
It takes 30 minutes to read
8
BackgroundI felt something was not right before the start of work, and I felt like I was going to take the blame. No, it’s the third day of work that makes me feel guilty. We have an amazing background program that can dynamically load modules and run them in threads. In this way, the functions of the plug-in can be realized. When the module is updated, the background program itself will not exit. It will only close the thread corresponding to the module, update the code and then start it. 6 is not possible.
So I wrote a module to show off my skills, but I forgot to write the exit function, which resulted in a new thread being created every time the module was updated. Unless the program was restarted, those threads would remain alive.
This is not possible. We have to find a way to clean it up, otherwise I am afraid it will explode.
So how to clean it? All I can think of is two steps:
Find out the thread numbers tid that need to be cleaned up;
Destroy them;
Find out the thread IDSimilar to the usual troubleshooting, first check the thread status of the target process through the ps command. Because the thread name has been set by setName, normally it should I saw the corresponding thread. Directly use the following code to simulate this thread:
-
#coding: utf8
import threading
import os
import time
def tt():
info = threading.currentThread()
while True:
print 'pid: ', os.getpid()
print info.name, info.ident
time.sleep(3)
t1 = threading.Thread(target=tt)
t1.setName('OOOOOPPPPP')
t1.setDaemon(True)
t1.start()
t2 = threading.Thread(target=tt)
t2.setName('EEEEEEEEE')
t2.setDaemon(True)
t2.start()
t1.join()
t2.join()Copy after login
Copy after login
Output:
root@10-46-33-56:~# python t.py
pid: 5613
OOOOOPPPPP 139693508122368
pid: 5613
EEEEEEEEE 139693497632512
...
Copy after login
Copy after login
You can see that the thread name output in Python is what we set That, but the result of Ps made me doubt life:
root@10-46-33-56:~# ps -Tp 5613
PID SPID TTY TIME CMD
5613 5613 pts/2 00:00:00 python
5613 5614 pts/2 00:00:00 python
5613 5615 pts/2 00:00:00 python
Copy after login
Copy after login
Normally it shouldn’t be like this. I’m a little confused. Did I remember it wrong all along? Use another language version of multi-threading to test:
C version of multi-threading
#include<stdio.h>
#include<sys>
#include<sys>
#include<pthread.h>
void *test(void *name)
{
pid_t pid, tid;
pid = getpid();
tid = syscall(__NR_gettid);
char *tname = (char *)name;
// 设置线程名字
prctl(PR_SET_NAME, tname);
while(1)
{
printf("pid: %d, thread_id: %u, t_name: %s\n", pid, tid, tname);
sleep(3);
}
}
int main()
{
pthread_t t1, t2;
void *ret;
pthread_create(&t1, NULL, test, (void *)"Love_test_1");
pthread_create(&t2, NULL, test, (void *)"Love_test_2");
pthread_join(t1, &ret);
pthread_join(t2, &ret);
}</pthread.h></sys></sys></stdio.h>Copy after login
Copy after login
Output:
root@10-46-33-56:~# gcc t.c -lpthread && ./a.out
pid: 5575, thread_id: 5577, t_name: Love_test_2
pid: 5575, thread_id: 5576, t_name: Love_test_1
pid: 5575, thread_id: 5577, t_name: Love_test_2
pid: 5575, thread_id: 5576, t_name: Love_test_1
...
Copy after login
Copy after login
Use the PS command to verify again:
root@10-46-33-56:~# ps -Tp 5575
PID SPID TTY TIME CMD
5575 5575 pts/2 00:00:00 a.out
5575 5576 pts/2 00:00:00 Love_test_1
5575 5577 pts/2 00:00:00 Love_test_2
Copy after login
Copy after login
This is correct. The thread name can indeed be seen through Ps!
But why can’t I see the Python one? Since the thread name is set through
setName, let’s take a look at the definition:
[threading.py]
class Thread(_Verbose):
...
@property
def name(self):
"""A string used for identification purposes only.
It has no semantics. Multiple threads may be given the same name. The
initial name is set by the constructor.
"""
assert self.__initialized, "Thread.__init__() not called"
return self.__name
@name.setter
def name(self, name):
assert self.__initialized, "Thread.__init__() not called"
self.__name = str(name)
def setName(self, name):
self.name = name
...
Copy after login
Copy after login
Seeing here is actually just setting the properties of the
Thread object, and If it has not been touched to the root, it must be invisible~
It seems that we can no longer use
ps or /proc/ to externally The python thread name is searched, so we can only solve it within Python.
So the question becomes, how to get all running threads inside Python?
threading.enumerate can perfectly solve this problem! Why?thread objects are returned, excluding terminated and unstarted ones.
[threading.py]
def enumerate():
"""Return a list of all Thread objects currently alive.
The list includes daemonic threads, dummy thread objects created by
current_thread(), and the main thread. It excludes terminated threads and
threads that have not yet been started.
"""
with _active_limbo_lock:
return _active.values() + _limbo.values()
Copy after login
Copy after login
Because we get the Thread object, we can get the information related to the thread through this!
Please see the complete code example: <div class="code" style="position:relative; padding:0px; margin:0px;"><div class="code" style="position:relative; padding:0px; margin:0px;"><pre class="brush:php;toolbar:false">#coding: utf8
import threading
import os
import time
def get_thread():
pid = os.getpid()
while True:
ts = threading.enumerate()
print '------- Running threads On Pid: %d -------' % pid
for t in ts:
print t.name, t.ident
print
time.sleep(1)
def tt():
info = threading.currentThread()
pid = os.getpid()
while True:
print 'pid: {}, tid: {}, tname: {}'.format(pid, info.name, info.ident)
time.sleep(3)
return
t1 = threading.Thread(target=tt)
t1.setName('Thread-test1')
t1.setDaemon(True)
t1.start()
t2 = threading.Thread(target=tt)
t2.setName('Thread-test2')
t2.setDaemon(True)
t2.start()
t3 = threading.Thread(target=get_thread)
t3.setName('Checker')
t3.setDaemon(True)
t3.start()
t1.join()
t2.join()
t3.join()</pre><div class="contentsignin">Copy after login</div></div><div class="contentsignin">Copy after login</div></div>Output:
root@10-46-33-56:~# python t_show.py
pid: 6258, tid: Thread-test1, tname: 139907597162240
pid: 6258, tid: Thread-test2, tname: 139907586672384
------- Running threads On Pid: 6258 -------
MainThread 139907616806656
Thread-test1 139907597162240
Checker 139907576182528
Thread-test2 139907586672384
------- Running threads On Pid: 6258 -------
MainThread 139907616806656
Thread-test1 139907597162240
Checker 139907576182528
Thread-test2 139907586672384
------- Running threads On Pid: 6258 -------
MainThread 139907616806656
Thread-test1 139907597162240
Checker 139907576182528
Thread-test2 139907586672384
------- Running threads On Pid: 6258 -------
MainThread 139907616806656
Checker 139907576182528
...
Copy after login
Copy after login
The code looks a bit long, but the logic is quite simple, Thread-test1 and
Thread-test2 prints out the current pid, thread id and thread name, and then exits after 3 seconds. This is to simulate the normal exit of the thread.
The
Checker thread outputs all active threads in the current process through
threading.enumerate every second. Thread-test1 and
Thread-test2 can be seen at the beginning. After they exit, only is left. MainThread and Checker are just themselves.
Destroy the specified threadThread-test2 has gone black and gone crazy, and we need to stop it, then we can solve it in this way:
def _async_raise(tid, exctype):
"""raises the exception, performs cleanup if needed"""
tid = ctypes.c_long(tid)
if not inspect.isclass(exctype):
exctype = type(exctype)
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(exctype))
if res == 0:
raise ValueError("invalid thread id")
elif res != 1:
ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None)
raise SystemError("PyThreadState_SetAsyncExc failed")
def stop_thread(thread):
_async_raise(thread.ident, SystemExit)
def get_thread():
pid = os.getpid()
while True:
ts = threading.enumerate()
print '------- Running threads On Pid: %d -------' % pid
for t in ts:
print t.name, t.ident, t.is_alive()
if t.name == 'Thread-test2':
print 'I am go dying! Please take care of yourself and drink more hot water!'
stop_thread(t)
print
time.sleep(1)Copy after login
Copy after login
Output
root@10-46-33-56:~# python t_show.py
pid: 6362, tid: 139901682108160, tname: Thread-test1
pid: 6362, tid: 139901671618304, tname: Thread-test2
------- Running threads On Pid: 6362 -------
MainThread 139901706389248 True
Thread-test1 139901682108160 True
Checker 139901661128448 True
Thread-test2 139901671618304 True
Thread-test2: I am go dying. Please take care of yourself and drink more hot water!
------- Running threads On Pid: 6362 -------
MainThread 139901706389248 True
Thread-test1 139901682108160 True
Checker 139901661128448 True
Thread-test2 139901671618304 True
Thread-test2: I am go dying. Please take care of yourself and drink more hot water!
pid: 6362, tid: 139901682108160, tname: Thread-test1
------- Running threads On Pid: 6362 -------
MainThread 139901706389248 True
Thread-test1 139901682108160 True
Checker 139901661128448 True
// Thread-test2 已经不在了
Copy after login
Copy after login
After a while, although we treat
Thread-test2 like this, it still cares about us :Drink more hot water,
PS: Although hot water is good, eight cups is enough, so don’t be greedy.
Back to the main story, the above method is extremely crude. Why do you say that?
Because its principle is: use Python's built-in API to trigger an exception on the specified thread so that it can automatically exit;
# #Don't use this method as a last resort, as there is a certain probability of triggering indescribable problems. Remember! Don't ask me why I know...Why is it so difficult to stop threadsMulti-threading itself is designed for collaborative concurrency under the process. It is the smallest unit of scheduling, and the threads share the process. resources, so there will be many locking mechanisms and state controls. If you use force to kill threads, there is a high chance that unexpected bugs will occur. And the most important lock resource release may also cause unexpected problems. We can't even kill the thread directly like killing the process through a signal, because kill can only achieve our expectations by dealing with the process, but it is obviously not possible to deal with the thread. No matter which thread is killed, the entire process will exit! And because of the GIL, many children think that Python's threads are implemented by Python itself and do not actually exist. Python should be able to destroy them directly, right? However, in fact, Python’s threads are genuine threads! What does that mean? Python threads are native threads created by the operating system through pthread. Python only uses GIL to constrain these threads to decide when to start scheduling. For example, after running a number of instructions, the GIL will be handed over. As for who wins the oiran, it depends on the operating system. If it is a simple thread, the system actually has ways to terminate it, such as: pthread_exit,pthread_kill or pthread_cancel, for details, see: https://www.cnblogs.com/Creat...
What a pity: There is no encapsulation of these methods at the Python level! Oh my god, so angry! Maybe people think that threads should be treated gently. How to exit a thread gentlyIf you want to exit a thread gently, it is almost nonsense~Either exit after running, or set the flag bit, and check the flag bit frequently. If you need to quit, just quit. Extension"How to correctly terminate a running child thread": https://www.cnblogs.com/Creat..."Don't destroy python threads roughly" ：http://xiaorui.cc/2017/02/22/...
Welcome all experts to give advice and exchange, QQ discussion group: 258498217
Please note for reprinting Ming source: https://segmentfault.com/a/11...
If you find my article useful to you, please feel free to appreciate it
You may feel Interested
##2 comments
Time sorting
##
If it were me, I might kill -9, I would rather kill a thousand by mistake than let one go, hehe
The above is the detailed content of Detailed introduction to positioning and destruction in Python threads (with examples). For more information, please follow other related articles on the PHP Chinese website!
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
