'Issue with nested functions, inheritance and multithreading in python
I'm trying to automatically take data from several websites. I defined a class
class Site1Scraper:
def __init__(self): ...
def f1(self): ...
def f2(self): ...
def All_f(self):
self.f1()
self.f2()
Everything about Site1Scraper works perfectly and has been tested. I'm now trying to define another class MultiThread_Site1Scraper that inherits from Site1Scraper and that starts a thread each time that a method is called and that waits for the previous thread call to finish before doing what it has been called to do. My MultiThread_Site1Scraper is:
class MultiThread_Site1Scraper(Site1Scraper):
pT = None
def __init__(self):
Site1Scraper.__init__(self)
def f1(self):
try:
self.pT.join()
except:
pass
finally:
self.pT = Thread(target=Site1Scraper.f1, args=(self))
self.pT.start()
def f2(self):
try:
self.pT.join()
except:
pass
finally:
#print(f"{self.p} \t {self.p.is_alive()} \t")
self.pT = Thread(target=Site1Scraper.f2, args=(self))
self.pT.start()
def All_f(self):
try:
self.pT.join()
except:
pass
finally:
self.pT = Thread(target=Site1Scraper.All_f, args=(self))
self.pT.start()
And my testing program is
def MultiThread_Site1Scraper_Test():
mt_b = MultiThread_Site1Scraper()
mt_b.All_f()
mt_b.f2()
I tried uncommenting the line print(f"{self.p} \t {self.p.is_alive()} \n") and I get: <Thread(Thread-2 (f1), initial)> False
I would expect that when the f2 method gets called, it sees that All_f is still ongoing and waits for it to end, but it doesn't. My final goal is to have the MultiThread_Site1Scraper manage by itself the whole multithreading without having to check whether something is still ongoing inside the main program. Any help is deeply appreciated!
P.S. Can I also multithread the __init__ in a similar way?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
