this is what I got when I ran it
so I think it’s not finding the threads at all
It almost certainly just spat out code nearly verbatim from its data set. Someone tried to show me how good ChatGPT was at writing boilerplate code and it spit out a partial, non-working example from the library’s docs that was the literal first link when you googled the question. Like… Yeah it did manage to grab the code marginally faster, but it literally wouldn’t run and just straight up copied from the docs lol.
But yeah point is the odds of a non-working sample is pretty good
1 Like
Right your list is length zero.
technically mine is running! it’s just not doing anything useful
anyways I am aware that the sample isn’t working but I don’t know how to fix it
Uh try like thread_links = soup.find_all('a', class_='post-subject')
You may wanna double check the class name. I went from memory
Am I crazy or did it have you import re without using it lol
how do I do that? from previous conversation with the bot I understand it to involve inspect element in some fashion but I’m not clear on what I’m actually supposed to do
I think it did yeah
to be fair to the bot I totally did that sort of thing all the time in high school robotics-
1 Like
You should be able to right-click a thread link, and see “inspect” in the context menu.
The class name would be what’s quoted in the class attribute. There may be multiple classes and there may be other a elements on the page with a given class than the one you want, but I’d have to look myself to say for sure
If you can’t figure it out tonight, I can help you tomorrow and I almost guarantee I can get it working in minutes. Lili could too and it’ll be up before me so you don’t even have to wait for me
it says <td class="post-subject vtop even">
does that mean I can do it with just thread_links = soup.find_all('a', class_='post-subject'), I don’t need to worry about vtop or even because those are separate classes?
ok I ran it and got the error message
C:\Users\Francis>python C:\Users\Francis\Downloads\glowfic_scraper.py
Traceback (most recent call last):
File "C:\Users\Francis\Downloads\glowfic_scraper.py", line 38, in <module>
main()
File "C:\Users\Francis\Downloads\glowfic_scraper.py", line 28, in main
thread_links = soup.find_all('a', class_='post-subject')
^^^^
NameError: name 'soup' is not defined
which I was going to say was weird since elsewhere I have soup = BeautifulSoup(response.text, 'html.parser'), but both of those are in the definitions of separate functions so maybe that’s why it’s having issues? iirc that’s one of the things Python’s annoying about
Is response.text supposed to be a .txt file?
Wait nvm i saw the full code
that was me
and yeah I mean GPT isn’t gonna do the whole thing for you but it can help point you in the right direction and take care of really menial stuff