×
  • remind me tomorrow
  • remind me next week
  • never remind me
Subscribe to the ANN Newsletter • Wake up every Sunday to a curated list of ANN's most interesting posts of the week. read more

Forum - View topic
One and only release scraper issues thread


Goto page Previous  1, 2, 3, 4

Anime News Network Forum Index -> Site-related -> Encyclopedia
View previous topic :: View next topic  
Author Message
bglassbrook



Joined: 29 Aug 2006
Posts: 1243
Location: Gaithersburg, MD
PostPosted: Sat Dec 11, 2021 2:46 pm Reply with quote
Really glad to see all of them (even Amazon) working again... or at least accepting input as valid, even if they [almost] never pull from the alternate sites.

Considering yet another platform overhaul at Rakuten, as to what their site actually does, is there any point in still listing them?
Back to top
View user's profile Send private message My Anime My Manga
BigOnAnime
Encyclopedia Editor


Joined: 01 Jul 2010
Posts: 1246
Location: Minnesota, USA
PostPosted: Sat Dec 11, 2021 4:55 pm Reply with quote
Yeah, Amazon is mostly working again, noticed that in the past week or so, though it currently works better for books than it does home video. Unfortunately for whatever reason for home video, it can't find the UPC, so this means regular users still can't use that to add old home video releases, they can use it only for books for now. For home video releases, Amazon can only be added after a release has been made. I'm currently in the process of adding Amazon links for 2018-2020 home video releases, and I'm almost done for 2021 releases.

On the bright side, it does at least still find ISBNs, and has both ISBN-10 and ISBN-13, so I don't have to spend time converting ISBN-13 to ISBN-10 anymore even though ISBN-10 was finally phased out a few years ago.

Rakuten, yeah there's not much of a point in listing them anymore. There's only one reason to, for some reason it's still managing to find cover images, which is useful for old releases which still lack them, but this affects just encyclopedia editors and admins, not regular users. You can pretty much ignore Rakuten until it likely eventually gets removed from the list.

[Edit] BTW, for books, I still recommend adding a release with just a RightStuf link, then after the release is created, add the Barnes & Noble and Amazon links, and it'll get everything filled out quicker.
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
bglassbrook



Joined: 29 Aug 2006
Posts: 1243
Location: Gaithersburg, MD
PostPosted: Sun Dec 12, 2021 2:28 pm Reply with quote
BigOnAnime wrote:
[Edit] BTW, for books, I still recommend adding a release with just a RightStuf link, then after the release is created, add the Barnes & Noble and Amazon links, and it'll get everything filled out quicker.

It does seem to load relatively quick with all three, except...
1 - There's a lot more "stuff" to load on a release page, which makes for more jumping around on the page, and longer time to process.
2 - RightStuf usually only has pre-orders going out up-to six-months, while the other two will list anything solicited.
3 - Historically, scrapping from RightStuf is the most likely to treat novels as manga, although my testing in the last couple weeks hasn't shown a different order (or not including them) to be any more effective.
4 - The scrapper seems to prioritize RightStuf over the other two (example: In Another World With My Smartphone volume 3 shows the wrong cover art, because they are using the volume 2 art,) so as long as it is present, is there harm in including the other two sites?

On the other hand, the release page seems to be more effective at finding them from B&N and Amazon, so there's a chance that method will result in less URL submissions. [EDIT: I see where you're going with it now... submit RightStuf only, add B&N to the release page, and everything links & pops instantly.] Hopefully we can get bulk input, once there has been sufficient time to recover from smacking Amazon until it worked.
Back to top
View user's profile Send private message My Anime My Manga
BigOnAnime
Encyclopedia Editor


Joined: 01 Jul 2010
Posts: 1246
Location: Minnesota, USA
PostPosted: Sun Dec 12, 2021 7:46 pm Reply with quote
bglassbrook wrote:
4 - The scrapper seems to prioritize RightStuf over the other two (example: In Another World With My Smartphone volume 3 shows the wrong cover art, because they are using the volume 2 art,) so as long as it is present, is there harm in including the other two sites?
There's no harm there. One of the things encyclopedia editors and admins have to do for releases is select a cover image. What you usually see is a temporary image from one of the stores. We select images from the stores or manually upload them (there's an image cache for each of the stores, even for long delisted releases which is very useful), and that ends up being the image you'll see every time, as well as in MyAnime and MyManga when moving a release between your collection lists.
Quote:
On the other hand, the release page seems to be more effective at finding them from B&N and Amazon, so there's a chance that method will result in less URL submissions. [EDIT: I see where you're going with it now... submit RightStuf only, add B&N to the release page, and everything links & pops instantly.] Hopefully we can get bulk input, once there has been sufficient time to recover from smacking Amazon until it worked.
Yeah, I noticed when I added a release with just a RightStuf URL, then after the release was made, added the B&N link, it filled everything out way faster. I don't know why the scraper has difficulty pulling stuff when you used multiple links to create the release.
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
bglassbrook



Joined: 29 Aug 2006
Posts: 1243
Location: Gaithersburg, MD
PostPosted: Mon Sep 19, 2022 10:51 pm Reply with quote
Amazon's been rather flighty for the last few days, but hopefully just a random weekend thing. Sadly B&N was volunteering itself more often (since adding that one is a good way to get profile pops,) but may be related.
Back to top
View user's profile Send private message My Anime My Manga
BigOnAnime
Encyclopedia Editor


Joined: 01 Jul 2010
Posts: 1246
Location: Minnesota, USA
PostPosted: Sat Sep 24, 2022 2:47 am Reply with quote
The Amazon scraper sadly seems to have broke again, so it's going to have to get adjusted. The RightStuf scraper broke again a few months back as well, but it got fixed quickly, so it doesn't surprise me too much something changed with Amazon as well.
animenewsnetwork.com/bbs/phpBB2/viewtopic.php?t=3177820
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
bglassbrook



Joined: 29 Aug 2006
Posts: 1243
Location: Gaithersburg, MD
PostPosted: Sun Oct 16, 2022 9:59 am Reply with quote
Had a few hiccups this weekend where Shortcake Cake manga volume 9 refused to take the RightStuf link. It kept saying there was a blank page, when it is still a current listing. However, it would take the link after-the-fact, but I get the feeling the threshold for acceptance is much lower. This problem also happened on Tamamo-chan's a Fox! volumes 2 and 6.
Back to top
View user's profile Send private message My Anime My Manga
BigOnAnime
Encyclopedia Editor


Joined: 01 Jul 2010
Posts: 1246
Location: Minnesota, USA
PostPosted: Sun Oct 16, 2022 9:31 pm Reply with quote
Sometimes you have to wait a while, I don't know why, but that's how it's been in the past. Like the very moment a new Discotek slate went up, sometimes I'd have to wait a day or two before I could add the RightStuf links.
Back to top
View user's profile Send private message Visit poster's website My Anime My Manga
IúkaSylvie



Joined: 18 Apr 2019
Posts: 79
Location: Kyoto, Japan
PostPosted: Thu Nov 24, 2022 8:59 pm Reply with quote
Back to top
View user's profile Send private message Visit poster's website
IúkaSylvie



Joined: 18 Apr 2019
Posts: 79
Location: Kyoto, Japan
PostPosted: Mon Dec 05, 2022 3:16 am Reply with quote
Back to top
View user's profile Send private message Visit poster's website
IúkaSylvie



Joined: 18 Apr 2019
Posts: 79
Location: Kyoto, Japan
PostPosted: Tue Dec 13, 2022 11:45 pm Reply with quote
Back to top
View user's profile Send private message Visit poster's website
bglassbrook



Joined: 29 Aug 2006
Posts: 1243
Location: Gaithersburg, MD
PostPosted: Sat Nov 25, 2023 11:36 am Reply with quote
Are there plans for adding the Crunchyroll store, and/or some other major retailer, with the retirement of the RightStuf site? While just soldiering on with the two remaining options is better than nothing, it is going to create significantly more need for corrections, as we don't have the ability to designate a format (most notably graphic vs. light novel, such as Sylvie's first entry above, or e-books of either) when adding the release.
Back to top
View user's profile Send private message My Anime My Manga
Display posts from previous:   
Reply to topic    Anime News Network Forum Index -> Site-related -> Encyclopedia All times are GMT - 5 Hours
Goto page Previous  1, 2, 3, 4
Page 4 of 4

 


Powered by phpBB © 2001, 2005 phpBB Group