Switch individual metadata fetch with bulk solr #11219
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR switches from individual metadata fetches to bulk Solr queries for ebook availability information. Instead of making individual calls to Internet Archive's metadata API for each book's availability status, the implementation now fetches ebook access information for all books in a single Solr query.
Key changes:
- Replaces individual
get_ia_availability()calls with bulk Solr query inadd_availability() - Removes the
get_ia_availability()function that made individual IA metadata calls - Updates the processing pipeline to use the new bulk availability fetching approach
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
318b182 to
e1ef793
Compare
f3197e1 to
b525402
Compare
for more information, see https://pre-commit.ci
`preview` is now required in certain contexts. To make that context clear, I added type hints to most of dynlinks to make sure it's only being added when it aught be. Also switch from Solr.select to Solr.get_many ; that's in theory a bit more performant.
Test Cases
Dynlinks:Borrowable + Data Borrowable + View Full + View Full + Data Full + Details ReadlinksBorrowable |
The key in the dictionary is NOT an open library key, but a "bibkey" eg `OCAID:goody`
Closes #11206
Technical
Testing
Screenshot
Stakeholders