Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: webrecorder/browsertrix
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: c73512d
Choose a base ref
...
head repository: webrecorder/browsertrix
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: e995811
Choose a head ref
  • 9 commits
  • 54 files changed
  • 4 contributors

Commits on May 14, 2025

  1. chore: Clean up data grid component (#2604)

    - Moves data grid styles to separate stylesheet.
    - Adds `rowsSelectable` option, renames `rows-` properties to match.
    - Adds WIP `rowsExpandable` option.
    - Fixes showing tooltip on focus.
    - Cleans up rows controller typing.
    
    ---------
    
    Co-authored-by: Emma Segal-Grossman <hi@emma.cafe>
    SuaYoo and emma-sg authored May 14, 2025
    Configuration menu
    Copy the full SHA
    7c9627f View commit details
    Browse the repository at this point in the history
  2. feat: Workflow "latest crawl" tab (#2605)

    - Combines "Watch" and "Logs" into single "Latest Crawl" tab
    - Updates workflow routes and adds redirects
    - Enables replaying and downloading latest crawl from the workflow
    detail view
    - Tweaks crawl list table header labels and and archived item download
    button labels for consistency
    - Fixes crawl queue showing error when stopping crawl
    SuaYoo authored May 14, 2025
    Configuration menu
    Copy the full SHA
    4b1e416 View commit details
    Browse the repository at this point in the history

Commits on May 19, 2025

  1. storage: use s3v4 signature for presigning urls (#2611)

    Use V4 ('s3v4') signature version for for all presigning URLs to support
    backblaze, fixes #2472
    - add 'access_addressing_style' to be able to choose virtual/path
    addressing for access endpoint (default to 'virtual' as before)
    - fix minio presigning with v4 by using 'path' addressing style for
    minio
    - if path matches '/data/' for internal minio bucket, then always use
    'path'
    - also make minio access path '/data/' configurable
    
    also simplify running in any namespace with default settings:
    - don't hardcode 'local-minio.default'
    - in crawlers namespace, add a 'local-minio' externalName service which
    maps to the main namespace service.
    ikreymer authored May 19, 2025
    Configuration menu
    Copy the full SHA
    f1fd11c View commit details
    Browse the repository at this point in the history

Commits on May 20, 2025

  1. Optimize presigning for replay.json (#2516)

    Fixes #2515.
    
    This PR introduces a significantly optimized logic for presigning URLs
    for crawls and collections.
    - For collections, the files needed from all crawls are looked up, and
    then the 'presign_urls' table is merged in one pass, resulting in a
    unified iterator containing files and presign urls for those files.
    - For crawls, the presign URLs are also looked up once, and the same
    iterator is used for a single crawl with passed in list of CrawlFiles
    - URLs that are already signed are added to the return list.
    - For any remaining URLs to be signed, a bulk presigning function is
    added, which shares an HTTP connection and signing 8 files in parallels
    (customizable via helm chart, though may not be needed). This function
    is used to call the presigning API in parallel.
    ikreymer authored May 20, 2025
    Configuration menu
    Copy the full SHA
    c134b57 View commit details
    Browse the repository at this point in the history
  2. docs: Document latest crawl (#2613)

    Follows #2603
    
    ## Changes
    
    - Updates documentation on "Latest Crawl" tab
    - Fixes extra fetch in workflow detail page
    - Reverts workflow detail labels from "Duration" back to "Run Duration"
    and "Pages" back to "Pages Crawled"
    SuaYoo authored May 20, 2025
    Configuration menu
    Copy the full SHA
    ef93c5a View commit details
    Browse the repository at this point in the history
  3. tests: fix nightly test config after #2611 (#2614)

    remove namespace from minio config to match settings
    ikreymer authored May 20, 2025
    Configuration menu
    Copy the full SHA
    e29db33 View commit details
    Browse the repository at this point in the history
  4. Add Org Check for Collection access (#2616)

    Ensure collection access checks org membership
    ikreymer authored May 20, 2025
    Configuration menu
    Copy the full SHA
    86e35e3 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2025

  1. remove deleted collections from crawlconfigs (#2615)

    simplified version of #2608, add a remove_collection_from_all_configs() in CrawlConfigs, also check org.
    update tests to ensure removal
    ikreymer authored May 21, 2025
    Configuration menu
    Copy the full SHA
    8a71315 View commit details
    Browse the repository at this point in the history
  2. version: bump to 1.16.2

    ikreymer committed May 21, 2025
    Configuration menu
    Copy the full SHA
    e995811 View commit details
    Browse the repository at this point in the history
Loading