Zamani et al., 2020 - Google Patents

Submarine: A subscription‐based data streaming framework for integrating large facilities and advanced cyberinfrastructure

Zamani et al., 2020

View PDF
Document ID
2446214734403612296
Author
Zamani A
AbdelBaky M
Balouek‐Thomert D
Villalobos J
Rodero I
Parashar M
Publication year
Publication venue
Concurrency and Computation: Practice and Experience

External Links

Snippet

Large scientific facilities provide researchers with instrumentation, data, and data products that can accelerate scientific discovery. However, increasing data volumes coupled with limited local computational power prevents researchers from taking full advantage of what …
Continue reading at onlinelibrary.wiley.com (PDF) (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30067—File systems; File servers
    • G06F17/30129—Details of further file system functionalities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogramme communication; Intertask communication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30861—Retrieval from the Internet, e.g. browsers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/10—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network
    • H04L67/1097—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network for distributed storage of data in a network, e.g. network file system [NFS], transport mechanisms for storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/10—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network
    • H04L67/104—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network for peer-to-peer [P2P] networking; Functionalities or architectural details of P2P networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/28—Network-specific arrangements or communication protocols supporting networked applications for the provision of proxy services, e.g. intermediate processing or storage in the network
    • H04L67/2823—Network-specific arrangements or communication protocols supporting networked applications for the provision of proxy services, e.g. intermediate processing or storage in the network for conversion or adaptation of application content or format
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a programme unit and a register, e.g. for a simultaneous processing of several programmes
    • G06F15/163—Interprocessor communication
    • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/02—Network-specific arrangements or communication protocols supporting networked applications involving the use of web-based technology, e.g. hyper text transfer protocol [HTTP]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Error detection; Error correction; Monitoring responding to the occurence of a fault, e.g. fault tolerance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/42—Protocols for client-server architectures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements or protocols for real-time communications
    • H04L65/40—Services or applications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Application independent communication protocol aspects or techniques in packet data networks

Similar Documents

Publication Publication Date Title
US20230004434A1 (en) Automated reconfiguration of real time data stream processing
CN109074377B (en) Managed function execution for real-time processing of data streams
US10560465B2 (en) Real time anomaly detection for data streams
US11676066B2 (en) Parallel model deployment for artificial intelligence using a primary storage system
US11061942B2 (en) Unstructured data fusion by content-aware concurrent data processing pipeline
US9619148B2 (en) Distributed data set storage and retrieval
Kamburugamuve et al. A framework for real time processing of sensor data in the cloud
Calderon et al. Monitoring framework for the performance evaluation of an IoT platform with Elasticsearch and Apache Kafka
US9811575B2 (en) Dynamic distributed generation of data representations from highly condensed data
Renart et al. An edge-based framework for enabling data-driven pipelines for iot systems
US20170277562A1 (en) Threadsafe use of non-threadsafe libraries with multi-threaded processes
US11895192B1 (en) Managing subscriptions to resource updates made via a target interface
Zamani et al. Submarine: A subscription‐based data streaming framework for integrating large facilities and advanced cyberinfrastructure
CN114731342B (en) Managed data export from edge devices to remote networks
US11757703B1 (en) Access requests processing and failover handling across multiple fault tolerance zones
Kuzlu et al. An end to end data collection architecture for iot devices in smart cities
CN117194562B (en) Data synchronization method and device, electronic equipment and computer readable medium
CN115695426A (en) Data processing method, device and equipment in multi-cluster scene and storage medium
Zhang et al. Efficient online surveillance video processing based on spark framework
WO2016201992A1 (en) Video storage and retrieval method for cloud storage server, and video cloud storage system
Darweesh et al. Real time adaptive approach for image processing using mobile nodes
Alaa El Din Talha A semantic based annotation technique for the internet of things
US11954541B1 (en) Highly available message ingestion by a data intake and query system
US12547482B1 (en) Cloud-based actions service for a data intake and query system
CN111858260A (en) Information display method, device, equipment and medium