WO2024216336A1 - Technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of time-series eyelid amplitude data collected during conscious state - Google Patents

Technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of time-series eyelid amplitude data collected during conscious state Download PDF

Info

Publication number
WO2024216336A1
WO2024216336A1 PCT/AU2024/050369 AU2024050369W WO2024216336A1 WO 2024216336 A1 WO2024216336 A1 WO 2024216336A1 AU 2024050369 W AU2024050369 W AU 2024050369W WO 2024216336 A1 WO2024216336 A1 WO 2024216336A1
Authority
WO
WIPO (PCT)
Prior art keywords
blink
artefacts
artefact
data
sleep apnoea
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/AU2024/050369
Other languages
French (fr)
Inventor
Scott Coles
Trefor Morgan
Hedi Ziv
Ali ALMASI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sdip Holdings Pty Ltd
Original Assignee
Sdip Holdings Pty Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2023901148A external-priority patent/AU2023901148A0/en
Application filed by Sdip Holdings Pty Ltd filed Critical Sdip Holdings Pty Ltd
Priority to EP24791588.7A priority Critical patent/EP4698042A1/en
Priority to AU2024256955A priority patent/AU2024256955A1/en
Publication of WO2024216336A1 publication Critical patent/WO2024216336A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/4806Sleep evaluation
    • A61B5/4818Sleep apnoea
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room
    • A61B5/004Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1103Detecting muscular movement of the eye, e.g. eyelid movement
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1126Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb using a particular sensing technique
    • A61B5/1128Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb using a particular sensing technique using image analysis
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/16Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/163Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271Specific aspects of physiological measurement analysis
    • A61B5/7275Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271Specific aspects of physiological measurement analysis
    • A61B5/7282Event detection, e.g. detecting unique waveforms indicative of a medical condition
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/59Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18Eye characteristics, e.g. of the iris
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18Eye characteristics, e.g. of the iris
    • G06V40/193Preprocessing; Feature extraction
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/16Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/18Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6887Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient mounted on external non-worn devices, e.g. non-medical devices
    • A61B5/6893Cars
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7264Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/59Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597Recognising the driver's state or behaviour, e.g. attention or drowsiness

Definitions

  • the present invention relates, in various embodiments, to technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of timeseries eyelid amplitude data collected during conscious state (i.e. during wakefulness).
  • Risk factors may include a metric representative of a risk that the subject suffers from sleep apnoea (e.g. OSA), and in some cases metrics representative of a predicted level of severity associated with that sleep apnoea.
  • Embodiments are described by reference to testing technology and methods which are conducted whilst a subject is conscious and performing a known task, with eyelid amplitude data collected during testing being processed to extract a plurality of first derivative artefacts which are used for the purposes of sleep apnoea risk determinations.
  • eyelid amplitude data collected during testing being processed to extract a plurality of first derivative artefacts which are used for the purposes of sleep apnoea risk determinations.
  • the technology is not limited as such, and has application in a broader range of context.
  • OSA Active Sleep Apnoea
  • any one of the terms comprising, comprised of or which comprises is an open term that means including at least the elements/features that follow, but not excluding others.
  • the term comprising, when used in the claims should not be interpreted as being limitative to the means or elements or steps listed thereafter.
  • the scope of the expression a device comprising A and B should not be limited to devices consisting only of elements A and B.
  • Any one of the terms including or which includes or that includes as used herein is also an open term that also means including at least the elements/features that follow the term, but not excluding others. Thus, including is synonymous with and means comprising.
  • exemplary is used in the sense of providing examples, as opposed to indicating quality. That is, an “exemplary embodiment” is an embodiment provided as an example, as opposed to necessarily being an embodiment of exemplary quality.
  • FIG. 1A illustrates an in-vehicle monitoring system according to one embodiment.
  • FIG. 1 B illustrates an in-vehicle monitoring system according to one embodiment.
  • FIG. 2 illustrates a method according to one embodiment.
  • the present invention relates, in various embodiments, to technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of timeseries eyelid amplitude data collected during conscious state (i.e. during wakefulness).
  • Risk factors may include a metric representative of a risk that the subject suffers from sleep apnoea (e.g. OSA), and in some cases metrics representative of a predicted level of severity associated with that sleep apnoea.
  • Embodiments are described by reference to testing technology and methods which are conducted whilst a subject is conscious and performing a known task, with eyelid amplitude data collected during testing being processed to extract a plurality of first derivative artefacts which are used for the purposes of sleep apnoea risk determinations.
  • eyelid amplitude data collected during testing being processed to extract a plurality of first derivative artefacts which are used for the purposes of sleep apnoea risk determinations.
  • the technology is not limited as such, and has application in a broader range of context.
  • a metric representative of a sleep apnoea risk factors For example, this metric may be representative of a risk of obstructive sleep apnoea (OSA), optionally in combination with a severity level (e.g. a risk of moderate or intense OSA).
  • OSA obstructive sleep apnoea
  • a severity level e.g. a risk of moderate or intense OSA.
  • the technology may in some embodiments be additionally and/or alternately applied in relation to other forms of sleep apnoea. More generally, technology described herein is in some embodiments applied in respect of other forms of sleep hygiene, such as ESS, excessive daytime sleepiness, and the like).
  • OSAM Obstructive Sleep Apnoea Metric
  • the metric representative of sleep apnoea risk factors is additionally associated with a severity metric.
  • OSAM output labels are provided as guidance for the reader: “OSA likelihood detected”; “OSA likelihood not detected”; “mild OSA likelihood detected”; “moderate OSA likelihood detected”; “severe OSA likelihood detected”; “high OSA likelihood detected”; “medium OSA likelihood detected”; “low OSA likelihood detected”; and/or combinations of the foregoing. It will be appreciated that these are examples only, and that such labels are associated numerical values provided via a data processing system as described further below.
  • an output might be presented as, for instance “your results conclude that there’s a >80% chance/probability that you suffer from moderate OSA”.
  • the monitoring includes collection of time-series eyelid amplitude data, which is subsequently processed thereby to detect individual blink events (including eyelid movement events, which occur in the context of blinking) and, for each of the individual blink events, generate a plurality of blink artefacts.
  • the blink artefacts include blink artefacts derived from of time-series amplitude data.
  • the present inventors have identified that a range of such artefacts are of particular use for the purposes of identifying sleep apnoea risks (e.g. OSA) for a particular individual.
  • the blink artefacts derived from time series amplitude data preferably include one or more of the following:
  • n th and n+1 th derivatives of the time series amplitude data where n is an integer greater than or equal to zero. This is referred to as use of “multi-ordered derivative artefacts” for the purposes of this specification. It will be appreciated that 0 th derivative artefacts may include the likes of maximum amplitude, relative amplitude, relative position, and the like.
  • a combination of artefacts including at least one artefact derived from a first derivative of the time series amplitude data.
  • artefacts derived from a first derivative of the time series amplitude data may include eyelid closure velocities and/or eyelid opening velocities, positive and negative amplitude-to-velocity ratios, and many others. Those familiar with blepharometric analysis techniques will appreciate a wide range of such artefacts.
  • temporal artefact is used to describe a data artefact for a blink event that is defined by a time period between two objectively identifiable sub-events within the blink event.
  • temporal artefacts used in the context of blepharometric analysis include one or more of: blink total duration; inter eyelid movement event duration (positive and/or negative); zero crossing intervals (positive and/or negative); duration of ocular quiescence (positive and/or negative); blink eyelid closed duration.
  • inter-blink artefact is used to describe artefacts which are determined based on a combination of two or more detected blink events in the time series amplitude data, for example using objectively defined start/end points for each blink event. Such “inter-blink artefacts” include blink rate/frequency, inter-blink duration, and the like.
  • temporary artefact specifically excludes any “inter-blink artefacts” from a definitional perspective. This clearly differentiates temporal artefacts from individual blink events from blink frequency metrics and the like.
  • Additional data artefacts may optionally, in some embodiments, be used in combination with those derived from the time series amplitude data.
  • data artefacts are extracted from monitoring of eye movement.
  • Example blink artefacts from the various artefact categories are provided in the section further below headed “example blink artefacts”.
  • the monitoring is facilitated by existing in-vehicle driver monitoring systems.
  • Some examples of such systems are presently configured to enable generation of time-series blink amplitude data, for example based on image processing of frames generated by a video capture system (these may include facial image data, for instance image data which includes at least part of a face).
  • Such technology has previously been deployed for the purposes of detecting drowsiness in drivers.
  • the technology described herein allows for such existing systems to be applied thereby to enable determination of sleep apnoea risks for vehicle operators. This in some cases is achieved via installation of additional software in an existing driver monitoring system.
  • data processing required for the purposes of determination of sleep apnoea risks is optionally performed by or in conjunction with processing components external of the vehicle.
  • eyelid amplitude data collection may include other in-vehicle monitoring systems such as occupant monitoring systems (e.g. in cars, aeroplanes, trains and other mass transport vehicles). Outside of a vehicle environment, this may include the use of cameras which monitor the face of a subject using a computer, mobile device, or other hardware including a screen, thereby to collect the eyelid amplitude data whilst the subject uses that device (including either or both of unstructured usage and usage comprising performance of designated tasks; in some embodiments eyelid movement data is collected during videoconferencing as an additional functionality for a videoconferencing platform).
  • occupant monitoring systems e.g. in cars, aeroplanes, trains and other mass transport vehicles.
  • eyelid amplitude data e.g. in cars, aeroplanes, trains and other mass transport vehicles.
  • eyelid movement data is collected during videoconferencing as an additional functionality for a videoconferencing platform.
  • Data collection hardware may include cameras and/or wearable devices (for example infrared oculography hardware, and VR/AR goggles and/or headsets).
  • contextual data is collected, thereby to provide context to eyelid movement data.
  • contextual data may include whether a subject is speaking or listening. In the context of driving, this may include whether the vehicle is in motion (and optionally other factors such as driving speed, urban/freeway style of driving, and so on).
  • the time series data input representative of eyelid amplitude is collected from the subject during a “testing period”.
  • This testing period is preferably an “unstimulated” period, meaning that there are no controlled stimuli delivered to the user as part of the testing protocol to trigger blinks, or otherwise instruct voluntary blinks (i.e. monitoring is performed without deliberate stimulation of blink events). That is, the monitoring is employed such that blink events detected for data corresponding to the testing are likely to be spontaneous involuntary blinks. It will be appreciated by those skilled in the art that characteristics of spontaneous involuntary blinks are capable of revealing a range of neurological conditions, given physiological mechanisms which control the relevant movements.
  • OSAM data for a given subject is tracked over an extended period of time, thereby to monitor changes in metric calculations. This is optionally applied thereby to assess the effectiveness of sleep apnoea treatments to which the subject is subjected. These treatments may include medical and/or non-medical interventions.
  • the OSAM includes a value defined on a scale representative of predicted severity of OSA, allowing a user to track their progress over time in terms of susceptibility/severity.
  • FIG. 2 one embodiment provides a method for determining a metric representative of sleep apnoea risk factors for a human subject.
  • Block 201 represents a process including receiving time series data input representative of eyelid amplitude for the subject during a testing period. For example, this may be derived from image processing techniques such as those disclosed in PCT patent publication W02020082125, whereby image frames collected by a capture device are processed thereby to define eyelid amplitude data for each frame and from that generate time series data (optimally including processes such as interpolation thereby to generate a sampling rate greater than a camera frame rate). Alternately, hardware such as that used for infrared oculography may be employed.
  • Block 202 represents a process including processing the time series data thereby to detect a plurality of blink events.
  • a process for identifying blink events may include applying algorithms to time series data thereby to identify specific markers which objectively delineate the start and end of a blink, for example based on changes and/or zero values for one or more multi-ordered derivative artefacts. In a simple case, this is used to identify a point where eyelid closure commences and a point where eyelid re-opening completes, and define a blink event as the time-series data between those points. Definitions for what constitutes start and end points for blink events may vary between embodiments. Blink events may be designated with sequential identifiers, thereby to facilitate analysis which is based on processing of data relating to consecutive blink events (e.g. inter-event artefacts).
  • Block 203 represents, for each blink event, determining a plurality of blink event artefacts, for example multi-ordered derivative artefacts and/or temporal artefacts (optionally in combination with inter-blink artefacts).
  • Block 204 represents a process including compiling one more artefact data sets, wherein each artefact data set includes blink artefacts (including blink artefacts derived from a first derivative of time-series amplitude). Each artefact data set corresponds to a defined plurality of blink events during the testing period.
  • block 204 includes two distinct sub-processes: • Block 204(a): segregating blink events into delineated blocks. These may be temporally delineated blocks. For example, each delineated block corresponds to a X-second time period, wherein Xis a value between 30 and 240 (or more preferably between 45 and 120). A preferred embodiment makes use of 60-second blocks. Alternately, segregation may be based on other factors, for example segregation such that each block contains an equal (e.g. predefined) number of blink events, for example between about 10 and 30 blink events (although other block sizes may be used).
  • Block 204(b) for each delineated block, performing statistical analysis of the blink artefacts thereby to derive block-specific blink artefact statistics. These are referred to as “block-based statistical artefacts”, being a second order collection of artefacts derived from multi-ordered derivative, temporal and/or inter-blink artefacts for blink events delimitated into a given block.
  • the process of deriving block-specific blink artefact statistics for each delineated block preferably includes applying a set of statistical analysis algorithms to the blink artefacts derived at block 203 for blink events corresponding to that block.
  • this includes Mean, Standard Deviation, Percentiles, and further Higher-Order Statistics.
  • Block 205 represents a process including processing each of the artefact data sets thereby to determine a metric representative to a risk of sleep apnoea. This preferably includes the following sub-processes:
  • This output value is in some embodiments a value between zero and one.
  • Block 205(b) Determining the metric based on processing of a plurality of output values derived from the plurality of artefact data sets. For example, where values are outputted by the process of block 205(a), the metric may be based on output of the first layer. This may result in a risk metric defined on a scale (for example representative of low risk to high risk), defined in a binary manner (for example “not determined to be at risk” or “determined to be at risk”), and/or include information relating to predicted severity level (e.g. mild/moderate/severe sleep apnoea). Majority decisions, heuristics and/or other techniques may be used to determine the metric at this stage.
  • a scale for example representative of low risk to high risk
  • a binary manner for example “not determined to be at risk” or “determined to be at risk”
  • Majority decisions, heuristics and/or other techniques may be used to determine the metric at this stage.
  • the classifier module preferably utilises a machine learning model (or other form of technology, such as Al technology, which enables classification).
  • a machine learning model or other form of technology, such as Al technology, which enables classification.
  • Various machine learning models may optionally be trained using various features engineered and described above.
  • An instance of such models includes a layered architecture of a Random Forest (RF) model stacked on top of multiple Decision Tree (DT) models.
  • Other model instances can include, but are not limited to, more black box models such as deep neural network (DNN) models like Deep Convolutional Neural Networks (DCNN), Recurrent Neural Networks (RNN) and Long-Short-Term Memory (LSTM) networks.
  • DNN deep neural network
  • DCNN Deep Convolutional Neural Networks
  • RNN Recurrent Neural Networks
  • LSTM Long-Short-Term Memory
  • the training data sets may be labelled based on a binary “sleep apnoea sufferer” or “no sleep apnoea” labelling protocol, or via a more complex protocol which separates “sleep apnoea sufferer” into condition levels such as mild, moderate and severe.
  • decision tree and/or other algorithmic logic may be configured to convert a numerical classifier output into one or more descriptive labels belonging to a predefined schema.
  • FIG. 1A illustrates an example in-vehicle monitoring system configured to enable prediction of sleep apnoea risk factors as described herein. It will be appreciated that this is an example only, and that a wide range of known commercial systems are configured to perform eyelid position tracking may be modified for the purposes of assessing sleep apnoea risk factors as discussed herein. For example, such known systems already record time series data for eyelid amplitude, and in some cases extract a range of artefacts from such data. Known systems such as these are optionally provided with additional software functionality (locally and/or in cloud-hosted systems) thereby to facilitate processing leading to generation of sleep apnoea relevant metric such as an OSAM based on data already being collected from a driver.
  • additional software functionality locally and/or in cloud-hosted systems
  • the system of FIG. 1A includes an image capture device 120.
  • This may include substantially any form of appropriately sized digital camera, preferably a digital camera with a frame rate of over 30 frames per second.
  • Higher frame rate cameras are advantageous, given that with enhanced frame rate comes an ability to obtain higher resolution data for eyelid movement.
  • other embodiments use cameras with frame rates of between 60 and 240 frames per second.
  • Device 120 is positioned to capture a facial region of a subject.
  • Device 120 is in one embodiment installed in a region of a vehicle in the form of an automobile, for example on or adjacent the dashboard, windscreen, or visor, such that it is configured to capture a facial region of a driver.
  • device 120 is positioned on or adjacent the dashboard, windscreen, or visor, such that it is configured to capture a facial region of a front seat passenger.
  • device 120 is positioned in a region such as the rear of a seat such that it is configured to capture a facial region of a back-seat passenger. In some embodiments a combination of these is used, thereby to enable eyelid movement monitoring for both a driver and one or more passengers.
  • FIG. 1A Although the system of FIG. 1A (and other systems) are described by reference to a vehicle in the form of an automobile, it will be appreciated that a system as described is also optionally implanted in other forms of vehicles, including mass-transport vehicles such as passenger airplanes, buses/coaches, and trains. In such embodiments there are preferably one or more analysis systems each supporting a plurality of image capture devices, each positioned to capture a respective passenger. Furthermore, the technology disclosed herein may also be implemented away from vehicles, for example to allow for OSA assessments to be performed in pharmacies, laboratories and at home (for example using computing devices such as PCs, laptops, smartphones and tablets with a screen that provides a “task” for a subject, and a camera to record eyelid movement whilst the task is performed).
  • OSA assessments for example to allow for OSA assessments to be performed in pharmacies, laboratories and at home (for example using computing devices such as PCs, laptops, smartphones and tablets with a screen that provides a “task” for a subject, and a camera to
  • An in-vehicle image processing system 110 is configured to receive image data from image capture device 120 (or multiple devices 120), and process that data thereby to generate time-series eyelid amplitude data.
  • a control module 111 is configured to control device 120, operation of image data processing, and management of generated data. This includes controlling operation of image data processing algorithms, which are configured to:
  • Identify, in the eye region(s), presence and movement of an eyelid For example, in a preferred embodiment this is achieved by way of recording an eyelid position relative to a defined “closed” position against time. This allows generation of timeseries eyelid amplitude data in the form of eyelid position (amplitude) over time. It will be appreciated that such data provides for identification of events (for example blink events) and velocity (for example as a first derivative of position against time).
  • Algorithms 112 optionally operate to extract artefacts from the time-series eyelid amplitude data, for example temporal artefacts, multi-ordered derivative artefacts and in some cases additionally inter-blink artefacts. It will be appreciated, however, that extraction of such artefacts may occur in downstream processing.
  • a data management module 113 is configured to coordinate storage of data generated by algorithms 112 in user time-series eyelid amplitude data 152.
  • the function of module 113 includes determining whether a set of generated time-series eyelid amplitude data meets threshold data quality requirements for storage, for example based on factors including a threshold unbroken time period for which eyelid tracking is achieved and time-series eyelid amplitude data is generated.
  • memory system 150 includes user identification data 151 for one or more users.
  • system 101 is configured to collect and analyse time-series eyelid amplitude data for only a single user (for instance the primary driver of a vehicle) and includes identification data to enable identification of only that user.
  • system 101 includes functionality to collect and analyse time-series eyelid amplitude data for multiple users, and includes identification data to enable identification of any of those users (and optionally, as noted above, defining of a new record for a previously unknown user).
  • the identification data may include login credentials (for example a user ID and/or password) which are inputted via an input device.
  • the identification data may be biometric, for example using facial recognition as discussed above or an alternate biometric input (such as a fingerprint scanner). In some embodiments this leverages an existing biometric identification system of a device/vehicle/other.
  • Blink artefact data 152 includes is configured to time-series eyelid amplitude data for one or more testing periods, and optionally/additionally store associated blink artefact data and/or delineated block-specific blink artefact data sets as described herein (although noting that various processing steps may instead be performed externally of system 101).
  • Analysis modules 130 are configured to cause analysis of one or more artefact data sets, each artefact data set being derived from the blink artefacts, including blink artefacts derived from a first derivative of time-series amplitude, for a defined plurality of blink events during a given testing period. This may include analysis relating to driver alertness monitoring, and associated notifications and/or intervention management.
  • module 130 additionally causes processing each of a plurality of the artefact data sets (e.g. via a classifier, which may be cloud hosted) thereby to determine a metric representative to a risk of sleep apnoea, such as an OSAM.
  • System 101 additionally includes a communication system 160, which is configured to communicate information from system 101 to human users.
  • This may include internal communication modules 161 which provide output data via components installed in the vehicle, for example an in-car display, warning lights, and so on.
  • External communication modules 162 are also optionally present, for example to enable communication of data from system 101 to user devices (for example via Bluetooth, WiFi, or other network interfaces), optionally by email or other messaging protocols.
  • communication system 160 is optionally configured to facilitate communication results of analysis by analysis modules 130.
  • a control system 140 included logic modules 141 which control overall operation of system 140. This includes execution of logical rules thereby to determine communications to be provided in response to outputs from analysis modules 130. For example, this may include in-vehicle notifications. These in-vehicle notifications might be unrelated to sleep apnoea assessment, for example where blink event data is processed for more time-critical purposes such as assessing impairment/drowsiness. An OSAM or other apnoea related metric may be communicated via the vehicle, or that function may be responsibility of a separate cloud-based system. It will be appreciated that these are examples only, and logic modules 140 are able to provide a wide range of functionalities thereby to cause system 101 to act in a predefined manner.
  • FIG. 1 B illustrates a further embodiment in the form of system 10T, which includes various common features with the embodiment illustrated in FIG. 1A.
  • external communication modules 162 facilitate communication with a remote server device in the form of a data analysis system 180, which performs analysis of time-series eyelid amplitude data analysis. This communication may rely on various intermediary devices to facilitate network connectivity (e.g. including a smartphone).
  • System 180 includes a control system 182 and logic modules 181 which are provided by computer executable code executing across one or more computing devices thereby to control and deliver functionalities of system 180.
  • System 180 additionally includes a memory system 183, which stores user testing data including and/or derived from time series eyelid amplitude data collected via system 10T, enabling such data to be passed to classifier modules for the purposes of generating an OSAM for each testing period.
  • memory system 183 optionally includes user identification data 184 thereby to facilitate tracking of changes in user-specific OASM metrics over time, for example to assess effectiveness of medical and/or non-medical treatment regimens.
  • System 180 additionally includes analysis modules 186.
  • analysis modules 186 perform analysis via classifier modules, including a classifier module which is configured to facilitate generation of an OSAM as described herein.
  • the OSAM is optionally communicated to the relevant subject via a range of technologies, including email and other electronic messaging platforms.
  • Analysis modules 130 perform in-vehicle algorithmic analysis, for example in relation to driver impairment/alertness/drowsiness. In this manner, data collected for the purposes of in-vehicle analysis as part of a driver monitoring system is complemented by cloud-based processing which delivers metrics relevant to sleep apnoea.
  • Example multi-ordered derivative artefacts include:
  • Example temporal artefacts include:
  • Blink total duration (BTD), which is preferably measured as a time between commencement of closure movement which exceeds a defined threshold and completion of subsequent opening movement.
  • Inter-event durations (positive and/or negative), which may be used to measure temporal artefacts for a period during which the eyelid is moving in a closing direction and/or opening direction.
  • BECD Blink eyelid closed duration
  • Example inter-blink artefacts include:
  • BFI Blink free interval
  • Example block-based statistical artefacts include: • Mean of any one or more blink artefact values.
  • the above disclosure provides technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of timeseries eyelid amplitude data collected during conscious state. This offers distinct advantages over existing technologies, in the sense that testing is far less obstructive/inconvenient, being performed during a wakeful state, optionally whilst using computing device, watching a screen, and/or operating a vehicle. Furthermore, the technology allows for convenient monitoring of sleep apnoea risk factors over time, thereby to enable assessment of effectiveness of treatments that are applied.
  • some of the embodiments are described herein as a method or combination of elements of a method that can be implemented by a processor of a computer system or by other means of carrying out the function.
  • a processor with the necessary instructions for carrying out such a method or element of a method forms a means for carrying out the method or element of a method.
  • an element described herein of an apparatus embodiment is an example of a means for carrying out the function performed by the element for the purpose of carrying out the invention.
  • Coupled when used in the claims, should not be interpreted as being limited to direct connections only.
  • the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other.
  • the scope of the expression a device A coupled to a device B should not be limited to devices or systems wherein an output of device A is directly connected to an input of device B. It means that there exists a path between an output of A and an input of B which may be a path including other devices or means.
  • Coupled may mean that two or more elements are either in direct physical or electrical contact, or that two or more elements are not in direct contact with each other but yet still co-operate or interact with each other.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Biophysics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physiology (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Psychiatry (AREA)
  • Ophthalmology & Optometry (AREA)
  • Evolutionary Computation (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Signal Processing (AREA)
  • Radiology & Medical Imaging (AREA)
  • Dentistry (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Child & Adolescent Psychology (AREA)
  • Developmental Disabilities (AREA)
  • Educational Technology (AREA)
  • Hospice & Palliative Care (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The present invention relates, in various embodiments, to technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of time-series eyelid amplitude data collected during conscious state (i.e. during wakefulness). Risk factors may include a metric representative of a risk that the subject suffers from sleep apnoea (e.g. OSA), and in some cases metrics representative of a predicted level of severity associated with that sleep apnoea. Embodiments are described by reference to testing technology and methods which are conducted whilst a subject is conscious and performing a known task, with eyelid amplitude data collected during testing being processed to extract a plurality of first derivative artefacts which are used for the purposes of sleep apnoea risk determinations.

Description

TECHNOLOGY CONFIGURED TO ENABLE DETERMINATION OF SLEEP APNOEA RISK FACTORS FOR SUBJECT BASED ON ARTEFACTS OF TIME-SERIES EYELID AMPLITUDE DATA COLLECTED DURING CONSCIOUS STATE
FIELD OF THE INVENTION
[0001] The present invention relates, in various embodiments, to technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of timeseries eyelid amplitude data collected during conscious state (i.e. during wakefulness). Risk factors may include a metric representative of a risk that the subject suffers from sleep apnoea (e.g. OSA), and in some cases metrics representative of a predicted level of severity associated with that sleep apnoea. Embodiments are described by reference to testing technology and methods which are conducted whilst a subject is conscious and performing a known task, with eyelid amplitude data collected during testing being processed to extract a plurality of first derivative artefacts which are used for the purposes of sleep apnoea risk determinations. However, it will be appreciated that the technology is not limited as such, and has application in a broader range of context.
BACKGROUND
[0002] Any discussion of the background art throughout the specification should in no way be considered as an admission that such art is widely known or forms part of common general knowledge in the field.
[0003] The most common form of sleep apnoea is OSA (Obstructive Sleep Apnoea). If left untreated can cause stroke, heart disease, diabetes, high blood pressure etc. OSA is believed to affect approximately 20% of US adults, of whom about studies indicate that around 80% to 90% are undiagnosed.
[0004] The current gold standard for OSA diagnosis is Polysomnography (PSG), measured during sleep. Other testing methods are beginning to be increasingly accepted in the medical sleep space, including home-based sleep testing. These approaches, whilst useful, have historically been unsuccessful in substantively reducing the extent of undiagnosed cases. In particular, there remain substantial practical and educational barriers to many people being tested.
SUMMARY OF THE INVENTION
[0005] It is an object of the present invention to overcome or ameliorate at least one of the disadvantages of the prior art, or to provide a useful alternative.
[0006] Example embodiments are described below in the sections entitled “detailed description” and “claims”.
[0007] Reference throughout this specification to “one embodiment”, “some embodiments” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases “in one embodiment”, “in some embodiments” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment, but may. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments.
[0008] As used herein, unless otherwise specified the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.
[0009] In the claims below and the description herein, any one of the terms comprising, comprised of or which comprises is an open term that means including at least the elements/features that follow, but not excluding others. Thus, the term comprising, when used in the claims, should not be interpreted as being limitative to the means or elements or steps listed thereafter. For example, the scope of the expression a device comprising A and B should not be limited to devices consisting only of elements A and B. Any one of the terms including or which includes or that includes as used herein is also an open term that also means including at least the elements/features that follow the term, but not excluding others. Thus, including is synonymous with and means comprising.
[0010] As used herein, the term “exemplary” is used in the sense of providing examples, as opposed to indicating quality. That is, an “exemplary embodiment” is an embodiment provided as an example, as opposed to necessarily being an embodiment of exemplary quality.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings in which:
[0012] FIG. 1A illustrates an in-vehicle monitoring system according to one embodiment.
[0013] FIG. 1 B illustrates an in-vehicle monitoring system according to one embodiment.
[0014] FIG. 2 illustrates a method according to one embodiment.
DETAILED DESCRIPTION
[0015] The present invention relates, in various embodiments, to technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of timeseries eyelid amplitude data collected during conscious state (i.e. during wakefulness). Risk factors may include a metric representative of a risk that the subject suffers from sleep apnoea (e.g. OSA), and in some cases metrics representative of a predicted level of severity associated with that sleep apnoea. Embodiments are described by reference to testing technology and methods which are conducted whilst a subject is conscious and performing a known task, with eyelid amplitude data collected during testing being processed to extract a plurality of first derivative artefacts which are used for the purposes of sleep apnoea risk determinations. However, it will be appreciated that the technology is not limited as such, and has application in a broader range of context.
[0016] In overview, technology described below passive monitoring of a human subject, (for example passive monitoring whilst they perform “ordinary” day-to-day tasks, and/or active monitoring whereby they perform known/designated tasks), thereby to generate a metric representative of a sleep apnoea risk factors. For example, this metric may be representative of a risk of obstructive sleep apnoea (OSA), optionally in combination with a severity level (e.g. a risk of moderate or intense OSA). However, it will be recognised that the technology may in some embodiments be additionally and/or alternately applied in relation to other forms of sleep apnoea. More generally, technology described herein is in some embodiments applied in respect of other forms of sleep hygiene, such as ESS, excessive daytime sleepiness, and the like).
[0017] For the purposes of embodiments described below, the metric representative of sleep apnoea risk factors is described by reference an Obstructive Sleep Apnoea Metric, or OSAM. This term is introduced for the purpose of convenience in description only, and that the OSAM is optionally replaced with any other metric representative of sleep apnoea risk factors (OSA and/or otherwise).
[0018] In some embodiments, the metric representative of sleep apnoea risk factors is additionally associated with a severity metric. For example, the following possible example OSAM output labels are provided as guidance for the reader: “OSA likelihood detected”; “OSA likelihood not detected”; “mild OSA likelihood detected”; “moderate OSA likelihood detected”; “severe OSA likelihood detected”; “high OSA likelihood detected”; “medium OSA likelihood detected”; “low OSA likelihood detected”; and/or combinations of the foregoing. It will be appreciated that these are examples only, and that such labels are associated numerical values provided via a data processing system as described further below.
[0019] The concepts of “sleep apnoea risk factors”, “sleep apnoea risk”, “sleep apnoea risk metrics”, and the like are used in the context of objective data which provides an indication as to a likelihood that a person suffers from sleep apnoea (such as OSA). This may be in practice presented, for instance, as a determination that a person is an OSA sufferer (e.g. that a person suffers from mild OSA, moderate OSA or severe OSA). However, it will be appreciated that predictions as to a person’s sleep apnoea status made by technology described herein are not definitive, and rather they are probabilistic (hence the use of the term “risk”). In another example, an output might be presented as, for instance “your results conclude that there’s a >80% chance/probability that you suffer from moderate OSA”. [0020] The monitoring includes collection of time-series eyelid amplitude data, which is subsequently processed thereby to detect individual blink events (including eyelid movement events, which occur in the context of blinking) and, for each of the individual blink events, generate a plurality of blink artefacts. The blink artefacts include blink artefacts derived from of time-series amplitude data. The present inventors have identified that a range of such artefacts are of particular use for the purposes of identifying sleep apnoea risks (e.g. OSA) for a particular individual. The blink artefacts derived from time series amplitude data preferably include one or more of the following:
• A combination of artefacts that nth and n+1th derivatives of the time series amplitude data, where n is an integer greater than or equal to zero. This is referred to as use of “multi-ordered derivative artefacts” for the purposes of this specification. It will be appreciated that 0th derivative artefacts may include the likes of maximum amplitude, relative amplitude, relative position, and the like.
• A combination of artefacts including at least one artefact derived from a first derivative of the time series amplitude data. For example, artefacts derived from a first derivative of the time series amplitude data may include eyelid closure velocities and/or eyelid opening velocities, positive and negative amplitude-to-velocity ratios, and many others. Those familiar with blepharometric analysis techniques will appreciate a wide range of such artefacts.
• A combination of artefacts including a plurality of temporal artefacts. The term “temporal artefact” is used to describe a data artefact for a blink event that is defined by a time period between two objectively identifiable sub-events within the blink event. For example, temporal artefacts used in the context of blepharometric analysis include one or more of: blink total duration; inter eyelid movement event duration (positive and/or negative); zero crossing intervals (positive and/or negative); duration of ocular quiescence (positive and/or negative); blink eyelid closed duration. Those familiar with blepharometric analysis techniques will appreciate a wide range of such artefacts.
• A combination of one or more temporal artefacts and one or more multi-ordered derivative artefacts. • A combination of any of the above with one or more inter-blink artefacts. The term “inter-blink artefact” is used to describe artefacts which are determined based on a combination of two or more detected blink events in the time series amplitude data, for example using objectively defined start/end points for each blink event. Such “inter-blink artefacts” include blink rate/frequency, inter-blink duration, and the like. For the purposes of this document, the term “temporal artefact” specifically excludes any “inter-blink artefacts” from a definitional perspective. This clearly differentiates temporal artefacts from individual blink events from blink frequency metrics and the like.
[0021] Additional data artefacts may optionally, in some embodiments, be used in combination with those derived from the time series amplitude data. For example, in some embodiments data artefacts are extracted from monitoring of eye movement.
[0022] Example blink artefacts from the various artefact categories (multi ordered derivative, temporal and inter-blink) are provided in the section further below headed “example blink artefacts”.
[0023] In a preferred embodiment, the monitoring is facilitated by existing in-vehicle driver monitoring systems. Some examples of such systems are presently configured to enable generation of time-series blink amplitude data, for example based on image processing of frames generated by a video capture system (these may include facial image data, for instance image data which includes at least part of a face). Such technology has previously been deployed for the purposes of detecting drowsiness in drivers. The technology described herein allows for such existing systems to be applied thereby to enable determination of sleep apnoea risks for vehicle operators. This in some cases is achieved via installation of additional software in an existing driver monitoring system. However, data processing required for the purposes of determination of sleep apnoea risks is optionally performed by or in conjunction with processing components external of the vehicle.
[0024] Whilst the present specification focusses in particular on application of the relevant processing technology in the context of in-vehicle driver monitoring systems, alternate hardware environments may be used for eyelid amplitude data collection. For example, this may include other in-vehicle monitoring systems such as occupant monitoring systems (e.g. in cars, aeroplanes, trains and other mass transport vehicles). Outside of a vehicle environment, this may include the use of cameras which monitor the face of a subject using a computer, mobile device, or other hardware including a screen, thereby to collect the eyelid amplitude data whilst the subject uses that device (including either or both of unstructured usage and usage comprising performance of designated tasks; in some embodiments eyelid movement data is collected during videoconferencing as an additional functionality for a videoconferencing platform). Data collection hardware may include cameras and/or wearable devices (for example infrared oculography hardware, and VR/AR goggles and/or headsets). In some embodiments contextual data is collected, thereby to provide context to eyelid movement data. For example, in videoconferencing, contextual data may include whether a subject is speaking or listening. In the context of driving, this may include whether the vehicle is in motion (and optionally other factors such as driving speed, urban/freeway style of driving, and so on).
[0025] The time series data input representative of eyelid amplitude is collected from the subject during a “testing period”. This testing period is preferably an “unstimulated” period, meaning that there are no controlled stimuli delivered to the user as part of the testing protocol to trigger blinks, or otherwise instruct voluntary blinks (i.e. monitoring is performed without deliberate stimulation of blink events). That is, the monitoring is employed such that blink events detected for data corresponding to the testing are likely to be spontaneous involuntary blinks. It will be appreciated by those skilled in the art that characteristics of spontaneous involuntary blinks are capable of revealing a range of neurological conditions, given physiological mechanisms which control the relevant movements.
[0026] In some embodiments, OSAM data for a given subject is tracked over an extended period of time, thereby to monitor changes in metric calculations. This is optionally applied thereby to assess the effectiveness of sleep apnoea treatments to which the subject is subjected. These treatments may include medical and/or non-medical interventions. In some embodiments the OSAM includes a value defined on a scale representative of predicted severity of OSA, allowing a user to track their progress over time in terms of susceptibility/severity. [0027] Referring to FIG. 2, one embodiment provides a method for determining a metric representative of sleep apnoea risk factors for a human subject.
[0028] Block 201 represents a process including receiving time series data input representative of eyelid amplitude for the subject during a testing period. For example, this may be derived from image processing techniques such as those disclosed in PCT patent publication W02020082125, whereby image frames collected by a capture device are processed thereby to define eyelid amplitude data for each frame and from that generate time series data (optimally including processes such as interpolation thereby to generate a sampling rate greater than a camera frame rate). Alternately, hardware such as that used for infrared oculography may be employed.
[0029] Block 202 represents a process including processing the time series data thereby to detect a plurality of blink events. A process for identifying blink events may include applying algorithms to time series data thereby to identify specific markers which objectively delineate the start and end of a blink, for example based on changes and/or zero values for one or more multi-ordered derivative artefacts. In a simple case, this is used to identify a point where eyelid closure commences and a point where eyelid re-opening completes, and define a blink event as the time-series data between those points. Definitions for what constitutes start and end points for blink events may vary between embodiments. Blink events may be designated with sequential identifiers, thereby to facilitate analysis which is based on processing of data relating to consecutive blink events (e.g. inter-event artefacts).
[0030] Block 203 represents, for each blink event, determining a plurality of blink event artefacts, for example multi-ordered derivative artefacts and/or temporal artefacts (optionally in combination with inter-blink artefacts).
[0031] Block 204 represents a process including compiling one more artefact data sets, wherein each artefact data set includes blink artefacts (including blink artefacts derived from a first derivative of time-series amplitude). Each artefact data set corresponds to a defined plurality of blink events during the testing period. In the illustrated example, block 204 includes two distinct sub-processes: • Block 204(a): segregating blink events into delineated blocks. These may be temporally delineated blocks. For example, each delineated block corresponds to a X-second time period, wherein Xis a value between 30 and 240 (or more preferably between 45 and 120). A preferred embodiment makes use of 60-second blocks. Alternately, segregation may be based on other factors, for example segregation such that each block contains an equal (e.g. predefined) number of blink events, for example between about 10 and 30 blink events (although other block sizes may be used).
• Block 204(b): for each delineated block, performing statistical analysis of the blink artefacts thereby to derive block-specific blink artefact statistics. These are referred to as “block-based statistical artefacts”, being a second order collection of artefacts derived from multi-ordered derivative, temporal and/or inter-blink artefacts for blink events delimitated into a given block.
[0032] The process of deriving block-specific blink artefact statistics for each delineated block preferably includes applying a set of statistical analysis algorithms to the blink artefacts derived at block 203 for blink events corresponding to that block. In a preferred embodiment this includes Mean, Standard Deviation, Percentiles, and further Higher-Order Statistics.
[0033] Block 205 represents a process including processing each of the artefact data sets thereby to determine a metric representative to a risk of sleep apnoea. This preferably includes the following sub-processes:
• Block 205(a): providing each artefact data set to a classifier module thereby to derive an output value. This output value is in some embodiments a value between zero and one.
• Block 205(b): Determining the metric based on processing of a plurality of output values derived from the plurality of artefact data sets. For example, where values are outputted by the process of block 205(a), the metric may be based on output of the first layer. This may result in a risk metric defined on a scale (for example representative of low risk to high risk), defined in a binary manner (for example “not determined to be at risk” or “determined to be at risk”), and/or include information relating to predicted severity level (e.g. mild/moderate/severe sleep apnoea). Majority decisions, heuristics and/or other techniques may be used to determine the metric at this stage.
[0034] The classifier module preferably utilises a machine learning model (or other form of technology, such as Al technology, which enables classification). Various machine learning models may optionally be trained using various features engineered and described above. An instance of such models includes a layered architecture of a Random Forest (RF) model stacked on top of multiple Decision Tree (DT) models. Other model instances can include, but are not limited to, more black box models such as deep neural network (DNN) models like Deep Convolutional Neural Networks (DCNN), Recurrent Neural Networks (RNN) and Long-Short-Term Memory (LSTM) networks. Those skilled in the art will understand how such machine learning models may be trained based on features and training data sets. The training data sets may be labelled based on a binary “sleep apnoea sufferer” or “no sleep apnoea” labelling protocol, or via a more complex protocol which separates “sleep apnoea sufferer” into condition levels such as mild, moderate and severe.
[0035] The manner by which the final risk metric is communicated to the subject varies between embodiments based upon the overarching technological framework. For example, decision tree and/or other algorithmic logic may be configured to convert a numerical classifier output into one or more descriptive labels belonging to a predefined schema.
Example Hardware Frameworks
[0036] FIG. 1A illustrates an example in-vehicle monitoring system configured to enable prediction of sleep apnoea risk factors as described herein. It will be appreciated that this is an example only, and that a wide range of known commercial systems are configured to perform eyelid position tracking may be modified for the purposes of assessing sleep apnoea risk factors as discussed herein. For example, such known systems already record time series data for eyelid amplitude, and in some cases extract a range of artefacts from such data. Known systems such as these are optionally provided with additional software functionality (locally and/or in cloud-hosted systems) thereby to facilitate processing leading to generation of sleep apnoea relevant metric such as an OSAM based on data already being collected from a driver.
[0037] The system of FIG. 1A includes an image capture device 120. This may include substantially any form of appropriately sized digital camera, preferably a digital camera with a frame rate of over 30 frames per second. Higher frame rate cameras are advantageous, given that with enhanced frame rate comes an ability to obtain higher resolution data for eyelid movement. For example, other embodiments use cameras with frame rates of between 60 and 240 frames per second.
[0038] Device 120 is positioned to capture a facial region of a subject. Device 120 is in one embodiment installed in a region of a vehicle in the form of an automobile, for example on or adjacent the dashboard, windscreen, or visor, such that it is configured to capture a facial region of a driver. In another embodiment device 120 is positioned on or adjacent the dashboard, windscreen, or visor, such that it is configured to capture a facial region of a front seat passenger. In another embodiment device 120 is positioned in a region such as the rear of a seat such that it is configured to capture a facial region of a back-seat passenger. In some embodiments a combination of these is used, thereby to enable eyelid movement monitoring for both a driver and one or more passengers.
[0039] Although the system of FIG. 1A (and other systems) are described by reference to a vehicle in the form of an automobile, it will be appreciated that a system as described is also optionally implanted in other forms of vehicles, including mass-transport vehicles such as passenger airplanes, buses/coaches, and trains. In such embodiments there are preferably one or more analysis systems each supporting a plurality of image capture devices, each positioned to capture a respective passenger. Furthermore, the technology disclosed herein may also be implemented away from vehicles, for example to allow for OSA assessments to be performed in pharmacies, laboratories and at home (for example using computing devices such as PCs, laptops, smartphones and tablets with a screen that provides a “task” for a subject, and a camera to record eyelid movement whilst the task is performed).
[0040] An in-vehicle image processing system 110 is configured to receive image data from image capture device 120 (or multiple devices 120), and process that data thereby to generate time-series eyelid amplitude data. A control module 111 is configured to control device 120, operation of image data processing, and management of generated data. This includes controlling operation of image data processing algorithms, which are configured to:
(i) Identify that a human face is detected.
(ii) In embodiments where subject identification is achieved via facial recognition algorithms (which is not present in some embodiments, for example embodiments that identify a subject via alternate means), perform a facial recognition process thereby to identify the subject. This may include identifying a known subject based on an existing subject record defined in user identification data 151 stored in a memory system 150, or identifying an unknown subject and creating a new subject user identification data 151 stored in a memory system 150.
(iii) In a detected human face, identify an eye region. In some embodiments the algorithms are configured to track one eye region only; in other embodiments both eye regions are tracked thereby to improve data collection.
(iv) Identify, in the eye region(s), presence and movement of an eyelid. For example, in a preferred embodiment this is achieved by way of recording an eyelid position relative to a defined “closed” position against time. This allows generation of timeseries eyelid amplitude data in the form of eyelid position (amplitude) over time. It will be appreciated that such data provides for identification of events (for example blink events) and velocity (for example as a first derivative of position against time).
[0041] Algorithms 112 optionally operate to extract artefacts from the time-series eyelid amplitude data, for example temporal artefacts, multi-ordered derivative artefacts and in some cases additionally inter-blink artefacts. It will be appreciated, however, that extraction of such artefacts may occur in downstream processing.
[0042] A data management module 113 is configured to coordinate storage of data generated by algorithms 112 in user time-series eyelid amplitude data 152. In some embodiments the function of module 113 includes determining whether a set of generated time-series eyelid amplitude data meets threshold data quality requirements for storage, for example based on factors including a threshold unbroken time period for which eyelid tracking is achieved and time-series eyelid amplitude data is generated.
[0043] As an optional feature, memory system 150 includes user identification data 151 for one or more users. As noted, in some embodiments system 101 is configured to collect and analyse time-series eyelid amplitude data for only a single user (for instance the primary driver of a vehicle) and includes identification data to enable identification of only that user. In other embodiments, system 101 includes functionality to collect and analyse time-series eyelid amplitude data for multiple users, and includes identification data to enable identification of any of those users (and optionally, as noted above, defining of a new record for a previously unknown user). The identification data may include login credentials (for example a user ID and/or password) which are inputted via an input device. Alternately, the identification data may be biometric, for example using facial recognition as discussed above or an alternate biometric input (such as a fingerprint scanner). In some embodiments this leverages an existing biometric identification system of a device/vehicle/other.
[0044] Blink artefact data 152 includes is configured to time-series eyelid amplitude data for one or more testing periods, and optionally/additionally store associated blink artefact data and/or delineated block-specific blink artefact data sets as described herein (although noting that various processing steps may instead be performed externally of system 101).
[0045] Analysis modules 130 are configured to cause analysis of one or more artefact data sets, each artefact data set being derived from the blink artefacts, including blink artefacts derived from a first derivative of time-series amplitude, for a defined plurality of blink events during a given testing period. This may include analysis relating to driver alertness monitoring, and associated notifications and/or intervention management. In this embodiment, module 130 additionally causes processing each of a plurality of the artefact data sets (e.g. via a classifier, which may be cloud hosted) thereby to determine a metric representative to a risk of sleep apnoea, such as an OSAM.
[0046] System 101 additionally includes a communication system 160, which is configured to communicate information from system 101 to human users. This may include internal communication modules 161 which provide output data via components installed in the vehicle, for example an in-car display, warning lights, and so on. External communication modules 162 are also optionally present, for example to enable communication of data from system 101 to user devices (for example via Bluetooth, WiFi, or other network interfaces), optionally by email or other messaging protocols. In this regard, communication system 160 is optionally configured to facilitate communication results of analysis by analysis modules 130.
[0047] A control system 140 included logic modules 141 which control overall operation of system 140. This includes execution of logical rules thereby to determine communications to be provided in response to outputs from analysis modules 130. For example, this may include in-vehicle notifications. These in-vehicle notifications might be unrelated to sleep apnoea assessment, for example where blink event data is processed for more time-critical purposes such as assessing impairment/drowsiness. An OSAM or other apnoea related metric may be communicated via the vehicle, or that function may be responsibility of a separate cloud-based system. It will be appreciated that these are examples only, and logic modules 140 are able to provide a wide range of functionalities thereby to cause system 101 to act in a predefined manner.
[0048] FIG. 1 B illustrates a further embodiment in the form of system 10T, which includes various common features with the embodiment illustrated in FIG. 1A. In general terms, in some embodiments, external communication modules 162 facilitate communication with a remote server device in the form of a data analysis system 180, which performs analysis of time-series eyelid amplitude data analysis. This communication may rely on various intermediary devices to facilitate network connectivity (e.g. including a smartphone).
[0049] System 180 includes a control system 182 and logic modules 181 which are provided by computer executable code executing across one or more computing devices thereby to control and deliver functionalities of system 180.
[0050] System 180 additionally includes a memory system 183, which stores user testing data including and/or derived from time series eyelid amplitude data collected via system 10T, enabling such data to be passed to classifier modules for the purposes of generating an OSAM for each testing period. In some cases, memory system 183 optionally includes user identification data 184 thereby to facilitate tracking of changes in user-specific OASM metrics over time, for example to assess effectiveness of medical and/or non-medical treatment regimens.
[0051] System 180 additionally includes analysis modules 186. In this example, analysis modules 186 perform analysis via classifier modules, including a classifier module which is configured to facilitate generation of an OSAM as described herein. The OSAM is optionally communicated to the relevant subject via a range of technologies, including email and other electronic messaging platforms. Analysis modules 130 perform in-vehicle algorithmic analysis, for example in relation to driver impairment/alertness/drowsiness. In this manner, data collected for the purposes of in-vehicle analysis as part of a driver monitoring system is complemented by cloud-based processing which delivers metrics relevant to sleep apnoea.
Example Blink Artefacts
[0052] Provided below are examples of artefacts belonging to various artefact categories which are optionally used in any of the preceding embodiments.
[0053] Example multi-ordered derivative artefacts include:
• Relative amplitude.
• Relative position.
• Maximum amplitude.
• Minimum amplitude.
• Blink start position.
• Blink end position.
Maximum velocities (opening and/or closing). • Zero crossing index (positive and/or negative).
• Amplitude-to-velocity ratios (positive and/or negative). ‘
• Composite artefacts generated from a linear or nonlinear combination of two or more the above and/or other artefacts.
[0054] Example temporal artefacts include:
• Blink total duration (BTD), which is preferably measured as a time between commencement of closure movement which exceeds a defined threshold and completion of subsequent opening movement.
• Inter-event durations (positive and/or negative), which may be used to measure temporal artefacts for a period during which the eyelid is moving in a closing direction and/or opening direction.
• Blink eyelid closed duration (BECD).
[0055] Example inter-blink artefacts include:
• Blink frequency.
• Blink rate.
• Inter-blink event duration.
• Blink free interval (BFI).
• Saccade rate.
• Cumulative Time No Movement.
[0056] Example block-based statistical artefacts include: • Mean of any one or more blink artefact values.
• Standard deviation for any one or more blink artefact values.
• Percentile measures for any one or more blink artefact values.
• Third-order moment statistic, i.e. skewness.
• Fourth-order moment statistic, i.e. kurtosis.
[0057] These are examples only, and others may be used.
Conclusions and Interpretation
[0058] It will be appreciated that the above disclosure provides technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of timeseries eyelid amplitude data collected during conscious state. This offers distinct advantages over existing technologies, in the sense that testing is far less obstructive/inconvenient, being performed during a wakeful state, optionally whilst using computing device, watching a screen, and/or operating a vehicle. Furthermore, the technology allows for convenient monitoring of sleep apnoea risk factors over time, thereby to enable assessment of effectiveness of treatments that are applied.
[0059] It should be appreciated that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, FIG., or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention. [0060] Furthermore, while some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention, and form different embodiments, as would be understood by those skilled in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.
[0061] Furthermore, some of the embodiments are described herein as a method or combination of elements of a method that can be implemented by a processor of a computer system or by other means of carrying out the function. Thus, a processor with the necessary instructions for carrying out such a method or element of a method forms a means for carrying out the method or element of a method. Furthermore, an element described herein of an apparatus embodiment is an example of a means for carrying out the function performed by the element for the purpose of carrying out the invention.
[0062] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
[0063] Similarly, it is to be noticed that the term coupled, when used in the claims, should not be interpreted as being limited to direct connections only. The terms "coupled" and "connected," along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. Thus, the scope of the expression a device A coupled to a device B should not be limited to devices or systems wherein an output of device A is directly connected to an input of device B. It means that there exists a path between an output of A and an input of B which may be a path including other devices or means. "Coupled" may mean that two or more elements are either in direct physical or electrical contact, or that two or more elements are not in direct contact with each other but yet still co-operate or interact with each other.
[0064] Thus, while there has been described what are believed to be the preferred embodiments of the invention, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the invention, and it is intended to claim all such changes and modifications as falling within the scope of the invention. For example, any formulas given above are merely representative of procedures that may be used. Functionality may be added or deleted from the block diagrams and operations may be interchanged among functional blocks. Steps may be added or deleted to methods described within the scope of the present invention.

Claims

1. A method determining a metric representative of sleep apnoea risk factors for a human subject, the method including: receiving time series data input representative of eyelid amplitude for the subject during a testing period; processing the time series data thereby to detect a plurality of blink events; for each blink event, determining a plurality of blink event artefacts including blink artefacts derived from time-series amplitude data; compiling one more artefact data sets, wherein each artefact data set is derived from the blink artefacts, including blink artefacts derived from the time-series amplitude data for a defined plurality of blink events during the testing period; and processing each of the artefact data sets thereby to determine a metric representative to a risk of sleep apnoea.
2. A method according to claim 1 wherein the time series data representative of eyelid amplitude for the subject during a testing period is collected via a hardware arrangement which is configured to conduct data capture whilst the subject is engaging in a known task.
3. A method according to claim 2 wherein the known task is operation of a motorised vehicle.
4. A method according to claim 3 wherein the time series data input representative of eyelid amplitude for the subject during a testing period is collected via a driver or occupant monitoring system.
5. A method according to claim 4 wherein the driver monitoring system includes a camera configured to capture image frames of a facial region of the subject, and wherein the facial image frames are processed thereby to derive the time series data input representative of eyelid amplitude.
6. A method according to claim 2 wherein the known task is utilisation of a computer or mobile device.
7. A method according to any preceding claim wherein the time series data input representative of eyelid amplitude is collected using a camera system which is configured to collect facial image frames, wherein facial image frames collected via the camera system are processed thereby to derive the time series data input representative of eyelid amplitude.
8. A method according to any preceding claim wherein the time series data input representative of eyelid amplitude is collected using a wearable device.
9. A method according to any preceding claim wherein the wearable device includes infrared oculography data collection hardware.
10. A method according to any preceding claim wherein the step of compiling one more artefact data sets includes: (i) defining a group of blink artefacts for each of the detected blink events; (ii) segregating detected blink events into delineated blocks; and (iii) for each delineated block, performing statistical analysis of the blink artefacts thereby to derive block-specific blink artefact statistics.
11. A method according to claim 10 wherein each artefact data set includes the blockspecific blink artefact statistics for a respective one of the delineated blocks.
12. A method according to claim 10 or claim 11 wherein the delineated blocks are defined based on time periods.
13. A method according to claim 12 wherein each delineated block corresponds to a X-second time period, wherein X is a value between 30 and 240.
14. A method according to claim 12 wherein each delineated block corresponds to a X-second time period, wherein X is a value between 45 and 120.
15. A method according to claim 10 or claim 11 wherein the delineated blocks are defined based on a prescribed number of blink events, such that each delineated block corresponds to an equal number of blink events.
16. A method according to any preceding claim wherein processing each of the artefact data sets thereby to determine a metric representative to a risk of sleep apnoea includes, where there is a plurality of artefact data sets: (i) providing each artefact data set to a classifier module thereby to derive an output value; and (ii) determining the metric based on processing of a plurality of output values derived from the plurality of artefact data sets.
17. A method according to claim 16 wherein the classifier module includes a machine learning model.
18. A method according to any preceding claim wherein the metric representative of sleep apnoea risk factors for the human subject is a metric representative of obstructive sleep apnoea risk factors.
19. A method according to any preceding claim wherein the blink artefacts include multi-ordered derivative artefacts.
20. A method according to any preceding claim wherein the blink artefacts include at least one multi-ordered derivative artefact and at least one temporal artefact.
21. A method according to any preceding claim wherein the blink artefacts include a combination of two or more of the following: at least one multi-ordered derivative artefact, at least one temporal artefact; and at least one inter-blink artefact.
22. A method according to any preceding claim wherein the blink artefacts include one or more first-ordered derivative artefacts.
23. A processing system configured to perform a method according to any preceding claim.
24. A driver or occupant monitoring system configured to provide data representative of eyelid amplitude to a processing system according to claim 23.
25. A method for determining a sleep apnoea risk factors metric via passive monitoring of a conscious subject without deliberate stimulation of blink events.
26. A method for determining a sleep apnoea risk factors metric via monitoring of a conscious subject performing a known task without deliberate stimulation of blink events.
27. A method for determining a sleep apnoea risk factors via passive monitoring of a conscious subject operating a motor vehicle.
28. A method for determining a sleep apnoea risk factors via monitoring of a conscious subject performing a vigilance test delivered via a computing device screen.
29. A method determining a metric representative of sleep hygiene for a human subject, the method including: receiving time series data input representative of eyelid amplitude for the subject during a testing period; processing the time series data thereby to detect a plurality of blink events; for each blink event, determining a plurality of blink event artefacts including blink artefacts derived from time-series amplitude data; compiling one more artefact data sets, wherein each artefact data set is derived from the blink artefacts, including blink artefacts derived from the time-series amplitude data for a defined plurality of blink events during the testing period; and processing each of the artefact data sets thereby to determine a metric representative to sleep hygiene.
30. Subject matter as disclosed herein.
PCT/AU2024/050369 2023-04-18 2024-04-17 Technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of time-series eyelid amplitude data collected during conscious state Ceased WO2024216336A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP24791588.7A EP4698042A1 (en) 2023-04-18 2024-04-17 Technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of time-series eyelid amplitude data collected during conscious state
AU2024256955A AU2024256955A1 (en) 2023-04-18 2024-04-17 Technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of time-series eyelid amplitude data collected during conscious state

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2023901148A AU2023901148A0 (en) 2023-04-18 Technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of time-series eyelid amplitude data collected during conscious state
AU2023901148 2023-04-18

Publications (1)

Publication Number Publication Date
WO2024216336A1 true WO2024216336A1 (en) 2024-10-24

Family

ID=93151710

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2024/050369 Ceased WO2024216336A1 (en) 2023-04-18 2024-04-17 Technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of time-series eyelid amplitude data collected during conscious state

Country Status (3)

Country Link
EP (1) EP4698042A1 (en)
AU (1) AU2024256955A1 (en)
WO (1) WO2024216336A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003039358A1 (en) * 2001-11-08 2003-05-15 Sleep Diagnostics Pty Ltd Alertness monitor
WO2006092022A1 (en) * 2005-03-04 2006-09-08 Sleep Diagnostics Pty. Ltd Measuring alertness
WO2012061871A1 (en) * 2010-11-08 2012-05-18 Optalert Australia Pty Ltd Fitness for work test
WO2020082125A1 (en) * 2018-10-23 2020-04-30 Sdip Holdings Pty Ltd Devices and processing systems configured to enable physiological event prediction based on blepharometric data analysis
WO2022061403A1 (en) * 2020-09-22 2022-03-31 Sdip Holdings Pty Ltd Devices and processing systems configured to enable assessment of a condition of a human subject based on sensorimotor gating of blinks
WO2022256877A1 (en) * 2021-06-11 2022-12-15 Sdip Holdings Pty Ltd Prediction of human subject state via hybrid approach including ai classification and blepharometric analysis, including driver monitoring systems

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003039358A1 (en) * 2001-11-08 2003-05-15 Sleep Diagnostics Pty Ltd Alertness monitor
WO2006092022A1 (en) * 2005-03-04 2006-09-08 Sleep Diagnostics Pty. Ltd Measuring alertness
WO2012061871A1 (en) * 2010-11-08 2012-05-18 Optalert Australia Pty Ltd Fitness for work test
WO2020082125A1 (en) * 2018-10-23 2020-04-30 Sdip Holdings Pty Ltd Devices and processing systems configured to enable physiological event prediction based on blepharometric data analysis
WO2022061403A1 (en) * 2020-09-22 2022-03-31 Sdip Holdings Pty Ltd Devices and processing systems configured to enable assessment of a condition of a human subject based on sensorimotor gating of blinks
WO2022256877A1 (en) * 2021-06-11 2022-12-15 Sdip Holdings Pty Ltd Prediction of human subject state via hybrid approach including ai classification and blepharometric analysis, including driver monitoring systems

Also Published As

Publication number Publication date
AU2024256955A1 (en) 2025-10-16
EP4698042A1 (en) 2026-02-25

Similar Documents

Publication Publication Date Title
Rastgoo et al. Automatic driver stress level classification using multimodal deep learning
US20260053407A1 (en) Driving state monitoring and feedback method and system based on multimodal human-factors intelligent data analysis, and edge computing terminal device
Kang Various approaches for driver and driving behavior monitoring: A review
US20250061356A1 (en) Devices and processing systems configured to enable physiological event prediction based on blepharometric data analysis
Oliveira et al. Driver drowsiness detection: a comparison between intrusive and non-intrusive signal acquisition methods
CN107428245B (en) Device and method for predicting the level of alertness of a driver of a motor vehicle
Celona et al. A multi-task CNN framework for driver face monitoring
CN112220480A (en) Driver state detection system and vehicle based on millimeter wave radar and camera fusion
Wang et al. Real-time driving danger-level prediction
Dehzangi et al. Unobtrusive driver drowsiness prediction using driving behavior from vehicular sensors
Pandey et al. A survey on visual and non-visual features in Driver’s drowsiness detection
Flores-Monroy et al. Visual-based real time driver drowsiness detection system using CNN
US20220036101A1 (en) Methods, systems and computer program products for driver monitoring
Xie et al. Revolutionizing road safety: YOLOv8-powered driver fatigue detection
Utomo et al. Driver fatigue prediction using different sensor data with deep learning
Von Jan et al. Don’t sleep and drive–VW’s fatigue detection technology
Sharma et al. Drowsiness warning system using artificial intelligence
Khandare et al. A deep learning framework for non-intrusive driver drowsiness detection
Manjula et al. Driver inattention monitoring system based on the orientation of the face using convolutional neural network
Karakan Real-Time and Deep Learning-Based Fatigue Detection for Drivers
JP2020057198A (en) Sleepiness prediction device
AU2024256955A1 (en) Technology configured to enable determination of sleep apnoea risk factors for subject based on artefacts of time-series eyelid amplitude data collected during conscious state
US20240153285A1 (en) Prediction of human subject state via hybrid approach including ai classification and blepharometric analysis, including driver monitoring systems
Bajaj et al. Performance analysis of hybrid model to detect driver drowsiness at early stage
Murad et al. Computing driver tiredness and fatigue in automobile via eye tracking and body movements

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24791588

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: AU2024256955

Country of ref document: AU

ENP Entry into the national phase

Ref document number: 2024256955

Country of ref document: AU

Date of ref document: 20240417

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2024791588

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2024791588

Country of ref document: EP

Effective date: 20251118

ENP Entry into the national phase

Ref document number: 2024791588

Country of ref document: EP

Effective date: 20251118

ENP Entry into the national phase

Ref document number: 2024791588

Country of ref document: EP

Effective date: 20251118

ENP Entry into the national phase

Ref document number: 2024791588

Country of ref document: EP

Effective date: 20251118

ENP Entry into the national phase

Ref document number: 2024791588

Country of ref document: EP

Effective date: 20251118

ENP Entry into the national phase

Ref document number: 2024791588

Country of ref document: EP

Effective date: 20251118

ENP Entry into the national phase

Ref document number: 2024791588

Country of ref document: EP

Effective date: 20251118

ENP Entry into the national phase

Ref document number: 2024791588

Country of ref document: EP

Effective date: 20251118

ENP Entry into the national phase

Ref document number: 2024791588

Country of ref document: EP

Effective date: 20251118

WWP Wipo information: published in national office

Ref document number: 2024791588

Country of ref document: EP