Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a multi-module linkage control and multi-dimensional synchronous evaluation first-aid minimally invasive simulation training system, which solves the problems that the existing minimally invasive simulation training system cannot accurately trace back causality due to the fact that the flow is separated from clinical reality, the physiology is lacked, the real-time coupling response of operation is lacked, and the fragmentation of multi-dimensional evaluation data is caused by the isolated operation of each module.
The invention aims at achieving the purposes through the following technical scheme that the multi-module linkage control and multi-dimensional synchronous evaluation first-aid minimally invasive simulation training system comprises a laparoscope simulation training system, an anesthesia monitoring simulation system, an advanced comprehensive simulation person, a multi-module linkage control unit, a full-flow data acquisition unit and a multi-dimensional synchronous evaluation unit;
The multi-module linkage control unit is used as a central processing core, is connected with the laparoscope simulation training system and the anesthesia monitoring simulation system through wired interfaces, is connected with the advanced comprehensive simulation person through a wireless communication protocol, and is connected with the full-flow data acquisition unit through a local area network;
The multi-module linkage control unit is configured with a full-flow control model based on a finite state machine, the full-flow control model comprises a preoperative preparation state, an operation progress state and an emergency rescue state, in the preoperative preparation state, the multi-module linkage control unit executes logic verification according to physiological parameter data and operation logs of the anesthesia monitoring simulation system, only sends an unlocking instruction to the laparoscopic simulation training system when the verification passes, in the operation progress state, the multi-module linkage control unit calculates physiological influences in real time according to instrument operation parameters of the laparoscopic simulation training system and drives the advanced comprehensive simulation person to generate physical sign change, in the emergency rescue state, the multi-module linkage control unit concurrently sends an emergency mode instruction packet to each subsystem, and pauses operation response of the laparoscopic simulation training system and activates cardiopulmonary resuscitation assessment mode of the advanced comprehensive simulation person;
The full-flow data acquisition unit is configured with a unified clock source and is used for marking time stamps of unified time references for acquired video data, instrument sensor data and physiological waveform data, and the multi-dimensional synchronous evaluation unit establishes a mapping relation between video frames and data streams according to the time stamps.
By adopting the technical scheme, as the multi-module linkage control unit is adopted as a central core, the full-flow logic control is carried out on preoperative, intraoperative and emergency states by combining a finite state machine model, and the synchronous acquisition and mapping of multi-source heterogeneous data are realized through a unified clock source, the situation that surgical operation and anesthesia monitoring are isolated in traditional simulation training is broken through, the real-time closed-loop coupling of physiological parameter change and operation of surgical instruments is realized, a training team is forced to strictly adhere to clinical operation specifications, accurate time and data reference are provided for a subsequent multi-dimensional complex disc, and the cooperation capability and crisis processing level of the emergency minimally invasive surgery team are effectively improved.
The laparoscopic simulation training system comprises a multi-screen display module and an instrument operation platform, wherein the instrument operation platform is provided with a puncture outfit interface, a magnetic force feedback system is integrated in the puncture outfit interface and comprises a damping motor, the damping motor is configured to generate reverse torque according to collision detection results in a virtual scene, and when the contact depth of the tip of the virtual instrument and the surface of the virtual organ is greater than zero, feedback force output by the damping motor is formed by superposition of the product of the elastic coefficient of the virtual organ and the contact depth and the product of the damping coefficient and the movement speed of the instrument, and the feedback force is transmitted to the hands of an operator.
By adopting the technical scheme, the virtual tissue contact depth and the motion speed are converted into the physical damping torque by using a force feedback calculation mechanism based on a physical model, so that the real tissue touch feeling and the instrument operation resistance are restored in the virtual simulation environment, and the immersion feeling of training and the hand-eye coordination training effect are enhanced.
The anesthesia monitoring simulation system comprises a gas supply module, an anesthetic supply module and a respiratory parameter adjusting module, wherein the gas supply module is provided with a physical knob and a rotary encoder for converting a physical rotation angle into a digital signal and transmitting the digital signal to the multi-module linkage control unit, and a pharmacokinetic operation unit is arranged in the anesthesia monitoring simulation system and used for calculating an anesthesia depth value and a blood concentration change curve according to the medicine dosage and the body weight. The advanced comprehensive simulation human chest is internally provided with a pneumatic bellows structure for generating chest relief synchronous with a set value of the anesthesia monitoring simulation system, and the arterial position of the advanced comprehensive simulation human is pre-embedded with a linear actuator for simulating pulse beating through electromagnetic pulse, wherein the pulse strength is in direct proportion to a systolic pressure value.
By adopting the technical scheme, the real-time linkage of anesthesia administration, breathing machine parameter setting and simulation of human physical signs is realized, so that the pharmacodynamics calculation result can be intuitively fed back to a trainer through a physical carrier, and a physiological monitoring environment combining deficiency and excess is constructed.
Preferably, the logic verification executed by the multi-module linkage control unit in the preoperative preparation state comprises a parameter verification algorithm, wherein the passing condition set by the parameter verification algorithm comprises that the partial pressure of end-tidal carbon dioxide uploaded by the anesthesia monitoring simulation system is in a preset stable interval, the blood oxygen saturation is greater than a preset oxygenation index, the four serial stimulation ratios monitored by muscle relaxation are smaller than the preset muscle relaxation index, and an instrument counting confirmation signal is received, and the multi-module linkage control unit releases the locking mode of a puncture outfit interface damping motor of the laparoscope simulation training system only when the conditions are simultaneously met.
By adopting the technical scheme, a forced preoperative safety interlocking mechanism is established, so that the operation can be started only under the conditions of sufficient anesthesia induction and error-free instrument preparation, the safety check consciousness is strengthened in the training link, and the clinical operation flow is standardized.
Preferably, the multi-module linkage control unit operates an intraoperative physiological and instrument coupling response mechanism in the operation progress state, the mechanism is used for monitoring output power and duration of an electrosurgical generator simulator, judging thermal damage and calculating a bleeding rate when the monitored output power is larger than a safe power threshold and the duration exceeds a safe tolerance time, the bleeding rate is in direct proportion to a difference value of the output power exceeding the safe power threshold and a difference value of the duration exceeding the safe tolerance time, the multi-module linkage control unit integrates the bleeding rate to obtain accumulated blood loss, calculates variation of average arterial pressure and heart rate according to the accumulated blood loss, and transmits a calculation result to the advanced comprehensive simulator to adjust the strength of the pulse actuator.
By adopting the technical scheme, the causal relationship between misoperation and pathophysiological consequences is quantized by utilizing a mathematical model, so that the operation error can trigger the deterioration of vital signs in real time, a trainer is forced to consider tissue protection and whole body physiological state in operation, and high-fidelity complication simulation is realized.
The multi-dimensional synchronous evaluation unit is configured to read the log file generated by the full-flow data acquisition unit, calculate the similarity between the instrument track of the trainer and the expert standard track by using a dynamic time warping algorithm, output the normalized minimum accumulated distance as an operation score by constructing a distance matrix and searching a warping path with the minimum accumulated distance, and calculate the cooperative response delay by using a time sequence association analysis model, namely, the time difference from the moment of triggering an event by the multi-module linkage control unit to the moment of detecting the occurrence of the corresponding response event, and generate an evaluation report containing skill ability images and based on causal logic analysis according to the similarity.
By adopting the technical scheme, the dynamic time warping algorithm is introduced to solve the problem of track similarity evaluation under different operation speeds, and quantitative assessment of team cooperation response speed is realized by utilizing time sequence association analysis, so that a two-dimensional objective assessment report containing technical skills and non-technical skills can be generated, and accurate data support is provided for teaching compound plates.
Preferably, the multi-module linkage control unit is further integrated with a surgical scene editor module, wherein the surgical scene editor module is used for generating a scene configuration file containing virtual anatomic structure definition, physiological parameter reference setting and complication triggering logic rules, and the surgical scene editor module provides an interactive interface to set space coordinate transformation parameters and physical attribute parameters of the anatomic model, allows a condition triggering rule tree to be constructed, and defines triggering conditions and associated result events.
By adopting the technical scheme, the system has the capability of user-defined construction of the operation cases and emergency logic, can flexibly adjust the training difficulty and the case types according to teaching requirements, and improves the expansibility and the application range of the system.
The invention provides a multi-module linkage control and multi-dimensional synchronous evaluation first-aid minimally invasive simulation training system. The beneficial effects are as follows:
1. The invention adopts a full-flow control model based on a finite state machine through a multi-module linkage control unit, performs safety interlocking check based on anesthetic physiological parameters and instrument counting under the preoperative preparation state, releases the operation restriction of the laparoscope only after the check is passed, and synchronously pauses operation response and activates a cardiopulmonary resuscitation mode when an emergency state is triggered in the operation. The method simulates the strict flow dependence of anesthesia induction and operation in real clinical operation, solves the defect of isolated functions of each module in the traditional training system, forces a training team to establish strict flow specification consciousness and multi-role cooperative thinking in a simulation environment, and shortens the adaptation period from simulation training to clinical practice operation.
2. The invention utilizes the whole-flow data acquisition unit to configure a unified clock source, marks millisecond-level time stamps of a unified time reference for surgical video stream, instrument sensor data and physiological waveform data, and establishes an accurate mapping relation between video frames and data streams by combining a multi-dimensional synchronous evaluation unit. The whole-course traceability analysis of misoperation and physiological consequences is realized, the specific moment of simulated bleeding caused by overhigh electric coagulation power and the abnormal fluctuation of corresponding physiological parameters can be accurately positioned, the problems of fragmentation of data and difficulty in positioning causality in the existing assessment means are solved, and therefore, the accuracy of skill correction and the teaching repeated disc efficiency are improved.
3. The integrated operation scene editor module supports a user to define virtual anatomy structure definition, physiological parameter reference and complication triggering logic rules through an interactive interface, and generates a standardized scene configuration file for loading by a system. The system can be adapted to the training requirements of various specialized operations such as laparoscopic cholecystectomy and appendectomy, and the linkage rules and evaluation indexes can be flexibly adjusted according to specific operation types, so that the application range of the system is effectively expanded, and the diversified teaching training requirements of different sub-specialized operations of minimally invasive surgery are met.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, the invention provides a multi-module linkage control and multi-dimensional synchronous evaluation first-aid minimally invasive simulation training system, which comprises a laparoscope simulation training system, an anesthesia monitoring simulation system, an advanced comprehensive simulation simulator, a multi-module linkage control unit, a full-flow data acquisition unit and a multi-dimensional synchronous evaluation unit. All the units are connected through a communication topology network combining wired and wireless to form a hardware closed-loop control system.
The multi-module linkage control unit is used as a central processing core of the system, adopts a Miao IPC-610 industrial control computer (provided with IntelCorei processors) and is internally provided with a VxWorks7 hard real-time operating system. The control unit is respectively connected with a main screen of the laparoscope simulation training system and a control end of the anesthesia monitoring simulation system in a wired manner through a USB3.0 interface, and the data transmission rate is not lower than 5Gbps and is used for transmitting high-bandwidth video stream data and real-time 3D rendering instructions. The multi-module linkage control unit establishes wireless communication connection with the control machine of the advanced comprehensive simulation man through the Bluetooth 5.0 protocol at the same time, the communication distance coverage radius is not less than 10 meters, and continuous instruction issuing and physiological parameter returning under the low power consumption state are supported. The system adopts a local area network architecture based on TCP/IP protocol, distributes instructions through MQTT3.1.1 message queue mechanism, encrypts transmission data by adopting TLS/SSL protocol, and ensures that control instruction delay among all modules is less than 10 milliseconds and data transmission is safe.
The laparoscopic simulation training system is composed of a multi-screen display module and an instrument operation platform on hardware. The multi-screen display module comprises 5 displays, wherein 1 display is a main control touch screen with the resolution of 1920 multiplied by 1080 and used for displaying a case selection interface, a system setting interface and an evaluation report interface, 2 display is a secondary screen and used for rendering a 3D anatomical structure of an operation area and a motion track of an instrument in a virtual space in real time, 1 display is an anesthesia parameter display screen and used for simulating an operation panel of an anesthesia machine, and 1 display is a vital sign monitoring screen and used for displaying real-time waveform data of a simulated person. The display is fixed on the operation trolley through the multi-joint cantilever bracket, and supports horizontal rotation for 360 degrees and vertical lifting adjustment so as to adapt to trainers with different heights.
The instrument operation platform, i.e. the laparoscopic stent medical equipment, comprises a pneumoperitoneum machine simulator, an aspiration and irrigation system simulator and an electrosurgical generator simulator. The platform is provided with a standard Tromar port (puncture outfit interface), a high-precision magnetic force feedback system is integrated in the platform, and a damping motor with a torque output range of 0 to 5 N.m is arranged. When a trainer operates the physical laparoscopic instrument insertion port, an internal damping motor generates reverse torque according to a collision detection result in a virtual scene. The contact depth between the tip of the virtual instrument and the surface of the virtual organ is set asThe elastic coefficient of the virtual organ isThe movement speed of the instrument isDamping coefficient isFeedback force of motor outputFollowing the following formula
;
Wherein, the Indicating that the instrument has contacted and pressed against the virtual tissue. The feedback force is transmitted to the hands of the trainer through the mechanical transmission structure, and the actual tissue touch feeling is simulated.
The anesthesia monitoring simulation system comprises a gas supply module, an anesthetic supply module and a respiratory parameter adjusting module in hardware. The gas supply module is provided with a physical knob for adjusting the flow of oxygen (O 2), AIR (AIR) and laughing gas (N 2 O), and a rotary encoder is connected to the rear of the knob for converting the physical rotation angle into a digital signal and transmitting the digital signal to the control unit. The anesthetic supply module simulates an evaporator structure and detects the remaining amount of the virtual liquid medicine through the liquid level sensor. The respiratory parameter adjusting module comprises a touch interface and is used for setting Tidal Volume (TV), respiratory frequency (RR) and respiratory ratio (I: E). A pharmacokinetic operation unit is arranged in the system, and anesthesia depth (MAC) value and blood concentration change curve are calculated according to the input medicine dosage and the weight of a patient.
The advanced comprehensive simulators are physical carriers of the system, the height of the simulators is set to be 180cm, the weight of the simulators is about 60kg, the outer skin is made of medical grade silica gel, and has touch feeling and elasticity close to those of a real human body. The simulated human head is provided with an eyelid which can be opened and closed and a pupil which reflects light, and the inside of the simulated human head is integrated with a miniature steering engine to drive the diameter change of the pupil. The pneumatic bellows structure is arranged in the chest cavity, the ventilation volume adjusting range is 0-1000 mL, and the pneumatic bellows structure is driven by the mute air compressor, so that the chest cavity fluctuation generated by spontaneous breathing can be simulated, and the fluctuation frequency is synchronous with the set value of the breathing parameter adjusting module in real time. Linear actuators are embedded in the carotid artery, femoral artery and radial artery positions of the simulated human body, the response time is less than 50ms, pulse beating is simulated through electromagnetic pulse, and the pulse intensity is in direct proportion to the systolic pressure value.
The analog human is integrated with a wireless monitor module, which comprises an Electrocardiograph (ECG) generating circuit, an oxygen saturation (SpO 2) analog probe interface and an Invasive Blood Pressure (IBP) analog channel. The wireless monitor sends the generated physiological electric signal data packet to the multi-module linkage control unit through a wireless network, wherein the data packet contains more than 18 physiological parameters such as heart rate, blood pressure, blood oxygen, intracranial pressure (ICP), end-tidal carbon dioxide (ETCO 2) and the like. The simulator is powered by the built-in high-capacity lithium battery pack, and supports continuous operation time not less than 6 hours in a wireless connection state.
The whole-flow data acquisition unit is responsible for capturing multi-source heterogeneous data in the system. The unit comprises an evaluation shooting set arranged in a laparoscope operation area, and consists of two groups of high-definition cameras, wherein the resolution is not lower than 1080P, the frame rate is not lower than 30fps, and the evaluation shooting set is respectively used for shooting a simulated visual field image in the laparoscope and external operation actions of hands of a trainer.
The whole-flow data acquisition unit also comprises a nine-axis inertial measurement unit (IMU, for example, MPU-9250 chip) which is arranged in the laparoscopic instrument and comprises a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer. The sensor acquires motion data of the instrument in space in real time at a sampling frequency of 1000Hz, wherein the motion data comprise displacement coordinates of an X axis, a Y axis and a Z axis and Euler angles (pitch angle, roll angle and yaw angle). The electrosurgical generator simulator is internally connected with a current sensor in series and is used for monitoring the electrocoagulation output power in real time, the measuring range is 0-150W, and the monitoring precision is +/-1W.
In order to achieve accurate correlation of multi-module data, the full-flow data acquisition unit is configured with a unified clock source. When the system is started, the main control computer sends a time setting instruction to the camera shooting suit, the instrument sensor and the analog person monitoring module, and a unified system time reference is established. All acquired data frames are time stamped on the order of milliseconds. For video data, metadata of each frame image is written inFor discrete sensor data, the data packet header contains. Data synchronization biasThe control is within 10 milliseconds, namely, the following conditions are satisfied:
;
wherein, the As a time stamp for a certain video frame,A time stamp for sensor data collected at the same time. This strict time synchronization mechanism provides a data basis for subsequent multidimensional synchronization assessment.
Referring to fig. 2, the emergency minimally invasive simulation training system provided by the invention executes a full-flow control model based on a finite state machine through a multi-module linkage control unit, and the control model divides the operation process of the system into a standby state, a preoperative preparation state, an operative operation state, an emergency rescue state and a postoperative resuscitation state. The multi-module linkage control unit circularly scans the state registers of the laparoscope simulation training system, the anesthesia monitoring simulation system and the advanced comprehensive simulation personnel at a preset frequency (for example, 100 Hz), and judges the state jump condition according to a preset Boolean logic. When the system initialization is completed and the handshake of the communication links of the modules is successful, the system state automatically jumps to the preoperative preparation state.
In the preoperative preparation state, the multi-module linkage control unit executes a preoperative safety interlocking mechanism, and the operation authority of the laparoscopic simulation training system is controlled in a locking mode through software logic. The control unit reads physiological parameter data and operation logs uploaded by the anesthesia monitoring simulation system in real time, and executes a parameter verification algorithm. The algorithm sets logic conditions which must be met simultaneously, wherein the partial pressure of carbon dioxide at the end of expiration of an airway monitoring index needs to be stabilized between 35mmHg and 45mmHg, the blood oxygen saturation of an oxygenation index needs to be more than 95%, the ratio of four serial stimuli of a muscle relaxation monitoring index needs to be less than 10%, the anesthesia depth monitoring index (MAC value) needs to be 1.0-1.2, and an instrument counting confirmation signal from a nurse role end is received. And if and only if the logical AND operation results of all the conditions are true, the control unit sends an unlocking instruction to the laparoscopic support medical equipment, releases the locking mode of the Tromar port damping motor and allows the main display screen to enter the operation interface of the surgical case. If any one of the conditions is not met, the main display screen continuously displays the prompt that anesthesia does not reach the standard, and the operation of the physical handle does not generate effective displacement in the virtual visual field.
When the system state jumps to the operation progress state, the multi-module linkage control unit activates an intraoperative physiological and instrument coupling response mechanism. The mechanism calculates the influence of the instrument operation parameters on the physiological state of the virtual patient in real time through a mathematical model, and particularly carries out quantitative calculation on pathological physiological changes caused by misoperation. Taking an electrocoagulation hemostasis operation as an example, the system monitors the output power of an electrosurgical generator simulator in real time(Units: watts) and duration. The system presets the safe power threshold of the current organization as followsAnd safe tolerance time. When the output power is detectedGreater than a safe power thresholdAnd the duration of operation exceeds the safety tolerance timeAt this time, the system decides that thermal damage is occurring and initiates active bleeding.
The system calculates the bleeding rate according to the following formula(Unit: ml/min):
;
wherein, the For the tissue vulnerability coefficients, the coefficients are determined by the virtual anatomical properties involved in the current operation. System versus bleeding rateTime integration is carried out to obtain the accumulated blood loss. The multi-module linkage control unit further utilizes a hemodynamic model to control the blood flow according to the accumulated blood lossDriving the change of vital signs of the simulated person. Mean arterial pressureAnd heart rateThe variation of (c) follows the following formula:
;
;
wherein, the AndBase blood pressure and base heart rate, respectively; preset total blood volume for the simulated person; is the blood pressure drop coefficient; And compensating the coefficient for heart rate. The calculation result is transmitted to an advanced comprehensive simulation man in real time, a pulse actuator of the advanced comprehensive simulation man is driven to adjust the pulse intensity, waveform display on a monitor screen is synchronously updated, and closed-loop feedback of physiological parameter change caused by misoperation is realized.
When the calculated physiological parameters deteriorate to a preset critical value, for example, the blood oxygen saturation is lower than 90%, the heart rate is lower than 50 times/min or higher than 150 times/min, or the contraction pressure is lower than 80mmHg, the multi-module linkage control unit automatically triggers multi-terminal cooperative control logic under the emergency condition, and the system state is forcedly jumped to an emergency rescue state. At this time, the control unit concurrently transmits an emergency mode instruction packet to each subsystem. The main screen of the laparoscope simulation training system immediately pops up a red alarm window and pauses operation response to prompt an operator to immediately treat pneumoperitoneum and hemorrhage, the anesthesia monitoring simulation system automatically switches display layout, amplifies and displays electrocardiograph, blood pressure and blood oxygen waveforms, gives out audible and visual alarm, and the advanced comprehensive simulation human control machine activates a chest compression detection sensor to enter a cardiopulmonary resuscitation assessment mode to start recording the depth, frequency and rebound rate of chest compression. Simultaneously, a timer is started at the background of the system, and the time interval from alarm triggering to the first effective emergency measure implementation of the training team is recorded. Only if the physiological parameter rises back within the safe range and continues for more than a preset time (e.g., 30 seconds) the system allows manual release of the emergency mode, restoring the surgical access rights.
Referring to fig. 3, the first-aid minimally invasive simulated training system provided by the invention constructs a unified clock reference and multi-source heterogeneous data fusion mechanism through the full-flow data acquisition unit. The whole flow data acquisition unit is internally integrated with a high-precision clock source, and is used as a local master clock server (MasterClockServer) of the system, and a master control computer of the laparoscope simulation training system, a control end of the anesthesia monitoring simulation system and a control machine of the advanced comprehensive simulation simulator are all used as slave clock nodes (SlaveClockClient). The modules are connected through a local area network based on TCP/IP protocol, and an NTPv4 (NetworkTimeProtocolversion 4) daemon is operated. In the system initialization stage, the whole-flow data acquisition unit broadcasts a time synchronization instruction to the local area network, each slave clock node calculates network transmission delay and clock drift amount, corrects the local system time of each slave clock node, and ensures that the absolute value of the system time deviation of all nodes in the whole local area network is less than 10 milliseconds.
After the unified clock reference is established, the full-flow data acquisition unit performs a differentiated time stamping strategy on different types of input data. For evaluating operation video data acquired by the shooting suit, the system adopts a mode of embedding a time code into a video stream, and writes the synchronized absolute time into a metadata track of a video container or a display time stamp (PTS) field of each frame of image. For the discrete operation data (such as shearing, clamping, electrocoagulation on/off) and continuous track data (X/Y/Z axis coordinates) generated by the laparoscopic instrument simulator and the continuous physiological waveform data (such as Electrocardiogram (ECG), photoplethysmogram (photo-volume pulse wave) PLETH) generated by the advanced comprehensive simulator, the data acquisition unit marks a millisecond level Unix timestamp with 64-bit precision for each data packet according to the corrected master clock at the moment of receiving the data packet.
In order to realize the structured storage and associated retrieval of multi-source heterogeneous data, a full-flow data acquisition unit is packaged by adopting a standardized data structure based on a JavaScript object notation (JSON). The system creates a root object that contains session metadata, a track stream, an event stream, and a physiological stream. Wherein unstructured surgical video data is stored in the metadata field in the form of a file path index. Structured data, such as instrument trajectory and physiological parameters, are serialized into array objects containing timestamp key value pairs. For example, a record of one electrocoagulation operation is packaged as an object containing the trigger time, duration, power setting and associated tissue location. The full-flow data acquisition unit writes the data streams uploaded by all the slave modules into the same JSON log file according to the time sequence, and adopts an AES-256 encryption algorithm to carry out local storage encryption on the log file, so that sensitive medical training data is prevented from being leaked, and a safe global database taking a time axis as a unique index is established.
In the stage of evaluating the multi-disc, the multi-dimensional synchronous evaluation unit utilizes the time synchronization mechanism to realize the accurate mapping of the video frames and the data streams. The system calculates any time according to the following mapping formulaCorresponding video frame index number:
;
Wherein, the Absolute time stamp for the start of surgical video recording; An inherent phase delay constant for the video encoding and sensor sampling measured by the system; Is the frame rate of the video stream; Representing a rounding down operation. Through the calculation mechanism, when a user clicks a certain fault event (such as a bleeding point) on a time axis of an evaluation report, the player can directly locate a video frame corresponding to the occurrence time of the event, simultaneously highlight the blood pressure and heart rate values at the time on a oscillogram, and reproduce the spatial position of the instrument at the time in a 3D view, so that synchronous playback and attribution analysis of multi-dimensional data are realized.
Referring to fig. 4, the emergency minimally invasive simulation training system provided by the invention has a multi-dimensional synchronous evaluation unit capable of executing an automatic evaluation method. The method comprises the steps of firstly reading a JSON format log file containing uniform time stamps and generated by a full-flow data acquisition unit, and deconstructing the data into feature vectors of three dimensions of operation skills, physiological maintenance and team cooperation. The operation skill feature vector comprises a motion track coordinate sequence, speed, acceleration, jerk (Jerk) of the instrument tip and time integral of electric coagulation power output, the physiological maintenance feature vector comprises variances of simulated human vital sign parameters (heart rate, blood pressure and blood oxygen saturation) relative to a reference value during a critical operation step, and the team cooperation feature vector comprises a time difference sequence of emergency command issuing time extracted from the multi-module linkage control unit and operation response time of a corresponding execution module.
The multidimensional synchronous evaluation unit calculates the similarity between the instrument track of the trainer and the pre-stored expert standard track by using a Dynamic Time Warping (DTW) algorithm. Because of the difference in the speed of different trainers completing the same surgical procedure, the length of the track sequence is inconsistent, and the linear alignment method cannot be accurately estimated. The evaluation unit defines the track sequence of the trainer asThe expert standard track sequence is defined asWhereinAndRespectively represent the firstAnd (b)Three-dimensional space coordinate vectors of the sampling points. Algorithm construction of oneAnd find a distance matrix from (1, 1) to (1) by dynamic programmingIs arranged such that the cumulative distance over the path is minimized. Cumulative distanceThe calculation formula of (2) is as follows:
;
wherein, the Spatial vector representing sampling pointsAnd (3) withEuclidean distance between them; representing taking a minimum function. The assessment unit finally outputs the normalized minimum accumulated distance as a quantitative score of the operation normalization, and the smaller the value is, the closer the operation path of the trainer is to the expert standard, so that objective assessment of the operation manipulation is realized.
In the aspect of quantitative evaluation of team cooperation and crisis processing capacity, the multi-dimensional synchronous evaluation unit constructs a time sequence association analysis model based on millisecond time stamps of the full-flow data acquisition unit. The system first identifies the trigger event triggered by the multi-module linkage control unit and the occurrence time of the trigger eventFor example, simulating the time of 90% of human oxygen saturation drop, and then searching the log file for predefined response event and the time of occurrence of the response eventFor example, the moment when the anesthesia monitoring simulation system inputs the pressure boosting medicine or the moment when the breathing parameters are adjusted. The evaluation unit calculates a cooperative response delay. If it isExceeding a preset clinical safety threshold (e.g. a delay of more than 10 seconds of compressions after cardiac arrest), or atAfter which no effective is detectedThe system will determine a collaboration fault. For compound operations involving multiple roles, the system builds a sequence of events that verifies whether the timing of the series of actions of the nurse delivering the instrument, the doctor's hemostatic operation, and the anesthesiologist's administration meets preset first aid procedure criteria (StandardOperatingProcedure, SOP).
Based on the calculation result, a report generation engine in the multi-dimensional synchronization evaluation unit performs data visualization and text generation processing. The engine maps the operation score, the physiological control score and the cooperative response score to various axes of the radar map to generate a skill ability image. Meanwhile, a Natural Language Generation (NLG) module calls a preset text template, and description fields are automatically filled according to quantitative data. For example, when it is detected that the period of the electrocoagulation operation coincides with the period of the bleeding event and the Z-axis coordinate of the trajectory depth exceeds the safety threshold, the engine automatically generates a description that, at X minutes and X seconds of the operation, the bleeding of the gallbladder triangle is caused due to the excessive depth of the electrocoagulation hook operation (exceeding Y mm), and the error causes the increase of Z milliliters of the blood loss of the simulated human. The generated evaluation report comprises a statistical chart, a video clip link with a key frame mark and a text comment based on causal logic analysis, and is rendered and displayed on a display terminal for a teacher and a trainer to carry out multiple disc analysis.
Referring to fig. 5, the present invention provides an emergency minimally invasive simulated training system including a scene editing function, which integrates a surgical scene editor module in a multi-module coordinated control unit. The editor module is configured to generate a standardized profile for defining a particular surgical training task, the profile including virtual anatomy definitions, physiological parameter baseline settings, and complication trigger logic rules.
The surgical scene editor module provides a graphical interactive interface comprising a three-dimensional anatomic model library, a physical attribute configuration panel, and a logical event orchestrator. In the anatomy definition phase, the system responds to user instructions to select a model of a pending organ (e.g., gall bladder, appendix, or uterus) from a model library and loads the selected model into a virtual abdominal space coordinate system. The system receives coordinate transformation parameters input by a user, including translation vectorsRotation matrix and scaling to determine the relative position of each anatomical model in the virtual field of view. At the same time, the system receives physical attribute parameters set by the user for specific anatomical regions (e.g., gall bladder triangle, vessel root), including tissue elastic modulus, texture map ID, and tissue smoke generation rate coefficient when cut by electrocoagulation.
In the physiological parameter and complication logic setting stage, the editor module receives initial physiological state reference values set by a user for an advanced comprehensive simulator, wherein the initial physiological state reference values comprise a systolic pressure/diastolic pressure range, a basic heart rate, basic blood oxygen saturation and respiratory frequency. The logical event orchestrator allows the user to build a conditional trigger rule tree. The user defines a trigger condition (e.g., the tip of the electrocoagulation device is less than 2 mm from the portal vein model and has a duration exceeding 3 seconds) and associates a corresponding outcome event (e.g., the simulated human bleeding rate is set to 50 ml/min while an instruction is sent to the anesthesia care simulation system that the heart rate increases 20 times per minute). The editor module supports setting random probability parameters such that the same trigger condition activates a consequence event only when a preset probability (e.g., 30%) is met to increase the unpredictability of the training.
After editing, the system executes serialization operation, and encapsulates the spatial coordinate data, the physical attribute parameters, the physiological reference values and the logic triggering rules into a structured scene configuration file (for example, in JSON or XML format). The file contains header information (surgical name, difficulty level), a list of model nodes (including model ID and transformation matrix), a list of parameter nodes (including vital sign key pairs), and a list of event nodes (including trigger threshold and response instructions). The configuration file is stored in a local memory or a connected database of the multi-module coordinated control unit as an input source for a subsequent training task.
In the training task starting stage, the multi-module linkage control unit executes a scene loading program, and reads and analyzes the selected scene configuration file. The control unit sends the analyzed anatomic model data and physical attribute parameters to a graphic rendering engine of the laparoscope simulated training system for reconstructing a three-dimensional operation scene and initializing force feedback parameters, sends the analyzed physiological reference values to a controller of an advanced comprehensive simulation simulator to initialize vital signs of the simulator to a preset state, and loads logic triggering rules into a real-time monitoring kernel of the multi-module linkage control unit. After training is started, the real-time monitoring kernel polls and matches logic triggering rules in a millisecond-level period according to real-time data (instrument position and operation time) fed back by the full-flow data acquisition unit, and once the conditions are met, corresponding control instructions are distributed to the related subsystems immediately, so that the self-defined operation flow and complication simulation is realized.
Referring to fig. 6, the invention provides a specific application flow of a multi-module linkage control and multi-dimensional synchronous evaluation first-aid minimally invasive simulation training system, which comprises four stages of preoperative preparation verification, intraoperative operation monitoring, sudden complication linkage response and postoperative multi-dimensional data multi-disc.
In the system initialization and preoperative preparation stage, the multi-module linkage control unit sends a state inquiry command to the anesthesia monitoring simulation system, the laparoscope simulation training system and the advanced comprehensive simulator. At this time, the main display screen of the laparoscopic simulation training system is in a locked state, only the basic information of the case is displayed, and the access to the operation interface is forbidden. The system monitors input data of the anesthesia monitoring simulation system in real time, when an anesthesia induction operation completion signal is detected, the anesthesia induction operation completion signal specifically comprises that the injection amount of atropine reaches 0.4 milliliter, the injection amount of sodium thiopentetate reaches 3 milliliters, and vital sign parameters of an advanced comprehensive simulation human are monitored to be stable in a preset range (heart rate is 60 to 100 times/minute, blood oxygen saturation is greater than 95 percent, blood pressure is contracted to 90 to 140 millimeters of mercury), and meanwhile, after an instrument counting confirmation signal is received, the multi-module linkage control unit judges that preoperative preparation conditions are met. Subsequently, the control unit sends an unlocking instruction to the laparoscopic simulation training system, the main screen unlocks and loads the three-dimensional anatomical scene of the laparoscopic cholecystectomy, allowing the trainer to activate the pneumoperitoneum simulator and insert the trocar.
And entering an operation stage in operation, and synchronously recording multi-source data by the full-flow data acquisition unit according to a unified time reference. When an operator controls a laparoscopic instrument simulator to dissect a virtual gallbladder triangle region, the system records the spatial coordinate track (X, Y, Z axis data) of the instrument end and the power set point of the electrosurgical generator simulator. Meanwhile, the system continuously collects real-time physiological parameters of the advanced comprehensive simulation man. If the operator sets the electric coagulation power to 120 watts and contacts the vascular model for more than 1 second during the process of separating the cholecyst artery, the system determines that the energy instrument is not used properly and immediately triggers a logic event of the cholecyst artery rupture bleeding.
During the burst accident and emergency linkage phase, the system performs a series of concurrent response actions. The three-dimensional rendering engine of the laparoscopic simulation training system generates a pulsatile bleeding pattern in the field of view, with the field of view definition decreasing exponentially with time. Synchronously, the multi-module linkage control unit modifies the physiological parameter output of the advanced integrated simulator, linearly reduces the blood oxygen saturation value from 98% to 88% in 10 seconds, reduces the blood pressure value to 80/50 mmHg, and increases the heart rate to 130 beats/min. At this time, the display interface of the anesthesia monitoring simulation system is automatically switched to the emergency mode, the abnormal waveform is highlighted, and an audible and visual alarm is sent out. Meanwhile, the control unit sends an instruction to the mechanical control core of the advanced comprehensive simulation man, and the chest compression detection sensor is unlocked to be in a state of being acceptable for cardiopulmonary resuscitation operation. The system plays the request support prompt tone through the voice module, records the anesthesia role adjusting the drug dosage (such as injecting epinephrine analog), the nurse role transmitting the hemostatic clip and the time stamp of the hemostatic operation of the surgeon. When the system detects that the virtual hemostatic clamp is accurately placed at the proximal end of the bleeding point and the physiological parameters of the simulated medicament are raised to a safe threshold value (the blood oxygen saturation is more than 95%) after the simulated medicament is injected, the system automatically releases the emergency state and restores the normal operation vision and the monitoring mode.
In the postoperative disc recovery and evaluation stage, when the system detects that all laparoscopic instruments exit from the sleeve port, pneumoperitoneum pressure returns to zero and the simulated life sign is stable, the multidimensional synchronous evaluation unit generates a comprehensive evaluation report. The report marks 10 minutes 25 seconds on the time axis as a bleeding error point. The system responds to clicking operation of a user on the time point, and synchronously plays back the contents of three windows on the same interface, wherein the first window displays an operation video frame of vascular rupture caused by exceeding of electric coagulation power, the second window displays a corresponding instrument tip Z-axis depth and power curve, and the third window displays a trend chart of synchronous sudden drop of blood oxygen saturation on a monitor. The system calculates team cooperation scores according to the time consumption and blood loss simulation values of the hemostatic operation, and generates a histogram to compare response time differences of the current training and the historical training for treating similar complications, so that quantitative evaluation of technical operation and team cooperation capacity is realized.