DE102024105566A1 - Control device, robotic assembly, surgical microscope and method for controlling the movement of a surgical instrument - Google Patents

Abstract

A control device (1) for controlling the movement of a surgical instrument (2) is described, wherein the control device (1) comprises a sensor device (6), an evaluation device (7) and a human-machine interface (8), and wherein the sensor device (6) is designed to optically capture image data of a region of a tissue (4), the evaluation device (7) is designed to determine at least one feature of the tissue (4) in the detected region based on image data captured by the sensor device (6), a change in the feature indicating contact between the surgical instrument (2) and the tissue (4), and the human-machine interface (8) is designed to output feedback signals based on the at least one determined feature.

Description

The present invention relates to a control device, a robotic assembly, and a surgical microscope, for example for microsurgery, in particular for ophthalmic surgery and neurosurgery. The invention also relates to a method for controlling the movement of a surgical instrument.

Currently, surgical procedures are typically performed manually. In certain areas, such as laparoscopy, telemanipulable surgical robots are gaining popularity. In the future, systems with task-level or higher levels of automation will become increasingly important. Such robotic systems will require advanced sensor technology tailored to the specific requirements and functions adapted to the specific procedure being performed.

During manual procedures, operators, such as surgeons, rely on their visual feedback to estimate depth and thus instrument-tissue interaction. Surgeons acquire this skill through years of experience. A robotic system therefore requires appropriate feedback to ensure safe and reliable contact with the tissue.

In many ophthalmic surgical procedures, surgical instruments must come into contact with delicate tissue, for example, to grasp a capsular bag during cataract surgery or to remove a membrane on the retina during membrane peeling in retinal surgery. The optimal forces that are applied are in the range of a few millinewtons (mN). These optimal forces between the tip of the surgical instrument and the tissue to be treated are often below the range perceptible to humans. If the forces are too high, there is a risk that the surgical instrument will damage surrounding tissue, e.g., the retina during retinal surgery or the zonular fibers during cataract surgery. If the forces are too low, the surgical step may not be possible, e.g., because reliable grasping of the tissue is not possible. Capturing or sensing force is also complicated because during many ophthalmic surgical steps, forces and torques are absorbed at the point of entry into the eye, e.g., through a trocar for retinal surgery or a corneal incision during cataract surgery. As a result, the surgeon (hereafter referred to as the operating surgeon) cannot rely on haptic feedback during many ophthalmic surgical steps and must rely on sensors outside of their own body perception.

Ophthalmic surgical microscopes are suitable for recording the relative lateral position between a surgical instrument and the tissue and for a rough estimation of depth, e.g., the depth of penetration of the instrument into the tissue. To compensate for inadequate force and depth detection, surgeons often rely on additional cues, such as observed shadows, color changes, or wrinkling of the tissue under stress. In other cases, surgical steps must be repeated at different depths until, for example, the instrument engages and the surgical step can be successfully performed. This is often only recognizable by tissue reactions, such as the propagation of a tear point. Many visual cues that a surgeon typically captures are difficult to detect with a camera system or computer algorithms.

Femtosecond lasers, for example, are available to assist in cataract surgery. These systems typically feature an integrated optical coherence tomography (OCT) scanner for enhanced depth sensing. However, this incurs additional costs, requires contact lenses to be placed on the eye, and, in general, OTC systems may not be able to adequately detect contact between the surgical instrument and the tissue.

In the document US 2011 / 0 106 102 A1 A robotic surgical system for eye surgery is described, using an optical sensor in the area of the tip of a surgical instrument. In the document US 2017 / 0 312 431 A1 An ophthalmic surgical system is described that enables pressure measurement during an infusion. The document US 2016 / 0 074 212 A1 discloses a device for introducing a medicament into an eye, which comprises a detection and visualization system for detecting and visualizing penetration of the choroid by an injection needle as feedback for a surgeon and/or for automated control.

In the document US 2007 / 0 151 390 A1 A robotic surgical instrument with a force and torque sensor at the tip is described. The documents US 2012 / 0 265 102 A1 and WO 2017 / 118949 A1 Cardiac catheters with force sensors at the tip are revealed.

Against the background described, the object of the present invention is to provide an advantageous control device, for example for microsurgery, for controlling the movement of a surgical instrument. Further objects include providing an advantageous robotic assembly, a surgical microscope, and a method for controlling the movement of a surgical instrument, each for example for microsurgery.

The above objects are achieved by a control device according to claim 1, a robotic arrangement according to claim 9, a surgical microscope according to claim 17, and a method for controlling the movement of a surgical instrument according to claim 18. The dependent claims contain further advantageous embodiments of the invention.

The control device according to the invention, preferably for microsurgery, for controlling the movement of a surgical instrument, in particular a surgical tool, comprises a sensor device, an evaluation device, and a human-machine interface. In other words, a sensor device can be understood as a means for detecting physically measurable quantities, such as a camera, a detector, a signal detection device, or a measuring device, etc. In other words, an evaluation device can be understood as a means for receiving, processing, and outputting data and/or signals, e.g., a computer or a data processing system. In other words, a human-machine interface (HMI) can be understood as a user interface for inputs by a user and/or outputs to the user.

The sensor device is designed for optically capturing, in particular visually capturing, image data of a region of tissue, i.e., the tissue region to be manipulated. The evaluation device is designed to determine at least one feature, i.e., an optically detected feature, of the tissue in the detected region based on image data captured by the sensor device, wherein a change in the feature indicates contact or interaction between the surgical instrument and the tissue. The human-machine interface is designed to output feedback signals, e.g., to a surgeon and/or a robotic manipulator, based on the at least one detected feature.

In other words, the control device according to the invention is designed to detect and evaluate tissue changes caused by contact with the surgical instrument. Determining the at least one feature of the tissue can comprise detecting and/or detecting and/or qualifying and/or quantifying the feature. For example, changes in the tissue with regard to its color and/or geometric shape can be detected and evaluated.

A surgical instrument is understood here to be a physical tool or device with which tissue can be physically influenced as part of a surgical procedure.

The at least one feature determined based on image data may be a parameter or quantity, e.g., a color or color change, or a distance, or a dimension, or a shape of a shadow, or a type of tissue reaction, or another parameter that characterizes a tissue reaction caused or induced by an interaction or contact with a surgical instrument.

The control device according to the invention, in particular for microsurgery, is preferably designed to control the movement of a surgical instrument during a surgical procedure or intervention. It can be a control device for ophthalmic surgery or neurosurgery. In particular, the control device according to the invention can be designed for individual steps, e.g., capsulorhexis or capsular bag polishing, during cataract surgery. Further application examples include membrane peeling in retinal surgery, e.g., the brushing step for detaching the torn membrane, corneal surgery, or generally avoiding excessive force during phacoemulsification or lens injections. Furthermore, the control device according to the invention can be designed for assisted or automated suturing in microsurgery, e.g., in ophthalmic surgery or neurosurgery.

The control device according to the invention has the advantage that it provides improved feedback for microsurgical applications when controlling the movement of the surgical instrument, ensuring optimal execution of the planned intervention or procedure. The control system is thus highly reliable. It is also cost-effective, as no expensive sensor technology is required.

In an advantageous variant, the evaluation device is designed to define a trajectory of the surgical instrument based on image data acquired by the sensor device and/or based on the at least one determined feature and/or based on the output feedback signal. Defining the trajectory can also include determining and/or adapting the trajectory. In particular, the trajectory can be continuously adapted depending on the behavior of the determined feature, in particular a change in the determined feature. This is advantageous for an assistance function for manual guidance of the surgical instrument by a surgeon or for an at least partially automated robotic application. The effect of the surgical instrument on the tissue can be optimized by means of a corresponding trajectory.

In an advantageous variant, the control device can be designed to output a trajectory for a movement, e.g., an at least partially automated movement, of the surgical instrument based on image data acquired by the sensor device and/or based on the at least one determined feature and/or based on the output feedback signal. The trajectory can comprise at least one repeating movement pattern. In a preferred variant, the control device is designed to control an at least partially automated movement of the surgical instrument, in particular the movement of a tip of the surgical instrument, along a defined trajectory comprising a repeating movement pattern.

The trajectory can, for example, comprise a continuous path along an imaginary surface, where the surface can be conical or cylindrical in shape and/or the path can have circular and/or elliptical and/or helical and/or parabolic and/or hyperbolic and/or linear sections. The movement can be configured at least partially as a grasping movement.

In a further variant, the trajectory can lie at least partially in a single plane. This plane can extend perpendicular to a lateral plane and/or perpendicular to an object plane of the sensor device. The object plane is understood to be a plane in which, for example, the tissue is optically detected. Using these variants, gripping movements or cutting movements of the surgical instrument can be optimized, for example.

The sensor device can comprise at least one camera for simultaneously capturing a plurality of laterally arranged pixels and/or a scanning device for time-sequentially capturing laterally arranged pixels. The sensor device can comprise a stereo camera and/or an optical coherence tomograph (OCT) and/or a surgical microscope and/or a confocal system. Low-resolution axial information can be fused, combined, or supplemented with more precise information, e.g., information derived from a tissue reaction. In this way, reliable data, particularly axial data, for controlling the surgical instrument can be obtained in a simple and cost-effective manner.

The human-machine interface can further be designed to input a lateral path of movement of the surgical instrument. For example, telemanipulation of the lateral position of the surgical instrument can be provided, while a specific, e.g., predetermined, axial position of the surgical instrument relative to the tissue is maintained by means of a robotic control. The control device can be designed to plan and/or define a trajectory of the surgical instrument for partially or fully automated control of the surgical instrument by means of a robotic arrangement.

The robotic assembly according to the invention, in particular for microsurgery, comprises at least one robotically guided surgical instrument, in other words a surgical instrument whose movement can be controlled by a robot, i.e., one that can be moved or operated by a robot, and at least one robotic manipulator, in other words a robotically movable device for physical interaction with the environment, having a fastening device, in other words, a means for establishing a reversible physical connection, for fastening the surgical instrument to the robotic manipulator. The robotic assembly comprises a previously described control device according to the invention, which is designed to control the movement of the surgical instrument by means of the robotic manipulator. The robotic assembly can be designed for partially or fully automated operation. The robotic assembly according to the invention has the same features and advantages as the previously described control device according to the invention.

The robotic manipulator preferably has at least 3, e.g. 6, degrees of freedom of movement for manipulation and the control device The device is preferably designed to control the movement of the surgical instrument by means of the at least 3 degrees of freedom, e.g. 6 degrees of freedom, of the robotic manipulator. The degrees of freedom can be translational degrees of freedom and/or rotational degrees of freedom. For example, 3 translational degrees of freedom relative to a Cartesian coordinate system and/or 3 rotational degrees of freedom, e.g. around the respective axes of the Cartesian coordinate system, can be provided. Depending on the requirements of the application and the reference system used, e.g. a Cartesian coordinate system or cylindrical coordinate system or spherical coordinate system, etc., a certain number of translational degrees of freedom and/or rotational degrees of freedom can be provided. A manipulator with 6 degrees of freedom comprising 3 translational degrees of freedom and 3 rotational degrees of freedom has the advantage that the movement can be manipulated without restriction in all directions. For certain applications, a smaller number of degrees of freedom may be sufficient. A manipulator with, for example, only 3 or 4 degrees of freedom of movement has the advantage that it is more cost-effective and potentially less prone to errors than a manipulator with a higher number of degrees of freedom of movement.

Advantageously, the robotic arrangement is designed such that an entry point into a specific tissue, e.g., a defined tissue of a specific organ, is defined or can be defined as a reference point, e.g., in the form of a coordinate origin, for a movement of the surgical instrument. The lateral position of the surgical instrument corresponds to an angular orientation of the surgical instrument or the manipulator with respect to the reference point, i.e., at least one rotation angle of the surgical instrument or the manipulator with respect to the reference point, and the axial position of the surgical instrument corresponds to a depth of entry of the surgical instrument or the manipulator through the entry point. The entry point can be defined by a trocar or a puncture into a tissue, e.g., the cornea. Defining the entry point as a reference point enables intuitive control of the movement of the surgical instrument, modeled on the movement pattern during a manual procedure and also computationally efficient.

The robotic arrangement can be designed to interrupt a movement, in particular the repetitive movement pattern, of the surgical instrument if contact between the tissue and the surgical instrument is detected, e.g., by means of the evaluation device. This has the advantage that a potentially undesired effect on the tissue can be avoided. The robotic arrangement can be designed to switch to a mode with a predetermined, e.g., user-defined, lateral movement path. Additionally or alternatively, the robotic arrangement can be designed to maintain a movement, e.g., a fixed axial movement and/or the repetitive movement pattern, of the surgical instrument while simultaneously moving the surgical instrument along a lateral path, e.g., a lateral path that can be determined by a surgeon or user, or a fixed or user-defined lateral path.

The robotic arrangement can be designed for partially or fully automated operation, wherein the lateral portions of a trajectory of the surgical instrument are controllable based on data acquired by the sensor device, wherein contact between the surgical instrument and the tissue is maintained during the execution of a movement, e.g., an axial movement, of the surgical instrument with a repetitive movement pattern. Partially automated operation has the advantage that certain movements of the surgical instrument are manually controllable, thus allowing a surgeon to act independently in this regard, while simultaneously utilizing the high accuracy and precision of automated operation for other movements. Fully automated operation has the advantage of ensuring a high accuracy and precision of the surgical procedure.

The term "control" also encompasses regulation in the sense of control and regulation technology. In particular, contact between the tissue and the surgical instrument can be controlled semi-automatically or fully automatically. In this case, the robotic arrangement can be designed to be controlled using a closed-loop controller to maintain contact between the tissue and the surgical instrument, e.g., for a specified period of time.

In a further variant, the robotic arrangement can comprise a human-machine interface, by means of which the surgical instrument can be controlled by a user with regard to its lateral movement, i.e., with regard to the lateral movement components, in relation to the tissue or the tissue surface, while an axial or medial movement, i.e., the axial or medial movement component, of the surgical instrument in relation to the tissue or the tissue surface can be controlled automatically. The control can be fully or partially automated and/or in the form of a closed-loop control. For example, an input device, e.g., a joystick or similar, can be provided, by means of which a surgeon or user can control the surgical instrument laterally while maintaining contact or the axial relationship between the surgical instrument and the tissue based on data acquired by the sensor device. The described variant has the advantage of avoiding excessive or insufficient force exerted by the surgical instrument on the tissue, while simultaneously enabling manual lateral control of the surgical instrument.

Furthermore, the robotic arrangement can comprise a human-machine interface with a planning function, i.e., a corresponding component, by means of which a tissue area to be treated can be defined, e.g., by a user, based on lateral position data acquired by the sensor device. The robotic arrangement can be configured to perform a specified surgical step at least partially, i.e., partially or fully, automatically based on data acquired by the sensor device regarding contact between the surgical instrument and the tissue surface, e.g., based on movement with a repetitive movement pattern. For example, by means of the planning functions, a surgeon or user can define a spatial area to be treated based on lateral position data acquired by the sensor device, and the robotic arrangement can be configured to perform a surgical step, e.g., gripping a capsular bag membrane, at the user-defined lateral position, while the surgical instrument is moved into a specific, e.g., fixed, axial position for the procedure. This is preferably done based on the at least one determined feature and/or the determined feedback signals. A closed-loop control function can be provided or present, which is designed to automatically move the surgical instrument to the correct lateral position and the correct distance from the tissue.

Advantageously, the robotic arrangement comprises a human-machine interface designed to output information for assistance, in particular warnings and recommendations, e.g., regarding the trajectory of the surgical instrument. This can improve the quality of the procedure and take possible safety requirements into account. Closed-loop control can also be provided in this context.

In a further variant, the robotic assembly can be switchable between two operating modes during the execution of a surgical procedure or task, wherein in a first operating mode the surgical instrument and the tissue are in contact, and in a second operating mode the surgical instrument and the tissue are not in contact. The second operating mode can be designed or used for repositioning the surgical instrument. The selection of the operating mode can be based on data acquired by the sensor device. The operating modes can be switched manually and/or automatically. Closed-loop controls can be provided within the operating modes. For example, in the first operating mode, a membrane can be grasped by the surgical instrument to tear it off, and in the second operating mode, the membrane can be released to reposition the surgical instrument. In both operating modes, the sensor device or multiple sensor devices provide the information required to maintain a fixed or definable contact between the surgical instrument and the tissue. The switching can be performed manually, for example, based on lateral position information, or automated based on the progress of a surgical step and information from the sensor device.

The control device according to the invention and/or the robotic arrangement according to the invention can be designed to differentiate between different, e.g., classified, types of contact between the tissue and the surgical instrument based on the at least one determined feature, e.g., a specific observed tissue reaction. The output of feedback signals and/or the control of the surgical instrument can be provided based on a type of contact determined by means of the differentiation. In one example, a distinction can be made or differentiated between good contact, moderate contact, and no contact between the tissue and the surgical instrument. Good contact can be recognized in the case of a caspulorhexis operation by an expansion of the tear point. Moderate contact can be recognized by the fact that a relatively large movement of the surgical instrument leads to a relatively small expansion of the tear point. The presence of no contact can be recognized by the lack of expansion of the tear point.

The robotic arrangement can be designed to detect a movement of the tissue and, in the event of a detected movement of the Tissue, for example as a result of heartbeat or breathing, at least the surgical instrument, and possibly also other components, can be tracked to the detected movement. This can reduce or avoid unplanned effects of the surgical instrument on the tissue caused by the movement of the tissue. The tracking can be designed or carried out in an automated manner, e.g. partially or fully automated. The tracking can be restricted by definable or fixed limits. A movement of the tissue can be detected by means of the sensor device or other sensor devices, e.g. a head tracker or an eye tracker. The robotic arrangement can be designed to detect a movement of the tissue in a lateral and/or axial direction. Furthermore, the robotic arrangement can be designed to track the surgical instrument and possibly other components in a lateral and/or axial direction. Specific restrictions on the movement can be provided in this case.

For example, a sensor system for acquiring lateral position data can be provided, which provides axial position data with low resolution, e.g., a stereo camera system or OCT system. The data acquired with this sensor system can be superimposed, supplemented, or fused with axial data derived from the at least one determined feature. The axial data or information can be used, in particular, to reposition the surgical instrument to a point at which a repetitive movement pattern is to be executed.

A proposed algorithm for evaluating tissue response in the case of capsulorhexis may take into account that the length of movement of the surgical instrument is twice the extension of the tear point.

The robotic arrangement can be designed to provide haptic feedback to a surgeon for axial control of the surgical instrument, e.g., in the case of teleoperative applications. This can be implemented via the human-machine interface, e.g., a joystick, in the form of vibrations, or through active resistance in a specific direction of movement. Furthermore, in an exemplary embodiment for ophthalmic surgery, axial movement below a capsular bag can be restricted if sufficient grip of the tissue is achieved by the surgical instrument.

In general, a tissue reaction to contact with a surgical instrument can be determined from an optical flow or sequentially acquired images of the tissue. The optical flow of an image sequence is the vector field of the velocity of visible points of the object space projected into the image plane in the reference system of the imaging optics. Furthermore, an area or region in which the surgical instrument is located can be excluded when detecting differences between sequentially acquired images. In a special embodiment, an algorithm for detecting differences between sequentially acquired images can be designed to distinguish between cases in which the entire image is moving (e.g., due to a vibration of the microscope) and cases in which different sub-areas of the image (e.g., captured tear edges of the tissue) move at different speeds.

The surgical microscope according to the invention, e.g., for microsurgery, comprises a control device according to the invention as described above or a robotic arrangement according to the invention as described above. The surgical microscope according to the invention has the features and advantages already described in connection with the control device according to the invention and the robotic arrangement according to the invention.

The inventive method for controlling the movement of a surgical instrument comprises the following steps: optically acquiring image data of a region of tissue using a sensor device; determining at least one feature of the tissue in the acquired region based on the image data acquired by the sensor device, wherein a change in the feature indicates contact between the surgical instrument and the tissue, using an evaluation device; and outputting feedback signals based on the at least one determined feature using a human-machine interface. The movement of the surgical instrument is preferably controlled based on the feedback signals. The control can be at least partially automated. The movement can comprise a repetitive movement pattern, as already described in detail above. The inventive method has the features and advantages described in connection with the inventive control device and the inventive robotic arrangement. It can be implemented in the variants described in connection with the inventive control device and the inventive robotic arrangement. The inventive control device, the inventive robotic arrangement, and the inventive surgical microscope can be designed to carry out the method according to the invention.

The invention will be explained in more detail below using exemplary embodiments with reference to the accompanying figures. Although the invention is illustrated and described in more detail by the preferred embodiments, the invention is not limited to the disclosed examples, and other variations may be derived therefrom by a person skilled in the art without departing from the scope of the invention.

The figures are not necessarily detailed or to scale and may be enlarged or reduced to provide a better overview. Therefore, the functional details disclosed herein are not to be considered limiting, but merely as an illustrative basis for providing guidance to one skilled in the art for variously employing the present invention.

• 1 zeigt schematisch eine erfindungsgemäße Steuervorrichtung in Form eines Blockdiagramms.
• 2 zeigt schematisch eine erfindungsgemäße robotische Anordnung in Form eines Blockdiagramms.
• 3 zeigt schematisch ein erfindungsgemäßes Operationsmikroskop in Form eines Blockdiagramms.
• 4 zeigt schematisch ein erfindungsgemäßes Steuerverfahren in Form eines Flussdiagramms.
• 5 zeigt schematisch eine perspektivische Ansicht auf eine Oberfläche eines Kapselsackgewebes.
• 6 zeigt schematisch die Bewegung einer Spitze eines chirurgischen Instruments während einer Kapselsacköffnung (Kaspulorhexis).
• 7 und 8 zeigen schematisch Beispiele für die Bewegung eines zum Bürsten ausgelegten chirurgischen Instruments.
• 9 zeigt schematisch ein Beispiel für eine Annäherung eines chirurgischen Instruments an ein zu manipulierendes Gewebe.
• 10 und 11 zeigen jeweils schematisch ein in der Hand haltbares mechatronisch steuerbares chirurgisches Instrument zum Bürsten von Gewebe.

As used herein, the term "and/or," when used in a series of two or more elements, means that any of the listed elements may be used alone, or any combination of two or more of the listed elements may be used. For example, if a composition is described containing components A, B, and/or C, the composition may contain A alone; B alone; C alone; A and B in combination; A and C in combination; B and C in combination; or A, B, and C in combination.

• 1 shows schematically a control device according to the invention in the form of a block diagram.
• 2 shows schematically a robotic arrangement according to the invention in the form of a block diagram.
• 3 shows schematically an inventive surgical microscope in the form of a block diagram.
• 4 shows schematically a control method according to the invention in the form of a flow chart.
• 5 shows schematically a perspective view of a surface of a capsular bag tissue.
• 6 shows schematically the movement of a tip of a surgical instrument during capsular bag opening (capsulorhexis).
• 7 and 8 show schematic examples of the movement of a surgical instrument designed for brushing.
• 9 shows a schematic example of an approach of a surgical instrument to a tissue to be manipulated.
• 10 and 11 each schematically shows a hand-held, mechatronically controllable surgical instrument for brushing tissue.

The 1 shows a schematic diagram of a control device according to the invention in the form of a block diagram. The control device 1 is designed to control 3 the movement of a surgical instrument 2, preferably for microsurgery. The control device 1 comprises a sensor device 6, an evaluation device 7, and a human-machine interface (HMI) 8. The sensor device 6 is designed to optically capture image data of a region of a tissue 4, in particular a region of a surface 5 of the tissue 4. The tissue 4 is preferably a biological tissue, for example a tissue of an organ, e.g., eye tissue, which is to be acted upon by means of the surgical instrument 2. The sensor device 6 can be a camera, for example a camera of a surgical microscope, or a scanning device for the time-sequential capture of laterally arranged pixels, or an OCT.

The evaluation device 7 is designed to determine at least one feature of the tissue 4 in the detected region based on image data acquired by the sensor device 6. This feature is a feature whose change characterizes contact between the surgical instrument 2 and the tissue 4. In other words, the feature depicts a tissue change resulting from an action by the surgical instrument 2. Advantageously, the evaluation device 7 can be designed to define a trajectory of the surgical instrument 2 based on image data acquired by the sensor device 6. The trajectory can comprise a repeating movement pattern. The repeating movement pattern can comprise circular and/or elliptical and/or helical and/or parabolic and/or hyperbolic and/or linear subregions. In a preferred variant, the control device 1 is designed to control an at least partially automated movement of the surgical instrument 2 along a defined trajectory, for example along a trajectory which comprises a repeating movement pattern.

The HMI 8 is designed to output feedback signals based on at least one determined characteristic. The output of feed Feedback signals can be sent to a surgeon, for example acoustically and/or visually and/or audiovisually and/or haptically. The HMI 8 can also be designed to input a lateral movement path of the surgical instrument 2. In other words, it can be provided that a surgeon manually controls the lateral movement of the surgical instrument 2 using the HMI 8. However, the feedback signals can also be output to a corresponding robotic device for at least partially automated control of the surgical instrument 2.

The signal transmission between the sensor device 6, the evaluation device 7 and the HMI 8 is each marked by an arrow with the reference number 9.

The 2 shows schematically a robotic arrangement according to the invention in the form of a block diagram. The robotic arrangement 10, which is preferably designed for microsurgery, comprises at least one robotically guided surgical instrument 2 and at least one robotic manipulator 12 with a fastening device 11 for fastening the surgical instrument 2 to the robotic manipulator 12. The robotic arrangement 10 also comprises a control device 1, which is designed to control the surgical instrument 2 by means of the robotic manipulator 12. The control device 1 can be a 1 the control device 1 already described.

The robotic manipulator 12 can have at least three degrees of freedom of movement for manipulation, and the control device 1 can be designed to control the movement of the surgical instrument 2 using the at least three degrees of freedom of movement of the robotic manipulator 12. The robotic manipulator 12 can further be designed to interrupt a movement of the surgical instrument and/or to maintain a movement of the surgical instrument depending on the determined feature of the tissue by means of which contact between the surgical instrument 2 and the tissue 4 can be determined. Variants of this have already been described in detail above. The robotic arrangement 10 can be designed for partially or fully automated operation.

The 3 shows schematically a surgical microscope according to the invention in the form of a block diagram. The surgical microscope 13 comprises a robotic arrangement 10 according to the invention, for example one used in connection with the 3 described robotic arrangement 10.

The 4 shows a schematic view of a control method according to the invention in the form of a flow chart. The method shown for controlling a surgical instrument 2 comprises the following steps: In step 21, image data of a region of tissue 4 are optically acquired by means of a sensor device 6, e.g., by means of a camera or a scanning device. In step 22, based on the image data acquired by means of the sensor device 6, at least one feature of the tissue in the acquired region is determined, a change in the feature indicating contact between the surgical instrument 2 and the tissue 4. This is done using an evaluation device, e.g., the evaluation device 7. In step 23, based on the at least one determined feature, feedback signals are output by means of an HMI 8. In an optional step 24, a trajectory for a movement or the control of a movement of the surgical instrument 2 can be determined and output based on image data acquired by the sensor device 6 and/or based on the at least one determined feature and/or based on the output feedback signal. The trajectory can have at least one repeating movement pattern. Also optionally, the movement of the surgical instrument 2 can be controlled along the trajectory.

In the following, embodiments of the invention are explained using ophthalmic surgical applications. First, individual aspects of the invention are described using the example of a capsular bag opening (capsulorhexis). 5 shows a schematic perspective view of a surface 5 of a capsular bag tissue 4. During a capsular bag opening, a portion of the capsular bag 31 is opened, and the underlying lens is exposed through the resulting opening 30. The movement of a surgical instrument used for this purpose is indicated by an arrow with the reference number 32, which depicts a trajectory of the tip of the surgical instrument. During the opening process, a tear point of the tissue 4 moves away from a first position indicated by the reference number 33. This is indicated by an arrow with the reference number 34. An exemplary second position of the tear point is indicated by the reference number 35.

The 6 illustrates the described process in two steps. 6 Above, the movement of the tip of the surgical instrument 2 used is shown in two xy-z diagrams. The xy plane indicates a lateral plane, in this case the plane in which the surface 5 of the capsular bag tissue 4 extends. The z-direction denotes the axial direction, which is perpendicular to the xy-plane. 6 Below are perspective views of the capsular bag tissue 4 corresponding to the diagram above, analogous to the 5 shown view.

In the 6 In the step shown above, the surgical instrument or the tip of the surgical instrument is moved along the part of the trajectory marked with the reference number 36 to the surface 5 of the tissue area 31 of the capsular bag 4 to be moved. Subsequently, as soon as contact is established between the surgical instrument 2 and the capsular bag 4, a lateral movement of the surgical instrument 2 takes place in the xy plane. This is achieved by the 6 The region of the trajectory is indicated below by reference number 37. The movement of the capsular bag tissue 31 caused by the action of the surgical instrument 2 is indicated by an arrow with reference number 38.

The contact between the tip of the surgical instrument and the capsular bag 4 is detected indirectly by the changes (dynamic effects) that can be visually observed in the geometry of the unfolded tissue area 31 or the geometry of the opening 30 or the tear point.

Based on acquired image data, the tip of the surgical instrument can be controlled so that the tear propagates or continues in a calculated direction in the x-y plane. As soon as a movement or change in the geometry of the unfolded tissue region 31 is detected or recorded, for example, by means of a camera and/or a microscope, the surgical instrument is held at the current position in the z-direction, i.e., the current height relative to the tissue region 31, and the movement of the surgical instrument in the x-y plane is continued until a specified target point is reached. Subsequently, the surgical instrument 2 is lifted from the tissue region 31. If no change in the geometry of the tissue region 31 is observed or recorded, the same movement is repeated with a smaller z-value, i.e., a smaller distance from the capsular bag 4 or the tissue region 31. This is repeated until a change in the geometry of the tissue region 31 is observed or recorded.

To detect or identify a change resulting from the action of the surgical instrument 2 on the tissue 4, the geometry of the resulting opening 30 and/or the geometry of the unfolded tissue region 31 and/or the geometry and/or the course of the tear point 33 can be recorded and evaluated. To detect a change, for example, the geometry of the tear, known image processing methods can be used, as can machine-learning image processing methods or methods for image processing or image enhancement. Image processing or image enhancement can be achieved, for example, by increasing the phase contrast or by using dark-field microscopy.

The 7 and 8 show schematic examples of the movement of a surgical instrument designed for brushing. Reference numeral 39 denotes a membrane or tissue 4 to be removed. Reference numerals 40 and 41 denote the trajectories of the tip of the surgical instrument in an axial plane, i.e., a plane running perpendicular to the lateral plane. The trajectories 40 and 41 shown each have a region, denoted by reference numerals 42 and 43, with a repeating movement pattern. 7 In the variant shown, the repeating movement pattern 42 is composed of parabolically shaped sections. 8 In the variant shown, the repeating movement pattern 43 is loop-shaped or elliptically continuous. The dimensions of the repeating movement patterns are preferably in the micrometer range. The movements of the surgical instrument shown along the exemplary 7 and 8 The trajectories shown can, for example, also be used within the framework of the 5 and 6 shown application, until a movement of the tear point is detected.

The 9 shows a schematic example of an approach of a surgical instrument to a tissue to be manipulated. This could be, for example, the 6 This could be the step shown above. In the variant shown, the surgical instrument 2 is repeatedly moved along an elliptical or parabolic trajectory designated by the reference number 44, wherein it is checked in each case whether the movement of the surgical instrument 2 has an effect on the geometry of the unfolded region 31 of the capsular bag 4. If no change in the tissue region 31 is observed, an offset of the movement in the direction of the surface 5 of the capsular bag 4 is added in the z-direction, i.e. the distance to the tissue region 31 or the surface 5 is reduced and the movement is repeated at this distance. The various successively executed movements are designated by the trajectories 44.

If a change in the geometry of tissue region 31 is detected or determined, which is the case here with trajectory 45, the z-position is limited to the corresponding height or a specified lower value, and the movement continues at this height until the planned lateral extension of the tear is reached. Subsequently, the surgical instrument 2 is lifted from the tissue 4. Once a specific, specified, or desired displacement of tissue region 31 is achieved, a subsequent step can be planned or performed.

In the 9 Reference numeral 46 indicates a desired point of application of the surgical instrument 2 on the tissue 4. Reference numeral 45 indicates the trajectory in which a movement of the tissue region 31 was determined. Upon reaching trajectory 45, the movement in the z-direction is simultaneously stopped, and the process continues with only a movement in the xy-direction, i.e., in the lateral direction.

In the following, the 10 and 11 A variant is described in which the surgical instrument 2 is controlled partly manually and partly automatically. 10 and 11 each schematically show a hand-held, mechatronically controllable surgical instrument 2, which comprises, for example, a cystotome 51 for brushing membranes with a tip 47. Here, the surgical instrument 2 is manually controllable in the lateral direction. By means of a control device 1 according to the invention or a robotic manipulator 12 according to the invention, the axial position, as indicated by the arrow 48 in the 10 marked, automatically adjusted, for example on a micrometer scale. The arrangement shown is designed to achieve a brushing movement, as in the 11 shown, to be carried out automatically. For this purpose, three robotic devices with dimensions in the micrometer range can be provided for controlling the movement of the tip 47 of the surgical instrument 2 in a three-dimensional space using three degrees of freedom of movement. A rotational movement of the cystotome around the point 50 can be provided.

A feedback signal output by the control device 1 according to the invention with regard to an axial position of a tip of a surgical instrument can also be made available to a surgeon as AR feedback (AR - Augmented Reality). As soon as the tearing point of the tissue moves, for example, an area around the tip of the surgical instrument 2 can be displayed in a specified color, for example, green. Alternatively, the tearing point itself can also be displayed in color. As an additional feature, the displayed color can change continuously depending on the determined feature that indicates contact between the surgical instrument 2 and the tissue 4. For example, a first color, for example, red, can be displayed when there is no contact, and a second color, for example, green, can be displayed when a good grip or contact is achieved. Whether contact or grip is good or not can be determined by comparing the movement of the tip of the surgical instrument 2 and the movement of the tearing point. It should be taken into account that the distance of movement of the surgical instrument 2 is greater than the distance of movement of the tear point due to the double layer of the capsular tissue, i.e. the unfolded tissue area 31 and the intact capsular tissue 5.

List of reference symbols:

1

Control device

2

surgical instrument

3

steering

4

tissue

5

Tissue surface

6

Sensor device

7

Evaluation device

8

HMI

9

Signal transmission

10

robotic arrangement

11

Fastening device

12

robotic manipulator

13

Surgical microscope

21

optical acquisition of image data of an area of tissue

22

based on image data acquired by the sensor device, determining at least one feature of the tissue in the acquired area

23

Output of feedback signals based on at least one determined feature

24

Determining and outputting a trajectory to control the movement of the surgical instrument

30

Opening / Cornea

31

unfolded tissue area

32

Movement of the surgical instrument

33

Tear point

34

Continuation of the tear point

35

Tear point

36

first area of the trajectory

37

second area of the trajectory

38

Direction of movement of the unfolded tissue area

39

Membrane / tissue

40

Trajectory

41

Trajectory

42

Area with repetitive movement pattern

43

Area with repetitive movement pattern

44

Trajectory

45

Trajectory

46

Point of attack

47

Great

48

controllable direction of movement

49

robotic device

50

Pivot point

51

Cystotomes

QUOTES CONTAINED IN THE DESCRIPTION

This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 2011 / 0 106 102 A1 [0007]
• US 2017 / 0 312 431 A1 [0007]
• US 2016 / 0 074 212 A1 [0007]
• US 2007 / 0 151 390 A1 [0008]
• US 2012 / 0 265 102 A1 [0008]
• WO 2017 / 118949 A1 [0008]

Claims (18)

Control device (1) for controlling the movement of a surgical instrument (2), characterized in that the control device (1) comprises a sensor device (6), an evaluation device (7) and a human-machine interface (8), wherein the sensor device (6) is designed to optically capture image data of a region of a tissue (4), the evaluation device (7) is designed to determine at least one feature of the tissue (4) in the detected region based on image data captured by means of the sensor device (6), wherein a change in the feature indicates contact between the surgical instrument (2) and the tissue (4), and the human-machine interface (8) is designed to output feedback signals based on the at least one determined feature. Control device (1) according to Claim 1 , characterized in that the evaluation device (7) is designed to define a trajectory (32, 40, 41, 44) of the surgical instrument (2) based on image data acquired by means of the sensor device (6) and/or based on the at least one determined feature of the tissue (4) and/or based on the output feedback signal. Control device (1) according to Claim 1 or Claim 2 , characterized in that the control device (1) is designed to output a trajectory (32, 40, 41, 44) for a movement of the surgical instrument (2) based on image data acquired by means of the sensor device (6) and/or based on the at least one determined feature of the tissue (4) and/or based on the output feedback signal, wherein the trajectory (32, 40, 41, 44) comprises at least one repeating movement pattern (42, 43), and/or the control device (1) is designed to control an at least partially automated movement of the surgical instrument (2) along a defined trajectory (32, 40, 41, 44). Control device (1) according to one of the Claims 2 until 3 , characterized in that the trajectory (32, 40, 41, 44) comprises a continuous path along an imaginary surface, wherein the surface is conical or cylindrical in shape and/or the path has circular and/or elliptical and/or helical and/or parabolic and/or hyperbolic and/or linear sub-regions. Control device (1) according to one of the Claims 2 until 4 , characterized in that the trajectory (32, 40, 41, 44) lies at least partially in a plane, wherein the plane extends perpendicular to a lateral plane and/or perpendicular to an object plane of the sensor device (6). Control device (1) according to one of the Claims 1 until 5 , characterized in that the sensor device (6) comprises at least one camera for simultaneously detecting a plurality of laterally arranged pixels and/or a scanning device for time-sequentially detecting laterally arranged pixels. Control device (1) according to one of the Claims 1 until 6 , characterized in that the human-machine interface (8) is designed to input a lateral path of movement of the surgical instrument (2). Control device (1) according to one of the Claims 1 until 7 , characterized in that the control device (1) is designed to differentiate between different types of contact between the tissue (4) and the surgical instrument (2) based on the at least one determined feature. Robotic arrangement (10) comprising at least one robotically guided surgical instrument (2) and at least one robotic manipulator (12) with a fastening device (11) for fastening the surgical instrument (2) to the robotic manipulator (12), characterized in that the robotic arrangement (10) comprises a control device (1) according to one of the Claims 1 until 8 which is designed to control the movement of the surgical instrument (2) by means of the robotic manipulator (12). Robotic arrangement (10) according to Claim 9 , characterized in that the robotic manipulator (12) has at least 3 degrees of freedom of movement for manipulation and the control device (1) is designed to control the movement of the surgical instrument (2) by means of the at least 3 degrees of freedom of movement of the robotic manipulator (12). Robotic arrangement (10) according to Claim 9 or 10 , characterized in that the robotic arrangement (10) is designed such that an entry point into a specific tissue is defined or can be defined as a reference point for a movement of the surgical instrument (2), wherein the lateral position of the surgical instrument (2) is correlated with an angular orientation of the surgical Instrument (2) or the robotic manipulator (12) and the axial position of the surgical instrument (2) corresponds to an entry depth of the surgical instrument (2) or the robotic manipulator (12) through the entry point. Robotic arrangement (10) according to one of the Claims 9 until 11 , characterized in that the robotic arrangement (10) is designed to interrupt a movement of the surgical instrument (2) if contact between the tissue (4) and the surgical instrument (2) is detected, and/or the robotic arrangement (10) is designed to maintain an axial movement of the surgical instrument (2) with a simultaneous lateral movement of the surgical instrument (2) along a lateral path that can be defined by a surgeon. Robotic arrangement (10) according to one of the Claims 9 until 12 , characterized in that the robotic arrangement (10) is designed for partially automated or fully automated operation, wherein the lateral parts of a trajectory (32, 40, 41, 44) of the surgical instrument (2) are controllable based on data acquired by means of the sensor device (6), wherein contact between the surgical instrument (2) and the tissue (4) is maintained during execution of a movement of the surgical instrument (2) with a repeating movement pattern (42, 43). Robotic arrangement (10) according to one of the Claims 9 until 13 , characterized in that the surgical instrument (2) can be controlled by a user by means of the human-machine interface (8) with regard to its lateral movement in relation to the tissue (4) or the tissue surface (5), while an axial or medial movement of the surgical instrument (2) in relation to the tissue (4) or the tissue surface (5) can be controlled automatically, and/or the human-machine interface (8) comprises a planning function by means of which a tissue region to be treated can be defined based on lateral position data acquired by means of the sensor device (6), and the robotic arrangement (10) is designed to carry out a defined surgical step at least partially automatically based on data relating to contact between the surgical instrument (2) and the tissue (4) acquired by means of the sensor device (6), and/or the human-machine interface (8) is designed to output information for assistance. Robotic arrangement (10) according to one of the Claims 9 until 14 , characterized in that the robotic arrangement (10) can be switched between two operating modes during the execution of a surgical procedure, wherein in a first operating mode the surgical instrument (2) and the tissue (4) are in contact and in a second operating mode the surgical instrument (2) and the tissue (4) are not in contact. Robotic arrangement (10) according to one of the Claims 9 until 15 , characterized in that the robotic arrangement (10) is designed to detect a movement of the tissue (4) and, in the event of a detected movement of the tissue (4), to guide at least the surgical instrument (2) according to the detected movement. Surgical microscope (13) comprising a control device (1) according to one of the Claims 1 until 8 or a robotic arrangement (10) according to one of the Claims 9 until 16 includes. Method for controlling the movement of a surgical instrument (2), characterized in that the method comprises the following steps: - optically capturing image data (21) of a region of a tissue (4) by means of a sensor device (6), - determining at least one feature (22) of the tissue (4) in the captured region using an evaluation device (7), wherein the at least one feature of the tissue (4) is determined based on image data captured by means of the sensor device (6) and wherein a change in the feature indicates contact between the surgical instrument (2) and the tissue (4), - outputting feedback signals (23) based on the at least one determined feature by means of a human-machine interface (8).