Artificial Intelligence in knee arthroplasty

Introduction

Artificial intelligence (AI) is defined as the study of algorithms that give machines the ability to reason and perform cognitive functions such as problem-solving, object, images and word recognition, and decision-making [1]. Over the last 70 years, AI has evolved rapidly with the development of computer models and algorithms designed to replicate human intelligence and perform specific tasks within various industries. Surgeons are key stakeholders in adopting AI-based technologies for medical care. It’s the responsibility of the health-care professionals to guide data scientists and engineers in the development of clinically relevant software. In fact, the goal is to see how AI can help to answer clinically relevant questions and appropriately interpret data to improve patient outcomes. Research and development in this domain are mostly industry driven. As often with new technologies, a rigorous validation process and a clinical relevance analysis are required. The goal of this process is to distinguish which AI-based tool is a clinically relevant tool and which one is just a hype.  Several sections of AI can be used in medical care including analytic models, predictive models and machine learning (ML), natural language processing, robotic, augmented, and mixed realities.

During the past years, many studies reporting the interest of AI-based tools in the orthopedic field have been published. With the growing interest toward AI-based tools in knee arthroplasty (KA), it was our goal to provide a clear and comprehensive analysis of the clinical relevance of the available AI-based tools for the pre, peri and post-operative management of the patients undergoing KA.

Different sub-groups of AI tools

The tools evaluated in this article included different sub-groups of AI tools defined and described as below:
Predictive modeling is a discipline of AI where algorithms generate estimates for a defined target output. Predictive models are “trained” to identify relationships between a set of features (e.g., age, BMI, gender) and the target (e.g., occurrence of myocardial infarction). Statistical models (e.g., regression models) and machine learning (ML) techniques (e.g., random forest models or neural networks) are used to learn the target-predictors relationship in the data [2]. ML enables computers to make predictions by recognizing patterns. ML allows a computer to utilize partial labeling of the data (supervised learning), or the structure detected in the data itself (unsupervised learning) to explain or make predictions about the data without explicit programming. Supervised learning is useful for training a ML algorithm to predict a known result or outcome, while unsupervised learning is useful in searching for patterns within data. Deep learning models (e.g., neural networks with several hidden layers) have seen wide success in image recognition and classification where the input is represented by unstructured data (e.g., pixel values). The predictive models and the ML can be used in several domains in surgical management (decision-making, aid to surgical planning).

Natural language processing (NLP) aims to understand human language and is crucial for large-scale analyses of content such as electronic medical record (EMR) data, such as physicians’ narrative documentation. Computer vision describes machine understanding of images and videos, and advances have resulted in machines in some circumstances achieving human-level capabilities in object and scene recognition. More recently digital technologies used in augmented and mixed realities have been developed to interact with the human senses. These technologies enable user projection into a reality described through a digital memory. Augmented Reality (AR) technologies aim to introduce virtual elements into the user’s environment (e.g., superimposition of the values of bone resection axis and the virtual bone cuts onto real-knee surfaces during TKA procedure) by measuring and understand the user’s reality, processing and then computing the information required, and finally rendering it to project this information to the user in correlation with reality. Mixed Reality (MR) presents the surgeon with holographic elements that align with the real world, and the surgeon can manipulate the digital content generated by the MR device.

A.I. in the preoperative period

Patient decision aid

A critical determinant of success in KA is the indication for surgery. Interest is growing around the decision-making algorithms to determine the appropriateness of TKA surgery as some studies are suggesting that up to 33% of TKA surgery may be performed inappropriately. These patients are then dissatisfied with the outcomes of the TKA [3]. To help surgeons and improve patient selection, organizations such as the American Academy of Orthopedic Surgeons, have developed best practice guidelines in an attempt to the standardize decision-making process. These guidelines are based on analytic models. Following the   development of ML and its use in medicine, ML-based predictive models have been developed to help the surgeon and improve the decision-making process [4] (Figure 1).

Shared decision-making is a ML-based concept that requires effective communication between the patient and the clinician to develop a relationship that promotes integration of patient preferences, values, and needs, with the transfer of knowledge regarding treatments, risks, benefits, and alternatives before making informed decisions. The decision-making process is based on relevant and validated predictive factors, for example demographics or preoperative patient-reported outcome measurements (PROMs) [5].  Other studies have examined the use of ML algorithms to assess specific preoperative parameters, which are useful in the decision-making process. ML algorithms have for example been used to grade knee osteoarthritis severity, reducing the interobserver variability [6], or to identify the models of implants before a TKA revision [7]. A recent study evaluated the ability of a ML algorithm to diagnose prosthetic loosening from preoperative radiographs [8]. The authors reported the ML algorithm identified loosening based on plain radiographs and clinical data with a precision of > 95%.

Surgical Education

Traditionally surgical training (beside theoretical training) used to be based on face-to-face time spent in the operating room, technical skills being acquired by the trainee under the supervision of senior surgeons. With the development of virtual reality, surgical training has evolved. Immersive virtual reality (IVR) is a teaching tool that provides surgical trainees of all levels (Figure 2), access to various techniques that replicate real-life procedures with a lot of advantages: no risk for the patient, no need for costly resources (e.g., cadavers), the learning experience can be obtained without direct supervision, data collection all along the training process. Actual limitations of IVR systems are image quality, degree of presence, cyber-sickness, haptic realism, device-related issues (e.g., battery capacity and wireless technology), and access/cost considerations. Immersive virtual reality can simultaneously assess the technical and cognitive ability of users by assessing decisions on implant choice, tracking procedural errors and efficiency, the use of adjunct operating equipment such as fluoroscopy or retractor placement, and completion times.

Whilst these IVR have been evaluated in hip and shoulder arthroplasty [9, 10], their use for training assessment in KA remains largely underutilized and represent an area for future development. Immersive virtual reality has been used in revision TKA to plan extraction of implants from a virtual bone model where it can be examined in a 360° fashion. Using an assistive mode, feedback on key steps, such as planning the bone resections, implant positioning, sizing, assessment of the virtual range of motion and gap balancing can be provided to surgeon to anticipate difficulties or to achieve a surgical target  [11]. These key steps provide both technical and cognitive practice for the surgeon.

Preoperative planning and modelization

Proper surgical planning in surgery in general and even more for KA is crucial to perform a successful operation. Aspects such as alignment, implant positioning, gap balancing, and the size of the implants can all be planned pre-operatively.  Recent studies have suggested formulas in an attempt to predict TKA component size based on demographic data such as gender, height, weight, age, ethnicity/race, and shoe size. However, these algorithms have shortcomings, as not all predictive factors were being considered, and sizing being limited to one implant system. More recent predictive models have been developed to predict the size of the implants preoperatively [12, 13]. These demographic-based multivariate linear regression models more accurately predicted implanted component size compared to digital templated sizes for both the femoral (p=0.04) and tibial (p<0.01) components [13]. A TKA implant sizing application was created, allowing users to input data and receive individualized sizing predictions and explanations (https://orthopedics.shinyapps.io/TKASizing_Calculator/) [12].

AI-based 3D model reconstruction can be used to improve preoperative surgical planning before TKA. The bone segmentation, first step of this process used to be performed manually by the engineers through a time-consuming and costly process. Using AI-based image recognition software, new tools have been developed reducing the time and the human involvement and consecutively the cost of the process. In a recent study based on a CT scan, AI-based 3D model segmentation tools recreated a 3D model reconstruction of the lower limb more efficiently compared to the usual operator-based reconstruction [14]. The AI-based 3D segmentation tools have been so far mainly developed for spine or trauma surgery. For KA, this process is part of the robotic-assisted systems currently available. In fact, current robotic systems require a 3D plan of the patient bones. This 3D model can be obtained intraoperatively using a process called bone-mapping (imageless system) [15] or using preoperative images acquired through CT scan or specific x-rays (image-based system). The image-based systems (Ct-based or X-ray based systems) are both using a segmentation process providing a 3D reconstruction of the patient knee and global anatomy of the lower limb (Figure 3).

These information have been shown to provide better surgical accuracy for the components alignment, the ligament balance and a better prediction of the implant sizes when performing robotic-assisted TKA compared to image-less robotic-TKA systems [16].

A.I. during the knee arthroplasty procedure

Robotic-assisted knee arthroplasty

A robot is defined as a machine capable of automatically carrying out a complex series of actions, especially programmable by a computer. The semiautonomous robotic-assisted systems in KA are a typical example of an AI-based tool. This system integrates information from preoperative imaging or intraoperative surface mapping and specific bone landmarks (bone shape, tibial and femoral alignment), as well as from the ligament balancing intraoperatively. The collected data are included in algorithms for both bone and implant alignment and soft-tissue balance to propose surgical planning, secondarily adjusted according to the surgeon's requests and targets. Then, a robotic arm allows performing the bone resections or positioning a cutting guide, with a real-time automatic feedback system following the knee movements or the cuts progression. Progressively, the algorithms of the robotic system integrate ML models to improve surgical planning according to the previous surgeries.

Based on the results of the literature, the main benefit offered by robotics systems for KA is accurate and reproducible bone preparation thanks to the  robotic interface, regardless the system used. Most currently available robotic platforms allow an assessment of the ligament balancing according to the bone cuts and the implant positioning during the surgery. The aim of robotic systems is not to replace the surgeon but to be an accurate and consistent delivery tool. Current evidence shows advantages of robotic-assisted TKA in knee alignment, implant positioning, ligamentous balance, and soft tissue protection [17-20].

Robotic-assisted systems have, however, some limitations. In addition to the cost related to the capital investment and the consumables in the operating room, there is also an amount of education required for surgeons and staff to optimize the safety and efficiency of robotics. The specific hardware required for the use of the robotic system remains unpractical, with bony trackers and bulky robotic unit. The operative time may be longer, especially during the learning curve. A robotic system is usually compatible only with one type of implant. The laxity assessment at the beginning of the surgery is manual and thus lacks accuracy. Current research on the topic aim not only to prove the clinical benefits of these systems, but also to precisely evaluate the cost-efficiency of such systems.

Augmented and Mixed reality

Augmented reality-based navigation systems superimpose clinical information into the sight of the surgeon and have been developed to guide TKA implantation. AR platforms require three process’s, tracking, computing, and visualization. Tracking of object position is achieved by contact, semi-contact, and contactless methods. The semi-contact system requires attachment to the anatomy and marker and a contactless link between the features and the cameras. The marker’s 3D positions are used to triangulate the marker’s 2D position in each camera of the optical system. Contactless systems are still to be developed. Tracking is done without the need for attachment to the object or patient, and has been made possible with the apparition of depth cameras. Depth cameras are active sensors that project a structured light pattern onto a scene, creating a pattern that is used to reconstruct the 3D surface of the object. The computing requires two operations, the first operation to register the anatomical features tracked with the preoperative images, and the second to compute the clinical index from raw information, which compares the actual situation with the preoperative plan. Registration prevents errors occurring from adopting different positions during imaging and the operating room. The final step is visualization which produces an image for the user. In augmented reality, the digital image must align with surgeons’ reality. Augmented reality technology helps the surgeon focus attention on the patient by overlaying feedback information directly into the surgical field.

AR systems have recently been studied in TKA. The proposed advantages of AR platforms compared to the robotic-assisted systems are a  smaller physical footprint, a lower cost (no capital investment) the ability to have the intra-operative data in the same field of view, the absence of intraosseous trackers and an easier workflow [21]. The Pixee Medical system is a computer-assisted orthopaedic surgery solution using AR to support TKA (Pixee Medical, Besancon, France) (Figure 4).

The software installed in smart glasses is combined with reduced-size MIS instrumentation. The connected glasses precisely calculate the 3D coordinates of the instruments thanks to the analysis of their specific markers (QR-Code), filmed by the integrated camera. The navigation information is displayed in the surgeon’s field of vision, which interacts with the application thanks to the glasses’ accelerometers. The NextAR™ system (Medacta, Castel San Pietro, Switzerland), another AR-based navigation system for TKA, requires sensors to be anchored to the femur and tibia using pins inserted within the surgical wound. A preoperative plan is generated based on CT imaging and a dedicated algorithm used to identify ligament origin and insertion to monitor balance during intraoperative navigation. To date to our knowledge, no clinical studies have yet been published on the accuracy and the clinical efficiency of these novel devices. Despite a current important mediatization, it remains unclear whether these devices lead to improved patient outcomes and/or are cost-effective.

A.I. in the postoperative period

Postoperative remote patient monitoring

Owing to the increasing number of patients owning smartphones, the applicability of technologies for patient monitoring and communication has increased significantly. A remote patient monitoring (RPM) platform that uses wearable technology may holistically capture patients' status after TKA to provide continuous subjective and objective data (Figure 5).

The first platforms have been limited by the absence of interconnectivity between applications, poor user engagement, high cost of sensors and deployment, and inability to scale [22-24]. A machine learning-based RPM system using an open-source software development kit designed for smartphones has been developed. These devices allow for real-time tracking of patient participation in physical therapy and home exercise programs through the patient’s smartphone. A pilot study of 25 patients who underwent TKA has been validated demonstrating the ability of this technology to passively collect data from each patient’s smartphone without interruptions [25]. A recent randomized clinical trial on 242 patients operated of hip or knee arthroplasty found no significant difference in the rate of discharge to home between the usual care arm and the RPM arm, but a statistically significant reduction in rehospitalization rate in the RPM arm [26].

Predictive models and machine learning for postoperative outcomes

The increasing availability of large digital healthcare datasets facilitates the development of predictive models (PM) for postoperative outcomes after TKA. These PM examine how variables such as patient-specific attributes, functional scores and preoperative pain [27], comorbidities [28], psychological features [29], socioeconomic indicators [29], or perioperative recovery location influence clinical outcomes. The goal is to use these PM to estimate and predict the likelihood of improvements in function and satisfaction after TKA to support patient and surgeon decision-making [30]. Development of PM requires the choice of the correct input factors, and the selection of the outcomes that can be considered really  useful measures of clinical satisfaction [31, 32]. The main preoperative predictive factors described in the literature for the postoperative outcomes after TKA are pain scores (VAS and back pain), knee specific PROMs (such as KOOS and WOMAC), range of motion, quality of life PROMs (EQ-5D), and mental health (assessed by anxiety and depression scales and SF-12). Other factors that have also been evaluated are comorbidities (ASA score), demographic data (such as BMI, gender, age), previous knee surgery, severity of osteoarthritis, and preoperative knee alignment [33]. The data set construction is another challenge while using PM. In fact, the quality of the initial data set used to evaluate the post-operative outcomes based on the pre-operative parameters will determine the accuracy of the PM.  As such the initial data set should be representative of the patient population on which the PM will be used.  For this reason, PM using data from very large populations, including several centers or countries, and objective preoperative 3D anatomy assessment are more reliable than those built on limited data sets. To overcome these limitations, increasingly complex algorithmic approaches have been developed and are still developed. So far none of the available PM have been able to replicate surgeon clinical acumen or become a practical tool for clinical use yet and these PM models are still in their research/pre-clinical phase.

Limitations and future expectations

Although the technology behind AI is exponentially advancing, it has various limitations to bear in mind. Surgeons are currently key stake holders between industry and the patient and have an ethical and moral responsibility to ensure that the technology is used appropriately and to benefit the patient. For this reason, evidence must be critically analyzed. As with any scientific endeavor, the use of AI hinges on whether the correct scientific question is being asked and whether one has the appropriate data to answer that question. The outputs of ML and other AI analyses are limited by the types and accuracy of available data sets. Systematic biases in clinical data collection can affect the type of patterns AI recognizes or the predictions it may make. For example, this can affect women and racial minorities due to long-standing under-representation in a clinical trial and patient registry populations. Ethical considerations regarding the ownership and use of data in AI also remain unanswered. The robotic platform stores surgeon and patient information sometimes without the express consent of the patient, and this is then used for product development. Whilst aggregate data is deidentified, the question of who should have access to this data and for what purpose has still been debated. The European Commission has proposed a regulatory framework (released on April 2021) to monitor AI with this aim.

Despite its pitfalls and potential shortcomings, AI provides a unique ability to create meaningful change in TKA to optimize patient-specific surgical pathway.  AI is now being applied to TKA procedures with the use of robotic systems that execute plans with a high level of accuracy and repeatability (Figure 6).

The next challenge will be to “close the loop”, using accurate interconnected data sets and PM during the different phases of the patient path (preoperatively, intraoperatively and during follow-up) to help surgeons and health-care providers and support decision-making (Figure 7).

The goal is not to replace the health-care providers but to assist the medical decision in a collaborative manner, combining the doctor’s experience and the AI-based tools. The answer is probably towards a form of collaborative intelligence to adjust the patient management using predictive models and clinical experience and make the next surgery better for every patient.

Conclusion

Artificial intelligence is a rapidly expanding field in surgery. Its applications are multiple and can improve the decision-making process, the surgical planning, the accuracy and repeatability of surgical procedures, the postoperative follow-up, with adapted and personalized targets for each patient. Whilst the proposed benefits are many, and the possibilities can potentially improve patient outcomes, clinical evidence is still needed to confirm the interest of AI-based tool for KA.