In sporting environments, the knowledge necessary to manage athletes is built on information flows associated with player management processes. In current literature, there are limited case studies available to illustrate how such information flows are optimized. Hence, as the first step of an optimization project, this study aimed to evaluate the current state and the improvement opportunities in the player management information flow executed within the High-Performance Unit (HPU) at a professional rugby union club in England. Guided by a Business Process Management framework, elicitation of the current process architecture illustrated the existence of 18 process units and two core process value chains relating to player management. From the identified processes, the HPU management team prioritized 7 processes for optimization. In-depth details on the current state (As-Is) of the selected processes were extracted from semi-structured, interview-based process discovery and were modeled using Business Process Model and Notation (BPMN) and Decision Model and Notation (DMN) standards. Results were presented for current issues in the information flow of the daily training load management process, identified through a thematic analysis conducted on the data obtained mainly from focus group discussions with the main stakeholders  of the process. Specifically, the current state player management information flow in the HPU had issues relating to knowledge creation and process flexibility. Therefore, the results illustrate that requirements for information flow optimization within the considered environment exist in the transition from data to knowledge during the execution of player management decision-making processes. In professional team sports environments, practitioners execute different operational processes to manage players. Such processes include managing an athlete's physiological, psychological, technical, and tactical preparation and performance  of the player management information flow in the performance department of a professional rugby union environment. Specifically, the study aimed to identify and unravel solutions to the following set of questions: (1) in the performance department of a professional rugby union environment, what different processes are executed to manage the preparation and performance of the players? (2) in its current state, how does information flows within the identified processes? (3) are there any requirements to improve the current information flow of the identified processes?The present study evaluated the player management processes at a rugby union club competing in the Gallagher Premiership in England. During the 2019/2020 season, an organizational objective was set at the club to enhance the use of information for decision-making within the performance department or specifically referred to as the High-Performance Unit (HPU). Therefore, the current study is an outcome of specific management intent and presents the first part of the results obtained from a stepwise approach conducted to optimize the information requirements for player management processes in the HPU. As a business unit, the overall objective of the HPU was to maximize the availability of players in the squad to the rugby program at the club. At the time frame of the study, the HPU consisted of physiotherapists (5), strength and conditioning (S&C) coaches (3), sports scientists (1), doctors (1), medical administrators (1), and data scientists (1). Additionally, the head of medical, head of strength and conditioning, and head of applied sciences and research provided leadership to the HPU and will be referred to as the HPU management team throughout the article. Furthermore, the study was approved by the ethics committee of the affiliated university.In the current study, we adopted fundamentals from Business Process Management (BPM)  is collected and documented through process modeling techniques.Process analysis: Issues within the documented As-Is state of the processes are identified for optimisation, potentially generating a list of prevailing issues.Process redesign: The process is redesigned (optimized) to overcome the issues identified in the process analysis stage to define the best future state (To-Be).Process implementation: Necessary changes to move the process from the As-Is to the To-Be state is performed by managing organizational changes.Process monitoring: The implemented process is monitored to determine the effectiveness of the changes. The cycle is repeated to the discovery stage if further issues are present or further continuous improvements are necessary.The initial step to assessing the current state and identifying improvement opportunities within a player management information flow is to understand the different athlete management processes that occur in a specific environment. Technically, for the current study, this led to establishing the player management process architecture  by taking a participant as an observer stance , management, and support processes were formulated into a process landscape diagram to demonstrate the high-level process architecture of the HPU.The semantic quality  of the formulated process architecture was assessed by the HPU management team based on a 1-h meeting. The process units in the architecture comprised of processes generating an operational  or decision  outcome. From these two outcome types, the HPU management team decided to only concentrate on processes providing a decision outcome for identifying current issues in the information flow.Since every organization has limited time and resources, it is not possible to evaluate the information flow of all the identified player management processes in a single attempt. Therefore, it was necessary to select the highest priority processes that required consideration. To achieve the latter objective, the HPU management team subjectively rated each of the identified process units using the below criteria : To define the significance of each player management process to the current core strategy of the organization.Health (poor/good): Perspective on the current quality of the information flow associated with the considered process.Feasibility (low/medium/high): The feasibility of each process for change or optimisation. For example, if a process was associated with an information system that could not be altered  or subjected to organizational politics, the feasibility was rated as low.From the process ratings, processes having high strategic importance, with poor current information flow quality  and medium or high feasibility, were selected to be analyzed for current issues in the information flows associated with them.Once the set of player management processes in the HPU was identified and selected, it was possible to analyse in detail the current state (As-Is) of the information flow associated with those processes. The final goal of this stage was to derive the current process and decision-level models of the selected processes with clear indications of the information flow, specifically highlighting the points at which data, information, and knowledge were generated during process execution. The BPM literature suggests both automated and non-automated methods for process discovery Jadhav, . AutomatData collection: For data collection, practical guidelines available in the literature were utilized : The main individual(s) responsible for executing the identified player management process in the current context. Relevant SMEs for each process were defined by the sponsor with further feedback from other members of the HPU management team (specific details of the identified SMEs have been provided in the Results section).○ Analyst: Responsible for collecting, organizing, analyzing, and presenting information regarding the process. Author JR acted as the analyst. Verner, , and theThe analyst, with the presence of the sponsor, conducted semi-structured interviews (each <1 h) with the subject-matter experts  to colleCurrent state (As-Is) process models: Information extracted from the process discovery interviews were translated into process models using collaboration and process diagrams in Business Process Model and Notation (BPMN), guided by the syntactic rules defined in the literature  explained the structural and behavioral rules used in the models to the relevant SMEs.Once the current state (As-Is) of the identified and prioritized processes in the HPU were modeled, it was important to identify the specific issues in the current information flow associated with those processes. Resolution of such issues will act as the basis for future optimisation tasks relating to the HPU information flow. The BPM literature provides different qualitative and quantitative approaches to process analysis  were conducted with the key stakeholder groups  of the identified processes to elicit data on existing issues within the information flow of the discovered processes during player management. Participant details of those focus groups have been provided in the Results section .Are there any issues with the data/information sources required for decision-making during the execution of the considered process?Are there any issues associated with transforming the available data to knowledge during the execution of the considered process (helped to determine noise in decision-making)?Author JR acted as the moderator of these group discussions. Each session was conducted with the As-Is state model of the process as a reference material to guide the information elicitation process. Therefore, at the beginning of each session, author JR explained the BPMN diagrams (including the syntactic rules of BPMN) of the considered process to the participants.Focus group data were collected as audio files for in-person sessions and video recordings for any virtual sessions. Thematic analysis was used to analyse the focus group data and was targeted at unraveling information on current issues within the information flow of the considered processes. This latter objective was achieved by using the six-phase thematic analysis method  identified in the HPU was illustrated using the process landscape model depicted in From the previous steps, 16 core and two management processes generating a decision outcome were identified for deeper investigation . From thFrom the management-rated process list, the process portfolio diagram illustrated in For the selected 7 player management processes, the sponsor (head of applied science and research) defined one other member of the HPU management team as the subject-matter expert (SME) of the weekly training load management (M1), daily training load management (M2), rugby conditioning design (C5), and rugby conditioning evaluation (C8) processes. Additionally, two physiotherapists were defined as the SMEs of acute health assessment (C1), rehab program planning (C14), and injury legacy (C16) processes refer to . This reFor illustration, the article will elaborate on the results obtained from evaluating the information flow of the daily training load management (M2) process. It was the most critical process occurring within the HPU since it generated the daily training recommendation for each player in the squad. Process sports science knowledge activity in a red color background). The core outcome of this decision was to provide knowledge relating to the training loads experienced by the players, which was taken into consideration when generating the final daily training recommendation as an outcome of the M2 process.Additionally, To analyse the improvement opportunities in the current state of the daily training load management (M2) process information flow, three focus group discussion sessions were conducted with the main stakeholder groups of the M2 process. First, at the onset of each focus group discussion, the relevant stakeholders verified the semantic quality of the As-Is process models. The initial model of the M2 process illustrated in Thematic analysis of the data collected from the focus groups to analyse the As-Is state of the daily training load management (M2) process yielded issues in the current information flow pertaining to two key themes, namely knowledge creation and process flexibility. Each of the main themes contained sub-themes, which highlighted specific issues in the current information flow associated with the M2 process.Knowledge creation: This theme focused on identifying current issues in the M2 process when generating knowledge from data, information, and evidence. Specifically, it highlighted the current issues associated with the information flow when strength and conditioning coaches, physiotherapists, sports scientists, and doctors accessed data sources and transitioned them to knowledge to generate a decision pertaining to player management . As specified in the following text, this theme consisted of five sub-themes that illustrated different current issues in the information flow.Inaccessible data: Although certain data sources of players such as resistance training and baseline testing data were collected during training, they were not accessible to the practitioners in certain instances. For example, a strength and conditioning coach stated the following regarding resistance training data:○ If I am sat at home in an evening and you get some information come through and then you want to cross reference it from er from er what they done in a gym perspective all the information (player resistance data) is either on a folder here (training center) or on a whiteboard here (training center)”“This latter inaccessibility of daily player resistance training data was evident from the current state data collection BPMN process diagram depicted in Unavailable data: This sub-theme highlighted the unavailability of certain player data parameters for decision-making. However, HPU practitioners were aware that those data sources were more “nice-to-have,” rather than being highly critical for managing players in the current context. As stated by a member of the HPU management team,○ I've always found S&C monitoring fascinating you know and then I think you can get as much from interpreting most of the humans as there is something there you know that's the big outcome fascinated by that you know whether that be through erm objective measurement of er jump height or er erm bar velocity or et cetera et cetera”“Information noise: This issue sub-theme mainly focused on HPU stakeholders specifying that the information inputs into the daily training load management (M2) decision process (e.g. rugby session plan) may be incomplete, inconsistent, or late. According to HPU staff,○ Having the session plan come through from the coaches that's probably one that I think is still not quite getting right consistently in terms of like having the information available it's like concrete in terms of session plan is quite few times where it's all like you know this is we talk through the session in theory what it is and then it's not that”“However, HPU practitioners were mindful of the fact that the output from the M2 process (player training list recommendation) may be necessary for coaches to create a finalized session plan. Therefore, the sub-theme further highlighted the issue associated with the improper synchronization of daily training load management (M2) and rugby conditioning design (C5) processes. From the HPU management team's perspective,The session plan that comes through and yet that has that session plan hasn't been practical to adjust to who is available for the session it can have designed the session which is it's not gonna work because the information hasn't been fed up on the availability (player training recommendations generated from M2 process)”“Unavailable information: Specifies the unavailability of weekly strategic objectives defined from rugby coaches' perspectives to guide HPU practice and macro-level strategic individual player development goals and targets. For example,○ I think the retrieval of information from to understand rugby context better whether it's acutely with a session plan or acutely with what bodies you need on deck and what points in the in the rugby training session it needs certain people to longer term vs. er er longer term picture of where the person sits in the depth what's their development plan where where do the coaches see them needing more or less input that's probably information that's hard for us to currently receive”“HPU flagging consensus: The management team of the HPU highlighted the importance of having a consensus among the practitioners on the different flags (notifications) generated during player management. Furthermore, the specific data/information input conditions associated with those flags were also not currently available. This illustrated the existence of noise in decision-making relating to the flagging of players in the current context.○ That we actually need to come to some sort of HPU consensus on this side of stuff (flagging) rather than me just making the decisions the decision will be made potentially by me but it must be made using the consensus of the HPU as to what we believe is right”“Process flexibility: This major theme specified issues in the current information flow associated with the dynamic and unpredictable nature of player management processes. Specifically, a major sub-theme was the issues in the current information flow associated with incomplete decisions pertaining to player management.Incomplete decisions: The current information architecture does not contain a concise information flow to manage scenarios of HPU practitioners being unable to make concrete decisions about the daily training categories of a player within the M2 process. The instances where HPU staff required additional time to observe a player in the morning to recommend his ideal daily training category are not well-supported with an information flow.○ We  erm suggest that more time may be needed to see how that clinical entity addresses one way or another and then we apply our knowledge and say well we need more time for that so that it does it it's not until one o'clock we've actually made a sound solid decision on what we're gonna do with that guy”“Given the nature of our game we always have a few guys who are hanging on (incomplete decision from M2 process) and rightly so medical team just gives them more time to see how they try to train”“The present study aimed to evaluate the current state of the information flow of player management processes occurring within the High-Performance Unit (HPU) at a professional rugby union club in England. The process architecture of the HPU consisted of two core process value chains (performance management and return to play) in normal operating conditions. On the specific level, 18 processes (core and management) that generated a decision outcome were executed to manage the players. From the list of identified processes, the HPU management team prioritized 7 processes for analyzing the current state of their information flows. Semi-structured interviews with the subject-matter experts of those processes yielded the As-Is state BPMN process models –7 that hThe derived value-chain of the player management process architecture in the HPU was based on the normal operating conditions during the in-season of the Gallagher Premiership. Hence, the identified process value chain during the week was shaped toward a game scheduled during the weekend and was linked based on a daily operational sequence . Although the pre-season value chains within a week would still resemble the process landscape model depicted in In the current case study, processes for discovering the micro-level information flow were selected subjectively based on the management perspective. However, if KPIs of processes that relate to an information flow  and specific thresholds of poor performance of those KPIs were available in a considered environment, practitioners could use such measures to objectively select processes for exploring current issues in their information flows based the KPI values.Interview-based process discovery was well-suited to unravel in-depth details on the current state of the identified player management processes due to two main reasons: (1) The nominated SMEs had vast knowledge of the current state of the selected processes since they were the main individuals responsible for executing them. (2) Other methods like automatic process discovery  were not suitable in the current rugby union environments since player management processes were executed based on human intuition, rather than system automation. However, in most situations, the SMEs were inclined to provide knowledge of the normal operating situation of a process. Therefore, as the analyst, author JR had to specifically discuss abnormal process scenarios to gather details on how the SMEs executed the processes during unconventional situations.The As-Is state process models developed from BPMN and decision models from DMN standards clearly illustrated how data and information flowed through the considered decision processes. Specifically, process models with swim lanes  corresponding to each stakeholder of the process demonstrated how different individuals interacted with the information and highlighted the specific points at which knowledge was generated during process execution. Additionally, having the As-Is state of the process as a model greatly supported the process analysis focus group discussions since the stakeholders were able to refer to it when elaborating on current state issues, rather than building verbalized visuals for the participants. This latter aspect was achievable since the syntactic representations of BPMN models were easily understood by the participants.However, it must be stated that process models like those derived from standards like BPMN and DMN are typically used to optimize automated processes. This is because normally, such workflow models are directly transformed to executable form and implemented as automation using information technology systems like Business Process Management Systems (BPMS). Whilst that is the common case, it does not limit process modeling techniques or BPM as an overall framework to be used to model or optimize just automated processes. Instead, as supported by the definition of BPM by the Association of Business Process Management Professionals (ABPMP), BPM as a change management tool and its inherited modeling techniques can be equally used to optimize non-automated processes as well ABPMP, . The resIn reference to the daily training load management (M2) process, daily resistance training and baseline testing data were the major inaccessible data sources for decision-making. Apart from the latter two data sources, the thematic analysis further illustrated that HPU practitioners were generally content with the currently available raw data sources. The practitioners had a positive interaction with GPS data and were readily accessible to HPU staff during decision-making. Whilst GPS data were optimized in terms of availability, there is still space to explore the state of other information quality dimensions  pertaining to GPS-based measurements. However, the analysis of processes with more coaching interaction like rugby conditioning design (C5) highlighted three data metrics that strongly aligned with the coaching/game strategy of the club and were currently unavailable. Those latter data parameters were not discussed in the article due to intellectual property restrictions.Moreover, the identified issues in the current state of information flow illustrated that the fundamental needs for optimisation within the considered case study environment existed in the transitioning from data to information  and information to evidence . Resultantly, from an optimisation perspective, typical digitization endeavors like data architecture development  , Carnegie School of Sport, Leeds Beckett University. The patients/participants provided their written informed consent to participate in this study.JR, DW, and GR conceptualized the study design and contributed to writing and editing the manuscript. JR, MZ, and GR participated in data collection. JR performed the initial data analysis. DW, MZ, and GR validated the resulting outcomes. All authors have read and approved the final version of the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


This paper proposes a tracking method combining feature enhancement and template update, aiming to solve the problems of existing trackers lacking global information attention, weak feature characterization ability, and not being well adapted to the changing appearance of the target. Pre-extracted features are enhanced in context and on channels through a feature enhancement network consisting of channel attention and transformer architectures. The enhanced feature information is input into classification and regression networks to achieve the final target state estimation. At the same time, the template update strategy is introduced to update the sample template judiciously. Experimental results show that the proposed tracking method exhibits good tracking performance on the OTB100, LaSOT, and GOT-10k benchmark datasets.  As one of the challenging and significant research tasks in the field of computer vision, object tracking is the tracking of the position of the target in each frame by using the features of the initial frame in successive video frames in the subsequent frames. Although object tracking algorithms have achieved good results in recent years, the difficulties in the tracking process, such as illumination variations, target appearance variations, occlusion and disappearance, still make object tracking challenging research. We systematically analyzed the literature ,2 for thCurrently, most attention is devoted to tracking using the matching ability of the Siamese network. SiamFC  uses a SSiamRPN  begins wDaSiamRPN  enrichesThe attention mechanism can effectively suppress the influence of background interference and target variations on the tracker. To improve the feature representation without introducing too many uncontrollable hyperparameters, researchers have shifted the attention mechanism to the tracking domain. SASiam  construcSince self-attention can establish long-term dependencies between features. TrTr  used a tSome classical tracking algorithms consider channel information alone, lacking attention to context information and yielding inadequate feature representation capabilities. Inspired by references ,19,20,21In the tracking process, the tracking algorithm should be able to adapt to the appearance change of the target in time. TA-Siam  designs GradNet  proposedThe related research work provides useful ideas for this paper to design feature enhancement networks combining channel attention and transformer models based on the architecture of Siamese networks, as well as being conducive to the paper to design a reasonable and effective model update strategy.In this paper, a feature enhancement module is designed to enhance the spatial–temporal and channel-wise saliency of the features extracted from the benchmark network, which can make the tracker automatically focus on the beneficial feature information and improve the feature characterization ability.In this paper, a template update strategy is introduced. By dynamically updating the target template, the impact of target appearance changes on the tracker is better mitigated.In this paper, the proposed tracking algorithm achieves state-of-the-art tracking performance on three challenging benchmarks, OTB100, LaSOT, and GOT-10k.In general, the main contributions of this work can be summarized as:The remaining contents of this paper are organized as follows. In φ (AlexNet). φ(.) refers to the AlexNet convolutional neural network. The fully convolutional Siamese network has two same network branches, which can be used to measure the similarity of two inputs and is suitable for the object tracking task. As a typical, full-convolution Siamese network tracking algorithm, SiamFC, the upper branch is the target template branch, and the lower branch is the search branch. The template image z and the search image x are sent to the upper and lower branch, respectively. The feature vectors b represents the bias, is the one-dimensional vector, and * represents the convolution operation.After obtaining two feature vectors by a fully convolutional Siamese network, we calculate the similarity of the two feature vectors. We use the sliding window to obtain the final response graph. The higher response is considered the location of the target. This method is simple and fast and specifically expressed as:This tracking algorithm uses a shallow network for feature extraction. Due to the small number of convolutional layers, it becomes difficult to extract deep semantic information. Therefore, it cannot take full advantage of the representation capability of a deep convolutional neural network. Meanwhile, the matching template is always the sample template of the initial frame during the tracking process. It cannot accurately adapt to the change of the target in real-time, which is not conducive to the robustness of the tracking algorithm. Moreover, the simple calculation of the distance between feature vectors to determine the similarity of targets has low accuracy.To achieve better tracking performance, this paper uses deep ResNet50 to extract deeper semantic information. Combining shallow information, channel attention, and transformer architecture, a feature enhancement network to improve feature representation and introduce template updates to adapt to changes in target appearance was proposed. Simultaneously, according to the characteristics of the upper and lower branches of the Siamese network, the tracking task is regarded as two sub-tasks of classification and regression. The classification branch separates the target from the background and the regression branch accurately obtains the bounding box of the target, and locates the tracking target accurately by combining the characteristics of both.Based on the SiamFC object tracking algorithm, an object tracking method combining feature enhancement and template update was proposed by dividing the tracking task into two branches, classification, and regression, according to the network structure characteristics of the Siamese network. Specifically, deep semantic features with ResNet50 to extract, and deep semantic features and shallow features for fusion after passing them through channel attention (attn), and the fused features were respectively fused through two encoders to construct the long-term dependence between features. Cross-attention in the transformer decoder was used to exchange information between the features of the two branches, highlight useful global context information and channel information, suppress the interference of similarity targets, and improve the ability of feature representation. In the meantime, a real-time template update strategy was introduced to alleviate the influence of target appearance changes and improve the robustness of the tracker. The overall framework is shown in ResNet50 network is a benchmark network for feature extraction as used. Compared with the original AlexNet network, the benchmark network can extract deeper semantic information and improve the ability of target judgment and interpretation. For the input to the network, a pair of images selected from the video frames of the training dataset, namely, the template image and search image, feed into the Siamese network architecture and obtain the desired features through the benchmark network.We obtained feature maps with increased representational ability by constructing feature enhancement networks in order to retain more significant target features during tracking and strengthen the correlation between them. The feature enhancement network includes a feature fusion part based on a channel attention mechanism and a long-term dependency establishment part of the transformer.Since the convolution operation only takes place in the local space, it is rare to gain enough information to extract the relationships between channels. The channel attention module calculates the weights of each channel of the input image. Then it fuses the shallow and deep features of the channel attention, focusing on the channel containing critical information and enhancing the feature representation ability.Template images and search images are passed through the ResNet50 convolutional neural network to obtain the features of the last two stages. Then, they are passed through the channel attention mechanism separately to calculate the channel weight coefficients. The original feature maps are corrected using the resulting channel weight coefficients to obtain the enhanced attention feature maps. Finally, the features of the two stages are fused, where the feature fusion module is up-sampling with bilinear interpolation to unify the features dimension. The fusion process of shallow features and deep features is shown in The template image and the image go through the four stages of ResNet50. Then the feature vectors δ and σ represent ReLU and Sigmoid activation functions respectively, Firstly, the spatial dimension of the input The traditional transformer includes an encoder and decoder. The encoder includes two sub-layers, a self-attention module, and forward propagation. The self-attention module serves to learn the internal relationship between features. It can effectively capture the internal correlation between data and features. The decoder consists of three sub-layers, a self-attention module, an interactive attention module, and forward propagation. The self-attention module and forward propagation have the same function as the modules in the encoder. The interactive attention module works to learn the relationship between the original feature and the target feature.Q), key(K), and value(V) based on the embedded feature vectors and calculates the similarity or association between them based on Q and K. In this paper, we selected to calculate the similarity using the dot product of two vectors, and the obtained scores as normalized. Then the weighted sum based on the normalized weight coefficients are applied to V. The calculation of self-attention can be defined as for Formula (10).Q, K and V are linear transformations derived from features.Self-attention gives query update their images to the position of the initial frame. This method can make full use of the temporal contextual information of the tracking process and alleviate the problem of changing target appearance. Where FC1, FC2 and FC3 denote fully connected layers, the classification branch-based template update strategy is shown in During the implementation of the whole tracking algorithm, offline training was carried out using the publicly available datasets LaSOT , GOT-10kAlgorithm 1 Procedure of the proposed methodrepeatuntil end of sequenceIn order to validate the proposed tracking method, the model was initialized using the parameters pre-trained on ImageNet . LaSOT , GOT-10kTo quantitatively analyze the overall performance of the tracker under the interference of complex background and other factors, the proposed method was compared with typical tracking algorithms on OTB100 , GOT-10kThe OTB100 dataset contains 11 tracking challenge factors, 98 videos, and 100 test scenarios, enabling fair and accurate evaluation and comparison of various tracking algorithms. In this paper, the compared method was evaluated with the one-pass evaluation (OPE), and two main metrics of precision plots and success rate plots were adopted to evaluate the performance.The trackers compared on the OTB100 dataset included SiamFC , SiamRPNAs can be seen from CFNet combines correlation filtering with CNN, using a correlation filtering layer to implement online update function, which allows the network to perform well with shallow features. The performance was still not as good as the state-of-the-art trackers due to the boundary effects that CF has difficulty handling.SiamRPN algorithm uses shallow AlexNet for feature extraction, but it cannot extract deep semantic features and ignores the context information. The tracking algorithm proposed in this paper uses ResNet50 to extract deep semantic features. In addition, the proposed feature enhancement network could effectively improve the feature representation ability. Compared with SiamRPN, the precision and success rate of the proposed method in this paper are improved. The accuracy of the GradNet algorithm exceeded that of the tracking algorithm proposed in this paper. However, the success rate was slightly lower than the algorithm proposed. It indicates that the proposed tracking algorithm could adapt the bounding box to the target size well when tracking the target. Experimental results show that this method is superior to some classical tracking algorithms in precision and success rate.Further to From As observed in The LaSOT dataset contains 1400 videos with an average of 2512 frames per sequence. Each frame is carefully inspected and manually labeled, with results visually verified and corrected when needed, representing by far the largest single-target tracking dataset with dense annotation for depth tracking training and realistic evaluation of tracking algorithms. We used two evaluation metrics, the normalized precision plots The trackers compared on the LaSOT dataset include SiamBAN , SiamRPNWe can see this in The biggest contribution of SiamRPN++ is to change the network structure of twin trackers, using a deeper network instead of a shallow network. The main drawback is that there is no template update, and it is not possible to change the template adaptively when the target appearance changes. Furthermore, the custom anchor frame setting introduces a fuzzy similarity score, which severely hinders robustness and leads to a lower performance compared to advanced trackers.SiamBAN is an anchor-free based object tracking algorithm that may not contain sufficient target information for accurate regression. Moreover, it lacks the flexibility to handle target changes because it does not consider the internal relationship between features and uses only the initial frame as a template. Therefore, the results are not very effective on the long-time dataset LaSOT.SiamMask adds a segmentation branch alongside the classification branch and the regression branch, which causes the whole network to be difficult to train. Although ResNet50 is used as the network for extracting features, there is still much room for improvement in robustness.The SiamDW tracking algorithm ignores changes in the target motion state, resulting in tracking failure when challenging factors such as rapid target motion and occlusion occur.In this paper, the proposed feature enhancement network not only considered the key information on the channel, but also used the self-attention mechanism to establish long-term dependencies between features and improve the correlation of features, and the template update strategy could effectively adapt to the changes of target appearance. Therefore, the performance was slightly higher on long-time datasets LaSOT compared to trackers such as SiamBAN, SiamRPN++, SiamMask, and SiamDW.Specifically, the compared performance evaluation results of the proposed method and typical template update trackers on LaSOT dataset are given in The method used only the feature enhancement network proposed in this paper before introducing the template update strategy. The performance was over 5.3% compared to the TrTr-offline tracking algorithm that also uses the transformer architecture. This result shows that the features in the upper and lower branch were fused separately after passing through the channel attention. Then, the association between the features within each branch was performed by the encoder. Finally, the information from the upper and lower branch interacted through the decoder. The performance was improved compared to the decoder process where the upper branch is passed through one encoder and then inputs it to the lower branch.The performance of TrTr-online tracking algorithm exceeded the tracking algorithm proposed in this paper. It is a combination of classification plus an online update method, which is slower. However, the tracking algorithm in this paper used a simpler linear update mechanism. The real-time performance of the tracking algorithm was preserved with a slight performance improvement.UpdateNet-DaSiamPRN and UpdateNet-SiamFC were tested by using UpdateNet network on the original DaSiamRPN and the base network SiamFC. However, the performance was lower compared with the tracking algorithm proposed in this paper that combines the transformer and simple update mechanism strategies. This reflects the effectiveness of the tracker proposed in this paper.The DSiam tracking algorithm learns an online update transformation in two separate branches to update the tracking model through a fast transformation module. However, the tracking method lacks the detection of tracking failures, resulting in low performance.The ECO tracking algorithm updates the model every few frames to avoid model overfitting. However, its adaptability is reduced due to the underutilization of information during model updates.As can be seen from The GOT-10k dataset contains more than 10,000 real-world video clips and more than 560 categories, with more than 1.5 million manually marked frames, featuring rich scenes and difficult algorithm challenges. Average overlap (AO) and success rate (SR) are two evaluation metrics for the GOT-10k dataset, and ATOM refines multiple initial bounding boxes iteratively, which provides a significant improvement in tracking accuracy. However, the performance was slightly lower than the tracking algorithm proposed in this paper. Meanwhile, the method of target estimation in ATOM not only brings a heavy computational burden but also presents many additional hyperparameters that require careful tuning.The SPM tracker distinguishes targets from backgrounds by introducing additional relational networks. However, the use of a matching mechanism only and the lack of online updates severely limit the performance of this tracker in dealing with similarity interferers.THOR is a tracking algorithm based on dynamically updated target templates. The tracker uses greedy search in a local search window, resulting in a less than friendly performance in challenging benchmark tests.GOTURN uses a fully connected network to estimate corner coordinates directly, while objects in the background also have corner points. Furthermore, the relationship between the template and the test image is not effectively probing. Therefore, it is not easy to distinguish them from the corner points of the tracked target. To some extent, it affects the performance of the tracker.The MDNet tracking algorithm does not incorporate any feature enhancement operation during feature extraction. Moreover, fine-tuning online makes it slower and requires time and effort to fine-tune. Therefore, the robustness of the tracking performance needs to be improved.As can be seen in To further verify the model performance, the qualitative performance of the designed method and eight typical tracking algorithms were evaluated on the OTB benchmark, where the comparisons included SiamFC , SiamRPNAs shown in As observed in As viewed in As evidenced in This paper proposes a tracking method combining feature enhancement and template update, which uses ResNet50 as a benchmark network for feature pre-extraction of template images and search images. The feature vectors obtained from it are fed into the proposed feature enhancement network, and the feature enhancement module consists of a channel attention mechanism and a reconfigured transformer. After passing the channel attention mechanism, the shallow features fuse with the deep features. Then, the internal relationships between features are established by encoders. A decoder is used to carry out the interaction of information with the output of the encoder. It enables the network to select the key information effectively and improve the feature representation.Thanks to the introduction of a dynamic template update strategy that allows adaptive template updates when the appearance of the target changes and, simultaneously, enriches the temporal-context information between video frames, the robustness of the tracker is improved. The experimental comparison results show that the proposed tracking algorithm improved the tracking accuracy and success rate, and outperformed some existing classical trackers.


Human motion analysis using inertial measurement units (IMUs) has recently been shown to provide accuracy similar to the gold standard, optical motion capture, but at lower costs and while being less restrictive and time-consuming. However, IMU-based motion analysis requires precise knowledge of the orientations in which the sensors are attached to the body segments. This knowledge is commonly obtained via time-consuming and error-prone anatomical calibration based on precisely defined poses or motions. In the present work, we propose a self-calibrating approach for magnetometer-free joint angle tracking that is suitable for joints with two degrees of freedom (DoF), such as the elbow, ankle, and metacarpophalangeal finger joints. The proposed methods exploit kinematic constraints in the angular rates and the relative orientations to simultaneously identify the joint axes and the heading offset. The experimental evaluation shows that the proposed methods are able to estimate plausible and consistent joint axes from just ten seconds of arbitrary elbow joint motion. Comparison with optical motion capture shows that the proposed methods yield joint angles with similar accuracy as a conventional IMU-based method while being much less restrictive. Therefore, the proposed methods improve the practical usability of IMU-based motion tracking in many clinical and biomedical applications. Marker-based optical motion capture (OMC) is considered the gold standard for human motion analysis. However, this method is time-consuming and confined to expensive laboratory environments. Ambulatory real-time motion analysis can be achieved at much lower costs with inertial measurement units (IMUs). Recent studies have shown that the accuracy of IMU-based motion analysis is comparable to marker-based OMC, see, e.g., ,2.However, in order to derive anatomically meaningful kinematic quantities, for example, joint angles, the orientation of each IMU with respect to its body segment must be known, as illustrated in anatomical calibration or sensor-to-segment calibration, which is not to be confused with sensor calibration. Sensor calibration determines parameters such as scaling and bias in order to increase the accuracy of the sensor orientation estimates. Anatomical calibration determines how the sensors are attached to the body segments to ensure that the rotation axes used for calculating joint angles match the anatomical axes of joint rotation.An alternative is to include a procedure that determines the orientation of each IMU with respect to its body segment based on data measured by the sensors. This procedure is called As detailed in Use two kinematic constraints for 2-DoF joints, one that must be fulfilled by the angular rates  and aDo not make use of magnetometer measurements and are therefore insensitive to magnetic disturbances Instead simultaneously estimate the heading offset to facilitate magnetometer-free joint angle tracking.The methods are evaluated based on two experiments. The first experiment, with a known sensor attachment as ground truth, compares a simple and a complex motion and is used to show that estimation over a short time window of just ten seconds of joint motion yields plausible and consistent joint axes. The second experiment, with OMC as ground truth, is used to validate that, while being much less restrictive, the proposed self-calibrating joint angle tracking provides the same accuracy as a conventional IMU-based approach.Anatomical calibration is the task of determining how the IMUs are attached to the body segments. In a broader sense, this also encompasses the pairing of IMUs to body segments ,9 and thassumed alignment),Relying on a precisely defined sensor attachment (augmented data),Calibration via measurements from additional devices ,Calibration based on precisely defined poses or motions .Calibration from arbitrary motions  hinge joints ,11. FurtFor anatomical calibration based on arbitrary motions of 2-DoF joints, the existing work is limited. Müller et al.  introducThe methods proposed in the present contribution perform automatic anatomical calibration for joints with two degrees of freedom (DoF). Those methods are suitable for any 2-DoF joint and can be applied to a range of biomechanical or robotic 2-DoF joints. To improve comprehensibility, the following description of the kinematic model and the calibration method focuses on the human elbow joint as an exemplary joint, which is later also used in the experimental evaluation.Furthermore, even though in the following we always only consider two body segments connected by a single joint, the proposed methods can be used to analzye longer kinematic chains consisting of multiple segments. In this case, the calibration methods can be applied to each pair of segments that are connected by a 2-DoF joint.As an approximation, we can model this joint—as well as any other 2-DoF joint—as a kinematic chain consisting of two hinge joints and one fixed rotation in between, as depicted in We use unit quaternions to denote rotations and orientations . In the We can use this notation to mathematically express the orientation of the forearm The International Society of Biomechanics (ISB)  also recInstead of using regular Euler angles, we could consider modeling a 2-DoF joint with axes that are all potentially non-orthogonal (including the carrying angle axis). However, as ndations  does notTwo IMUs In the following, we derive two different kinematic constraints for 2-DoF joints, one based on the joint rotation and one based on the relative segment orientations. Both constraints are suitable for 6D sensor fusion with unknown heading offset. Given a short sequence of recorded IMU data, we use the Gauss–Newton algorithm to determine the joint axes coordinates in the sensor frame and the heading offset that best fit either the rotation-based or the orientation-based constraint in a least-squares sense. We use these joint axes coordinates to determine segment orientations from the sensor orientations and use the heading offset to align the reference frames of the orientations. From this result, we calculate the relative segment orientation, which we then decompose via Euler angles to obtain magnetometer-free estimates of the joint angles.As shown in Using the addition theorem for angular velocities, we express the relationship between the gyroscope measurements The common rotation of the whole arm, also observed by IMU The FE rotation around The PS rotation around The scalars Note that the carrying angle does not appear, since it is time-invariant. Furthermore, note that in , the angFor hinge joints, in , the folSince this version of the constraint only uses quantities given in local sensor coordinates, it is independent of sensor orientations with respect to a fixed frame and thus not affected by magnetic disturbances.For joints with two degrees of freedom, we need to know the relative sensor orientation or sensor orientations with respect to a common fixed frame. In order to derive a similar constraint from  for 2-DoNote that normalizing the axis was found to improve robustness compared to the constraint presented in .For perfect 2-DoF joints and ideal IMU measurements, this constraint must be fulfilled for each sampling instant. For biological joints, and when taking soft tissue motion and measurement errors into account, the constraint is still valid in a least-squares sense when considering a short motion sequence consisting of multiple samples.z-axis, i.e.,However, the constraint as formulated in  uses theisturbed , we wantisturbed . This imThe heading offset ope bias . Please We take the heading offset into account by evaluating the constraint  in one oThis version of the constraint implicitly depends on stead of  we can ustead of  with magAs an alternative, we derive a second kinematic joint constraint. In contrast to the constraint introduced in the previous section, this constraint is not based on the joint rotation but on the joint orientation, i.e., the relative orientation between the two body segments connected by the joint.As in lgorithm . As befoMathematically, we can formulate this by calculating the joint orientation and then decomposing this orientation into Euler angles. First, we determine the shortest-possible rotations that align the estimated sensor axes with the joint axes: Using those quaternions we calculate the joint orientationTherefore, Due to the joint constraint, this angle has to be constant over the whole measurement window, i.e., with the fixed constant carrying angle Similar to , the conThe aim of the anatomical calibration is to identify the joint axes With the parametrization given in , ∂ji∂φi=Note that both spherical parametrizations represent exactly the same 3D vector. Therefore, changing the parametrization in between optimization iterations does not influence the joint axis vectors or the value of the cost function but ensures that the derivatives with respect to the joint axes are always sufficiently sensitive.M data setsSample selection is performed to fill a sample buffer of In order to determine the joint axes and heading offset that best satisfy the rotation-based constraint  in a leaSimilarly, for the orientation-based constraint , we defiM samples, we find the parameter vector To estimate the joint axes lgorithm . AppendiUsing the optimization results, we calculate FE and PS joint angles based on the ISB recommendations . Those jfined in .From 6D inertial orientation estimation, we obtain the sensor orientation quaternions First, we calculate E1E2 via  and use fined in :(19)qS1BUsing those, we calculate the relative segment orientationFor any quaternion By calculating rding to  by a conWe can eliminate this offset by exploiting knowledge of the actual joint angles e β0 cf. , which iTo further improve the proposed method, in We evaluate the proposed magnetometer-free anatomical calibration and joint angle calculation methods based on two experiments.The first experiment is designed to evaluate if the obtained joint axis estimates are plausible and consistent. To this end, IMU data from two different motions are recorded from five subjects and a mechanical joint, while carefully attaching the sensors in a known orientation. Each trial is split into overlapping time windows to which the anatomical calibration methods are applied. The obtained joint axis estimates are compared to the axes obtained by the more restrictive method of careful manual sensor placement.The second experiment is designed for the evaluation of the accuracy of the obtained joint angles with the proposed self-calibrating magnetometer-free joint angle calculation method. This experiment consists of recordings of natural everyday life motions of two subjects. It uses marker-based OMC as a reference, which allows for the comparison of the obtained joint angles to joint angles obtained from optical markers and from a conventional 9D IMU-based approach. As a further validation step, we consider the variability of the expected-to-be-constant carrying angle as a metric for how well the estimated joint axes describe the functional joint motion.Note that in all experiments, the sensors are carefully attached in a known orientation to facilitate a plausibility check of the obtained results. To still verify that the proposed methods do not make assumptions regarding the sensor orientation, we simulate a random sensor attachment by multiplying all gyroscope and accelerometer measurements with a random rotation matrix that is different for each time window.The extension for on-chip sensor fusion introduced in Are the estimated joint axes plausible, i.e., do they agree with the values expected based on careful manual placement?Are the estimated joint axes consistent, i.e., do we always obtain the same result when using different parts of the trial?The first experiment is performed to answer the following two research questions:Data from five healthy subjects is recorded. The subjects are adult volunteers with no history of upper-limb injury that might affect upper-limb movement. Inertial sensors  are placed on the upper arm close to the elbow and on the forearm close to the wrist. The sensors are placed in a defined orientation on the skin so that one local sensor axis coincides roughly with the functional joint axis.simple motion consists of FE of the elbow and PS of the forearm, performed alternatingly while keeping the longitudinal axes of upper arm and forearm parallel to the sagittal plane.The complex motion, we ask the subject to perform random combinations of FE and PS, allowing for 3D rotation of the shoulder including humeral rotation.For the We define two different motions:Each subject performs both motions for approximately one minute.In addition to the five human subjects, an additional data set is recorded using a mechanical joint. This joint has dimensions similar to the human arm and consists of two hinge joints as shown in w, For each recording, the proposed methods are used on 21 partially overlapping moving windows w being the index for the estimation window. For a compact notation, we omit the segment index, denoting whether the axis is a flexion and extension (FE) axis or a pronation and supination (PS) axis. To assess if the estimates are consistent, we define the variability angle∢ denotes the positive angle between two 3D vectors andw and the mean of all estimates. If this angle is always small, the estimation results agree well for all time windows.Since the only available ground truth are approximate axis coordinates that we know due to the orientation in which the sensor was attached, we define suitable evaluation metrics that allow us to quantify the consistency and plausibility of the estimates. See misalignment angleTo also check if this result is plausible, we introduce the The results obtained with the mechanical joint agree very well with the expected axes  while the FE axis deviations are larger than for all other subjects. In This correlation can be explained when considering the kinematic constraint in  for the In summary, the evaluation of the first experiment has shown that the proposed methods yield consistent and plausible joint axis estimates. The rotation-based constraint performs better than the orientation-based constraint. To ensure that the axes converge, the subject’s motion should include sufficient motion from both the upper arm and the forearm.To further enrich the evaluation, we use the data from the first experiment to investigate the influence of the the cutoff frequency for the low-pass filter, the sample selection frequency, and the window duration. The results are presented in The second experiment is performed to validate that the proposed methods can be used to obtain accurate elbow joint angles for functional motions without relying on a precisely known sensor attachment and without relying on the magnetic field. An optical motion capture system (Vicon Motion Systems Ltd. UK) is used as reference. In addition to the two inertial sensors positioned as in the previous experiment, optical markers are placed on bony landmarks in a way that facilitates joint angle measurement as recommended by the ISB . Note thpick-and-place motion, the subject placed a small box in a sequence of predefined orientations and locations on a table.During the drinking motion consists of the subject repeatedly placing the hand on a table, grabbing a cup, simulating a drinking motion, and then placing the cup back on the table.The Two healthy adult volunteers, with no history of upper-limb injury that might affect upper-limb movement, performed two motions:Each of the two subjects repeats the two motions four times (twice slow and twice fast), resulting in a total of 16 trials, with durations between 14 and 44 For each trial, calculate four different joint angles.ground truth angles are derived from the optical markers placed on anatomical landmarks and calculated as described in [The OMC-based ribed in .Conventional IMU-based joint angles are calculated using 9D sensor fusion , i.e., proposed IMU-based joint angles use 6D sensor fusion , and throtation-based joint constraint and theorientation-based joint constraint.Note that the application of the proposed methods tests the most challenging case, i.e., we use a standard everyday motion to identify both the joint axes and the heading offset without requiring a separate calibration phase.To determine the sign and the required offset for the joint angles, we use the OMC-based angles. The IMU-based joint angles obtained by the different methods are compared to the OMC-based ground truth, and the RMSE is calculated. Results from all trials are shown in When comparing the two variants of the proposed method, we see that the rotation-based constraint outperforms the orientation-based constraint. This coincides with the results of the first experiment presented in However, when considering the results obtained with the proposed method and the rotation-based constraint, the accuracy is similar to the conventional 9D IMU-based method. For the FE angles, the mean RMSE of 2.1° is 0.2° lower than for the conventional method, while for the PS angles, the mean RMSE of 3.7° is 0.1° larger. In contrast to the results with the orientation-based constraint, there are no outliers, and the maximum RMSE of the proposed method and the conventional method is comparable. Note that the conventional method relies on properly calibrated magnetometer measurements, a controlled environment without ferromagnetic material or electric devices, and a precise and known sensor attachment and is therefore much more restrictive than the proposed magnetometer-free plug-and-play method.To illustrate the performed motions and the obtained results, Note that the joint constraint is only used for identifying the joint axes and that the joint angle calculation uses standard Euler angles and therefore not directly restricted by this constraint. As a result, the obtained carrying angles, which are also shown in We can use the carrying angle as an indicator of how well the measured joint motion adheres to the 2-DoF joint model . For a pTherefore, we calculate the standard deviation of the carrying angle as a measure of variability, which is shown in In summary, the evaluation of the second experiment has shown that for the challenging case of using recordings of everyday motions for calibration, the proposed methods are able to obtain joint angles with the same accuracy as a conventional IMU-based approach, while not relying on precise sensor placement or magnetometer measurements. As also shown via the first experiment, the rotation-based constraint performs better than the orientation-based constraint and should therefore be used for anatomical calibration.The present contribution introduced methods for automatic anatomical calibration for 2-DoF joints, such as the elbow, that do not require the subject to perform precise calibration movements but instead work on arbitrary motions by exploiting one of two kinematic constraints: a rotation-based constraint for the angular rates and an orientation-based constraint. The methods do not make use of magnetometer measurements. Instead, the heading offset is simultaneously estimated via the kinematic constraint, which facilitates plug-and-play magnetometer-free joint angle estimation.The proposed methods were evaluated using two experiments. The first experiment, without OMC ground truth, showed that the proposed methods yield consistent and plausible joint axis estimates from only ten seconds of motion data. The second experiment, performed with OMC as ground truth, showed that the proposed plug-and-play method can estimate accurate joint angles while being much less restrictive than a conventional IMU-based approach. In both experiments, the rotation-based joint constraint performed better than the orientation-based joint constraint.The proposed methods overcome mounting and calibration restrictions and facilitate magnetometer-free motion tracking. Therefore, they are highly suitable for indoor environments and improve the practical usability of IMU-based motion tracking in many clinical and biomedical applications.To further advance the proposed methods, it should be evaluated if combining the rotation-based and the orientation-based constraint can increase the robustness and consistency of the joint axes estimates. Furthermore, introducing and evaluating metrics to quantify the estimation uncertainty and methods for automatic (re-)triggering of the calibration when suitable motions are detected are important next steps to increase the usability of the method.


This study attempts to address the issue that present cross-modal image synthesis algorithms do not capture the spatial and structural information of human tissues effectively. As a consequence, the resulting photos include flaws including fuzzy edges and a poor signal-to-noise ratio. The authors offer a cross-sectional technique that combines residual modules with generative adversarial networks. The approach incorporates an enhanced residual initial module and attention mechanism into the generator network, reducing the number of parameters and improving the generator's feature learning capabilities. To boost discriminant performance, the discriminator employs a multiscale discriminator. A multilevel structural similarity loss is included in the loss function to improve picture contrast preservation. On the ADNI data set, the algorithm is compared to the mainstream algorithms. The experimental findings reveal that the synthetic PET image's MAE index has dropped while the SSIM and PSNR indexes have improved. The experimental findings suggest that the proposed model may maintain picture structural information while improving image quality in both visual and objective measures. The residue initial module and attention mechanism are employed to increase the generator's capacity for learning, while the multiscale discriminator is utilized to improve the model's discriminative performance. The enhanced method in this study can maintain the structure and contrast information of the picture, according to comparative experimental findings using the ADNI dataset. The produced picture is hence more aesthetically similar to the genuine print. With the development of science and technology, there are various ways of acquiring medical images, and different modalities of medical images have distinct advantages and disadvantages. For example, magnetic resonance imaging (MRI) has no radiation on the human body, the soft tissue structure is displayed, and rich diagnostic information can be obtained, but the acquisition time is extended, which is prone to artifacts; positron emission computed tomography (positron emission tomography (PET)) can make an early diagnosis of the disease through the functional changes of the tissue in the diseased area, but it is expensive, and the image resolution is low. Furthermore, studies have shown that the morphological or functional abnormalities of the human body caused by diseases are often manifested in various aspects. Therefore, the information obtained by a single modality imaging device usually cannot fully reflect the complex characteristics of the disease . Still, The majority of medical picture cross-modality synthesis techniques are based on deep learning, which may be classified as cross-modality synthesis methods based on paired data or cross-modality synthesis methods based on unpaired data depending on the kind of data employed. This work investigates cross-modal synthesis approaches based on paired data since cross-modal synthesis based on unpaired data cannot provide subject-specific visuals. PET scans (positron emission tomography scans) are frequently performed in combination with CT scans or MRI scans (magnetic resonance imaging scans). While CT and MRI scans provide images of the inside organs and tissues of your body, PET scans can provide your doctor with a glimpse of complicated systemic disorders because they highlight cellular concerns. PET scans employ positrons as opposed to MRIs. Your body is given an injection of a tracer that enables the radiologist to view the region being scanned. While PET scans are used to examine your body's function, an MRI scan may be utilized to determine the form of your organs or blood arteries. 3D CNN was utilized in the literature  to prediTo summarize, this study presents a technique for the cross-modal synthesis of PET pictures from MRI images by combining residual modules and generative adversarial networks to enhance the synthesis of subject-specific PET images with little paired data. The three primary points of the main work are as follows: the generator now includes an enhanced residual initial module and an attention mechanism to completely extract the features of MRI pictures; the pix2pix network architecture has been upgraded, and the discriminator now uses a multiscale discriminator to increase discriminate performance and reduce loss; the function incorporates a multilevel structural similarity loss based on the classic adversarial loss and L1 loss, which improves picture contrast preservation.The ability to promptly and precisely diagnose sickness and determine its severity or harmless nature is one of the potential advantages of imaging tests. It might not be essential to perform invasive diagnostic techniques such as exploratory surgery, angiography, or cardiac catheterization. Medical imaging is crucial when a person has a chronic illness or a kind of cancer, not only for the initial diagnosis but also for tracking how the illness is responding to therapy, determining whether the illness is advancing, and determining when treatment should be discontinued or changed.One of the drawbacks of medical imaging is that there is a slight increase in the likelihood that a person exposed to X-rays would acquire cancer later in life. Cataracts, skin reddening, and hair loss are all tissue consequences that occur at quite high levels of radiation exposure and are uncommon for many types of imaging tests.G and a discriminator D. The generator takes noise z from distribution as input, maps it to the data space, records the data distribution of the actual sample x, and creates a sample G(z) that looks like the original data. The produced samples and the real samples are sent to the discriminator, and the purpose is to categorize the generated samples G(z) as false and the actual samples as true. GAN is a process in which the generator and discriminator are always in conflict, playing a game of maximum and minimum values until they strike a dynamic equilibrium. GAN's goal function is as follows:The fundamental generative adversarial networks (GAN) model consists of the input vector, generator, and discriminator. The generators and discriminators are both implicit function expressions that are often employed by deep neural networks. GAN can train the predictive model of any distribution of data using adversarial approaches and get great results. GAN's primary job is to train an adversarial generator and discriminator. The objective goal is either a stronger generator or a more sensitive discriminator, depending on the project's needs. Thus, generative adversarial networks are used in CNN-based cross-modal residual networks for image synthesis. Cross-modality image estimation includes creating pictures for one form of medical imaging from those for another. It has been demonstrated that convolutional neural networks (CNNs) are effective in recognizing, classifying, and extracting picture patterns. CNNs are used as generators in generative adversarial networks (GANs), and estimated pictures are classified as true or false based on a second network. In the context of the image estimating paradigm, CNNs and GANs may be seen more broadly as deep learning techniques since imaging data frequently have a large number of network weights. The CNN/GAN image estimate literature almost exclusively uses MRI data, with PET or CT being the two main modalities. Literature  created y to the generator and discriminator as a condition to construct a conditional generative adversarial network (CGAN). The loss function of CGAN is defined asSince the results generated by the unconditional generative adversarial network have great uncertainty, literature  proposedThe pix2pix network  is a typThe structure of the initial residual block  is shownxl by gating the signal g, allowing the network to focus on areas more relevant to the task and suppress irrelevant background regions. The neural network with the added attention module has higher sensitivity and accuracy proposedWe have provided a generally and locally aware GAN framework for cross-modality transfer from MRI to PET in this research. To improve the quality of generated PET scans, the suggested multipath GAN architecture assists in simultaneously collecting both global structure and local texture. To assist the generative model in accurately learning the fundamental bimodal data distribution, the overall framework and the combined synthesis goal function were created. Experimental findings show that our methodology not only produces PET scans with higher-quality images. The model framework of the improved fusion residual module and generative adversarial network cross-modality PET image synthesis method is shown in PET and natural PET are spliced with accurate MRI to form a true and false image pair. Next, the two discriminators use the true and false image pairs as input to perform true and false discrimination. Finally, the weighted average of the two discrimination results is used as the final result.Due to its good performance and efficient use of memory, U-Net  is widelThe generator structure is shown in To better learn the pixel information in the image, the algorithm in this paper introduces the improved residual initial module into the encoding and decoding paths to ensure a better generation effect. Increasing the size of the convolution kernel in the neural network can expand the receptive field, but blindly increasing the convolution kernel will increase the network parameters and bring specific difficulties to the training of the network. Therefore, the algorithm in this paper adds a 3 × 3 convolution to the 3 × 3 convolution path of the initial residual module, replaces the larger convolution kernel with 3 small convolution kernels, and reduces the receptive field as much as possible while expanding the receptive field—network parameters. In addition, the introduction of the initial residual module can also solve the problem of gradient disappearance caused by the depth of the network.Since the structural information and spatial information of medical images are more complex than natural images, to better extract the critical structural features in MRI images, the algorithm in this paper sets the encoder-decoder path depth of the generator to 7 layers. However, considering the network complexity and memory consumption, the algorithm does not put the improved residual initial module in all convolutional layers of the encoding and decoding paths but compares the generation effect through multiple experiments and finally puts the initial residual module in the middle four layers of the network; only two 3 × 3 convolutions are used in the first 3 layers of the encoding path and the last 3 layers of the decoding path, which reduces network parameters and training time while improving the generation quality.The skip connection in U-Net can capture the contextual features from the encoding path to the decoding path. The fusion of low-level features and high-level features can retain more detailed information of high-level feature maps, but it may also contain feature information irrelevant to the synthesis task. Therefore, to improve the synthesis quality, the algorithm in this paper introduces a self-attention mechanism in the skip connection path and combines the features extracted by the decoding path before the skip connection operation. The event features through the attention gate mechanism and further eliminates the skip connection.The interference caused by irrelevant features and noise in the MRI images highlights the critical elements in the skip connections to capture the essential information of MRI images better.In addition, to prevent the network from overfitting, the algorithm also introduces a dropout operation in the generator. Finally, the synthesized PET image is obtained through the Tanh activation function after encoding and decoding the feature information.To better learn the local and global features of PET images, improve the game ability of the discriminator, and enable the generator to generate PET images that are more in line with the actual distribution, this paper adopts multiscale discriminators, namely, local discriminator and global discriminator. With two discriminators with different receptive fields (70 × 70 and 128 × 128), the generator and discriminator can learn the relationship between the spatially shorter and longer distance pixels.Based on the idea of patchGAN, the discriminator network first divides the image into N × N blocks and then discriminates whether each subblock is true or false. The two discriminator networks are 5 layers and 7 layers, respectively, composed of convolution layers, batch normalization layers, and activation layers alternately. Finally, the weighted average of all results is used as the output of the discriminator.This paper uses adversarial loss, L1 loss, and multiscale structural similarity loss (MS-SSIM) as loss functions.Adversarial loss can constrain the generated results to a certain extent, making the results closer to the actual distribution. The damaging loss is shown inThe L1 loss is passed through the generator to reduce the difference between natural and synthetic images. The L1 loss is shown inStructural similarity (SSIM) was initially described by literature  and propThe MS-SSIM loss is shown in equationLM, cj, and sj represent the brightness, contrast, and structural similarity of the image, respectively, and αM, βj, and γj represent the weights occupied by different parts, respectively.Among them, The final loss function of the model is as follows:λi is the weight coefficient of each loss.Among them, The experiments in this article are run using the PyTorch framework, with an Intel i7-6700 CPU and an NVIDIA GeForce GTX1080Ti GPU as the hardware setup. The software environment consists of Ubuntu 16.04, CUDA 9.0, cuDNN 7.6, PyTorch 1.1.0, and Python 3.7.The Alzheimer's disease neuroimaging initiative (ADNI) public dataset  was utilBefore training, data is preprocessed. The FSL software  is utiliThe weight coefficients of the loss function are changed throughout the network training phase. Because the input image's pixel range is , the resultant L1 loss is much less. When the MS-SSIM loss coefficient is more important, the brightness of the synthesized picture is higher, and when the volume is lower, it has a less significant influence on the outcome. As a result, the weight of each loss function is eventually adjusted to 1 = 1, 2 = 100, and 3 = 1 after numerous testing and troubleshooting. Furthermore, the batch size is set to 16, the initial learning rate is set at 0.0002, the network is optimized using the Adam optimizer, and 200 epochs are iteratively trained. The learning rate for the first 100 epochs remains intact, whereas the learning rate for the final 100 epochs is lowered linearly to zero.To verify the performance of the improved algorithm in this paper, this paper conducts experiments on the ADNI dataset. The algorithm uses the pix2pix model as the benchmark model and, at the same time, compares with RIED-Net, pGAN , and GANThe qualitative comparison between the generated results of the algorithm in this paper and the generated results of other algorithms is shown in In addition, due to the different brain sizes of other subjects, there is still a specific deviation after linear registration to the standard space. As shown in the second and third rows of In this paper, mean absolute error (MAE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) are used as evaluation indicators. MAE  and PSNRImax represents the maximum value of the image color, which is expressed as 255 using 8-bit sampling points.Among them, MSE is the mean square error of the two images, and SSIM  is calcuμx, μy, σx2, σy2, and σxy are the mean, variance, and covariance of the pictures x and y, respectively; c1 = (k1L)2 and c2 = (k2L)2 are two constants that prevent the denominator from being 0. Among them, As a consequence, the method in this study may improve the quality of the synthesized image and increase the edge synthesis impact of the picture by integrating the qualitative and quantitative findings of the experiment.Aiming at the problems of blurred edges and low signal-to-noise ratio of synthetic results in cross-modal synthesis tasks of medical images, this paper proposes a cross-modal PET image synthesis method that fuses initial residual modules and generative adversarial networks. The authors provide a cross-sectional method that combines generative adversarial networks with residual modules. The method reduces the number of parameters and enhances the generator's capacity for feature learning by incorporating an improved residual initial module and attention mechanism. The discriminator uses a multiscale discriminator to improve discriminant performance. To better preserve visual contrast, a multilevel structural similarity loss is incorporated into the loss function. The algorithm is contrasted with the common algorithms using the ADNI data set. The experimental results show that the MAE index of the synthetic PET picture has decreased while the SSIM and PSNR indexes have increased. The experimental results imply that the suggested approach could preserve picture structural information while enhancing image quality in both subjective and visual metrics. By introducing improvements in the generator, the residual initial module and attention mechanism are used to improve the learning ability of the generator, and the multiscale discriminator is used to enhance the discriminative performance of the model. The comparative experimental results under the ADNI dataset show that the improved algorithm in this paper can preserve the image's structural information and contrast information. As a result, the generated image is visually closer to the actual print. However, there are still some shortcomings in this paper. For example, the medical images collected by instruments with different parameters have certain deviations. Furthermore, this paper uses the same preprocessing steps to process all data. Therefore, the data collection method and preprocessing method will have a particular impact on the experimental results. In addition, in the cross-modal PET composite image experiment, only the axial slice of the image is taken for the investigation, which cannot fully reveal the three-dimensional structural information of the brain. Therefore, different preprocessing methods and cross-modal synthesis methods for 3D PET images will be investigated next.Beyond the use of machine learning in medical imaging, we think the focus in the medical community may also be used to develop the overall computational attitude among healthcare practitioners and researchers, mainstreaming the discipline of computational medicine. The acceptance for further such systems will probably increase until there is enough high-impact application software based on mathematics, computer science, physics, and engineering entering the everyday workflow in the clinic. A new medical paradigm known as P4 medicine will most likely be made possible by the availability of biosensors and (edge) computing on wearable devices for monitoring illness or lifestyle, as well as an ecosystem of machine learning and other computing medicine-based technologies.