In this paper, a novel deep learning dataset, called Air2Land, is presented for advancing the state-of-the-art object detection and pose estimation in the context of one fixed-wing unmanned aerial vehicle autolanding scenarios. It bridges vision and control for ground-based vision guidance systems having the multi-modal data obtained by diverse sensors and pushes forward the development of computer vision and autopilot algorithms targeted at visually assisted landing of one fixed-wing vehicle. The dataset is composed of sequential stereo images and synchronised sensor data, in terms of the flying vehicle pose and Pan-Tilt Unit angles, simulated in various climate conditions and landing scenarios. Since real-world automated landing data is very limited, the proposed dataset provides the necessary foundation for vision-based tasks such as flying vehicle detection, key point localisation, pose estimation etc. Hereafter, in addition to providing plentiful and scene-rich data, the developed dataset covers high-risk scenarios that are hardly accessible in reality. The dataset is also open and available at https://github.com/micros-uav/micros_air2land as well. The cover image is based on the Research Article Air2Land: A deep learning dataset for unmanned aerial vehicle autolanding from air to land by Tianjiang Hu et al., https://doi.org/10.1049/csy2.12045.

In this work, a new active fault tolerant control (FTC) is developed for an unmanned bicycle robot based on an integration between a sliding mode control (SMC), fault detection (FD), and fault estimation (FE) via a residual signal. A sliding surface in accordance with the fault tolerant sliding mode control (FTSMC) is designed for the bicycle robot to get multiple exciting features such as fast transient response with finite time convergence, small overshoot and quick stabilisation in the presence of an actuator fault. To obtain an effective performance for the FTSMC, a fault estimation system is employed and in order to attain estimation, an extended Kalman filter (EKF) as an estimator and a change detection algorithm called cumulative sum (CUSUM) as a residual evaluation function are developed. The innovative features of the proposed approach, that is FTSMC, are verified when compared with the other up-to-date control techniques like fault tolerant model-based predictive control with feedback linearisation (FTMPC + FBL) and fault tolerant linear quadratic regulator with feedback linearisation (FTLQR + FBL) on an unmanned bicycle robot.

Real time path planning from image information is of vital importance in the fields of robots as it has various applications in real time navigation, autonomous driving, robot arm manipulation and human robot cooperation and so on. To achieve this task, a two stage approach is proposed. At the first stage, a novel change point detection approach is proposed to process the image to extract the shape of the obstacles. At the second stage, several novel approximations are adopted to make the path planning problem tractable. Firstly, the irregular shapes of the obstacles in the environment are approximated as lines and circles, which simplify the distance constraint significantly. Secondly, the non-convex path planning problem is iteratively decomposed as a sequence of subproblems and alternating minimisation method is proposed to efficiently solve the subproblem. To improve the quality of the solution, good initial points obtained by A* algorithm is provided. Both numerical experiments and real experiments are conducted to demonstrate the effectiveness of the proposed algorithm.

A method is proposed for real-time vision-based localization in the 360° area around a three-dimensional (3D) reference object with a single camera. The problem is split into three subproblems. First, 360° 3D object recognition is proposed, in which a computer vision solution can recognize a reference object from all possible 360° locations. Second, 360° pose estimation is presented, in which the pose of a robot at all locations is estimated. Third, a 360° localization application is integrated with a closed-loop real-time trajectory tracking controller. The successful results of simulations and real experiments of trajectory tracking are also presented.

In recent years, formation control has received a great deal of attention as one of the most interesting issues in multiple mobile robots systems. In variable formation control systems, multiple mobile vehicles can form an appropriate formation from a list of possible formations, such that all mobile vehicles can pass even when the width of the course is narrow, and simultaneously the vehicles maintain a set distance between the course and each vehicle. This paper presents a predictive-based automatic formation control. In our formation control system, a unique formation is determined from feasible formations at any given sampling point. This control is formulated by its corresponding mixed-integer quadratic programing by introducing binary variables that are used to specify the formation. The effectiveness of the proposed methods is verified by applying two-wheeled mobile vehicles through simulations and experiments.

Model-based gait recognition with skeleton data input has attracted more attention in recent years. The model-based gait recognition methods take skeletons constructed by body joints as input, which are invariant to changing carrying and clothing conditions. However, previous methods limitedly model the skeleton information in either spatial or temporal domains and ignore the pose variety under different view angles, which results in poor performance for gait recognition. To solve the above problems, we propose the Multi-Branch Angle Aware Spatial Temporal Graph Convolutional Neural Network to better depict the spatial-temporal relationship while minimising the interference from the view angles. The model adopts the legacy Spatial Temporal Graph Neural Network (ST-GCN) as its backbone and relocates it to create independent ST-GCN branches. The novel Angle Estimator module is designed to predict the skeletons' view angles, which enables the network robust to the changing views. To balance the weights of different body parts and sequence frames, we build a Part-Frame-Importance module to redistribute them. Our experiments on the challenging CASIA-B dataset have proved the efficacy of the proposed method, which achieves state-of-the-art performance under different carrying and clothing conditions.

Simultaneous localization and mapping methods are fundamental to many robotic applications. In dynamic environments, SLAM methods focus on eliminating the influence of moving objects to construct a static map since they assume a static world. To improve localization robustness in dynamic environments, an RGB-D SLAM method is proposed to build a complete 3D map containing both static and dynamic maps, the latter of which consists of the trajectories and points of the moving objects. Without any prior knowledge of the moving targets, the proposed method uses only the correlation between map points to discriminate between the static scene and the moving objects. After the static points are determined, camera motion estimation is performed only on reliable static map points to eliminate the influence of moving objects. The motion of the moving objects will then be estimated with the obtained camera motion by tracking their dynamic points in subsequent frames. Moreover, multiple groups of dynamic points that belong to the same moving object are fused by a volume overlap checking step. Experimental results are presented to demonstrate the feasibility and performance of the proposed method.

This study considers a control problem related to the regulation of fractional-order systems controlled by fractional order proportional-integral-derivative controllers (PIλDμ). The stability result of PIλDμ-based control systems is provided, such that the closed-loop systems can accomplish global stabilisation under some suitable conditions related to the system uncertainties. Finally, a simulation is demonstrated to verify the results.

Artificial intelligence is penetrating various fields. The demand for interdisciplinary talent is increasingly important, while interdisciplinary educational activities for high school students are lagging behind. Project-based learning (PBL) in artificial intelligence (AI) and robotic education activities supported by a robotic sailboat platform, the sailboat test arena (STAr), has been shown to popularise AI and robotic knowledge in young students. In the implementation of the programme, PBL was provided for students, and gamification pedagogy was applied to increase participants' learning motivation and engagement. The results show that the proposed STAr-based programme is capable of delivering the desired knowledge and skills to students at high school levels. The assessment results suggest that most students achieve learning outcomes on average. Students showed more interest in AI and marine disciplines and were willing to participate in more such educational programs. The findings fill the research gap that few existing education platforms have facilitated the teaching and learning of AI and marine disciplines for high school students.

In recent years, several progressive studies promote the development of aerial tracking. One of the representative studies is our previous work Fast-Tracker which is applicable to various challenging tracking scenarios. However, it suffers from two main drawbacks: (1) the oversimplification in target detection by using artificial markers and (2) the contradiction between simultaneous target and environment perception with limited onboard vision. In this study, we upgrade the target detection in Fast-Tracker to detect and localise a human target based on deep learning and non-linear regression to solve the former problem. For the latter one, we equip the quadrotor system with 360° active vision on a customised gimbal camera. Furthermore, we improve the tracking trajectory planning in Fast-Tracker by incorporating an occlusion-aware mechanism that generates observable tracking trajectories. Comprehensive real-world tests confirm the proposed system's robustness and real-time capability. Benchmark comparisons with Fast-Tracker validate that the proposed system presents better tracking performance even when performing more difficult tracking tasks. The cover image is based on the Original Article Fast-Tracker 2.0: Improving autonomy of aerial tracking with active vision and human location regression by Can Cui et al., https://doi.org/10.1049/csy2.12033.

A fundamental task in robotics is to plan collision-free motions among a set of obstacles. Recently, learning-based motion-planning methods have shown significant advantages in solving different planning problems in high-dimensional spaces and complex environments. This article serves as a survey of various different learning-based methods that have been applied to robot motion-planning problems, including supervised, unsupervised learning, and reinforcement learning. These learning-based methods either rely on a human-crafted reward function for specific tasks or learn from successful planning experiences. The classical definition and learning-related definition of motion-planning problem are provided in this article. Different learning-based motion-planning algorithms are introduced, and the combination of classical motion-planning and learning techniques is discussed in detail.

Preterm births have been seen to have psychological and financial implications; current surveys suggest that amongst the various methods of preterm prediction, there is yet to exist a reliable and standard means of predicting preterm births. This study investigates the application of electrohysterogram and tocogram signals acquired at various points during the third pregnancy trimester, alongside information from the patients' medical health record regarding the pregnancy, towards preterm prediction and an associated delivery imminency timeline. In addition to this, the impact of both linear and non-linear dimensional embedding methods towards the preterm prediction is explored. The classification exercises were carried out using a support vector machine and decision tree, both of which have a certain degree of model interpretability and have potential to be introduced into a clinical operating framework.

YOLO (You Only Look Once), as a target detection algorithm with good speed and precision, is widely used in the industry. In the process of driving, the vehicle image captured by the driving camera is detected and it extracts the license plate and the front part of the vehicle. Compared with the network structure of YOLOv3-tiny algorithm, the acquisition method of anchor box is improved by combining the Birch algorithm. In order to improve the real-time performance, the original two-scale detection is added to the multi-scale prediction of three-scale detection to ensure its accuracy. Finally, the experimental results show that the improved YOLOv3-tiny network structure proposed in this study can improve the performance of mean-average-precision, intersection over union and speed by 5.99%, 17.52% and 48.4%, respectively, and the algorithm has certain robustness.

The expected penetration of renewable sources is driving the islanded microgrid towards uncertainties, which have highly influence the reliability and complexities of frequency control. To alleviate the influence caused by load fluctuations and inherent variability of renewable sources, this article proposes an optimised robust proportional-integral-derivation (PID) frequency control method by taking full advantage of a robust control strategy while simultaneously maintaining the basic characteristics of a PID controller. During the process of iterated optimisation, a weighted objective function is used to balance the tracking error performance, robust stability and disturbance attenuation performance. Then, the robust PID frequency (RPIDF) controller is determined by an adaptive constrained population extremal optimisation algorithm based on self-adaptive penalty constraint-handling technique. The proposed control method is examined on a typical islanded microgrid, and the control performance is evaluated under various disturbances and parametric uncertainties. Finally, the simulation results indicate that the fitness value of the proposed method is 1.7872, which is lower than 2.9585 and 3.0887 obtained by two other evolutionary algorithms-based RPIDF controllers. Moreover, the comprehensive simulation results fully demonstrate that the proposed method is superior to other comparison methods in terms of four performance indices on the most considered scenarios.

Image matching is a well-studied problem in computer vision. Conventional image matching is solved using image feature matching algorithms, and later deep learning techniques are also applied to tackle the problem. Here, a slope-modelling framework is proposed by adopting the image matching techniques. First, image pairs of a slope scene are captured and camera calibration as well as image rectification are performed. Then, PatchMatch Filter (PMF-S) and PWC-Net techniques are adapted to solve the matching of image pairs. In the proposed PatchMatch Filter-Census (PMF-Census), slanted-plane modelling, image census transform and gradient difference are employed in matching cost formulation. Later, nine matching points are manually selected from an image pair. Matching point pairs are further used in fitting a transformation matrix to relate the matching between the image pair. Then, the transformation matrix is applied to obtain a ground truth matching image for algorithm evaluation. The challenges in this matching problem are that the slope is of a homogenous region and it has a slanted-surface geometric structure. In this work, it is found out that the error rate of the proposed PMF-Census is significantly lower as compared with the PWC-Net method and is more suitable in this slope-modelling task. In addition, to show the robustness of the proposed PMF-Census against the original PMF-S, further experiments on some image pairs from Middlebury Stereo 2006 dataset are conducted. It is demonstrated that the error percentage by the proposed PMF-Census is reduced significantly especially in the low-texture and photometric distorted region, in comparison to the original PMF-S algorithm. This further verifies the suitability of the PMF-Census in modelling the outdoor low-texture slope scene.

The upper limb rehabilitation robot technology integrates rehabilitation medicine, human anatomy, mechanics, computer science, robotics, and many other disciplines. Its main function is to drive the affected limb to carry out rehabilitation training to restore the condition of patients with upper limb dyskinesia, which plays a great role in improving the quality of life. In this study, to resolve the problems of slow convergence speed and poor tracking accuracy due to the interference of patient spasms with the trajectory-tracking control of the upper limb rehabilitation robot, a novel algorithm based on active disturbance rejection control (ADRC) is adopted, and the convergence of its main structure is proved by the time-domain analysis method. First, this ADRC algorithm can obtain better trajectory-tracking performance due to its non-linear extended observer and non-linear feedback mechanism, even if the model suffers a strong disturbance or receives inaccurate information. Second, the non-linear tracking differentiator can guarantee to gain quick convergence speed. To validate this algorithm, a model of three degrees of freedom upper limb rehabilitation robot is established using MATLAB R2019b and three situations including strong spasm and weak spasm are carried out to prove the effectiveness and reliability of the control algorithm designed.

Without the dependence of depth ground truth, self-supervised learning is a promising alternative to train monocular depth estimation. It builds its own supervision signal with the help of other tools, such as view synthesis and pose networks. However, more training parameters and time consumption may be involved. This paper proposes a monocular depth prediction framework that can jointly learn the depth value and pose transformation between images in an end-to-end manner. The depth network creatively employs an asymmetric convolution block instead of every square kernel layer to strengthen the learning ability of extracting image features when training. During inference time, the asymmetric kernels are fused and converted to the original network to predict more accurate image depth, thus bringing no extra computations anymore. The network is trained and tested on the KITTI monocular dataset. The evaluated results demonstrate that the depth model outperforms some State of the Arts (SOTA) approaches and can reduce the inference time of depth prediction. Additionally, the proposed model performs great adaptability on the Make3D dataset.

Flood prevention and disaster mitigation have a great impact on people's lives and properties, and so it is urgent to realise high-accuracy inflow predictions for flood early warning. To this end, a prediction model based on a machine learning algorithm via a multimodel combination method is proposed to predict the inflow of Jinshuitan reservoir. Firstly, a data formatting scheme called the ’hydrological regime profile‘ is designed for input data. The whole data set is partitioned into a low-flow subset and a high-flow subset. Considering the high dimensions of the complex input data, convolutional neural networks (CNN), EXtreme gradient Boosting model (XGBoost) and a partial least squares model (PLS) are used. In the CNN and XGBoost models, a special loss function weighted on inflow is designed to improve the performance on high-inflow predictions. Finally, a multi-model combination method is proposed to improve the prediction performance. Compared with XGBoost, CNN and PLS, the root mean square error of the combined model is reduced by 41.64%, 72.29% and 3.41%, respectively. As a consequence, the combined model is able to predict the inflows with higher accuracy compared to the single models.

Shared control of mobile robots integrates manual input with auxiliary autonomous controllers to improve the overall system performance. However, prior work that seeks to find the optimal shared control ratio needs an accurate human model, which is usually challenging to obtain. In this study, the authors develop an extended Twin Delayed Deep Deterministic Policy Gradient (DDPG) (TD3X)-based shared control framework that learns to assist a human operator in teleoperating mobile robots optimally. The robot's states, shared control ratio in the previous time step, and human's control input is used as inputs to the reinforcement learning (RL) agent, which then outputs the optimal shared control ratio between human input and autonomous controllers without knowing the human model. Noisy softmax policies are developed to make the TD3X algorithm feasible under the constraint of a shared control ratio. Furthermore, to accelerate the training process and protect the robot, a navigation demonstration policy and a safety guard are developed. A neural network (NN) structure is developed to maintain the correlation of sensor readings among heterogeneous input data and improve the learning speed. In addition, an extended DAGGER (DAGGERX) human agent is developed for training the RL agent to reduce human workload. Robot simulations and experiments with humans in the loop are conducted. The results show that the DAGGERX human agent can simulate real human inputs in the worst-case scenarios with a mean square error of 0.0039. Compared to the original TD3 agent, the TD3X-based shared control system decreased the average collision number from 387.3 to 44.4 in a simplistic environment and 394.2 to 171.2 in a more complex environment. The maximum average return increased from 1043 to 1187 with a faster converge speed in the simplistic environment, while the performance is equally good in the complex environment because of the use of an advanced human agent. In the human subject tests, participants' average perceived workload was significantly lower in shared control than that in exclusively manual control (26.90 vs. 40.07, p = 0.013).

With a wide range of applications in autonomous driving and robotics, semantic segmentation for large-scale outdoor point clouds is a critical and challenging issue. Due to the large number and irregular arrangement of point clouds, it is difficult to balance the efficiency and effectiveness. In this paper, we propose LessNet, a lightweight and efficient voxel-based method for LiDAR-only semantic segmentation, taking advantage of cylindrical partition and intra-voxel feature fusion. Specifically, we use a cylindrical partition method to distribute the outdoor point clouds more evenly in voxels. To better encode the voxel features, we adopt an intra-voxel aggregation method without querying neighbours. The voxel features are further input into a lightweight and effective 3D U-net to aggregate local features and dilate the receptive field. Extensive experiments prove the satisfied semantic segmentation performance and the improvement of each component in our proposed framework. Our method is capable of processing more than one million point clouds at a time while retaining low latency and few parameters. Moreover, our method achieves comparable performance with state-of-the-art approaches and outperforms all projection-based methods on the SemanticKITTI benchmark.

Driving and tilt-hovering help unmanned aerial vehicles extend their time of operation and the visible range of the equipped sensors. Standard quadrotors are underactuated, whose translational and rotational motions are strongly coupled. As a result, it is impossible to maintain the attitude invariably while changing position. Meanwhile, larger flying speed results in greater air resistance, which is also not desirable. A novel aerial platform with tiltable rotors is proposed. Compared with standard quadrotors, the aerial vehicle has three special capabilities: vector-flying, tilt-hovering, and driving. The platform consists of two tilting-axes to change the direction of thrusts with respect to the aircraft body. To guarantee flexible manoeuvres, a geometric tracking control scheme that adapts on the fly to the thrust vectoring is exploited. Manoeuvrability of the proposed vehicle in different manoeuvres is demonstrated, such as flight with vectoring thrust that can reduce the flight resistance, tilt-hovering expanding the reachable set of the equipped sensors. The prototype equipped with an electromechanical controller is constructed, and several associated preliminary experiments are performed. The feasibility of the mechanical design and the geometric control strategy are demonstrated.

The authors analyse a Bayesian framework for posing and solving inverse reinforcement learning (IRL) problems that arise in decision-making and optimisation settings. The authors propose a non-parametric Bayesian model using Gaussian process (GP) and preference graphs, which offer an effective and computationally efficient method for ill-posed IRL problems in large or infinite state space. This approach only requires a finite number of demonstrations that is much less than the amount required for approximating the feature expectation or value functions in previous IRL methods. The proposed learning framework is expressive as it relies on a Bayesian structure in which assumptions are explicit and changeable. It is also robust in that it formalises on convex optimisation, which retains the promise of computationally manageable implementations for practical problems. To deal with more realistic IRL problems where the dynamics is also unknown, the GP model can be easily combined with the method to learn the dynamics at the same time. Experimental results prove the superiority of the authors method to current prevailing IRL algorithms with fewer numbers of demonstrations in both discrete and continuous state space.

Software validation aims to ensure that a particular software product fulfils its intended purpose, and needs to be performed against both software requirement as well as its implementation (i.e. product). However, for diagnostic software (i.e. a diagnoser) performing online diagnosis against certain fault models and reports diagnosis decision, the underlying fault models are usually not explicitly specified, neither by formal language nor by descriptive language. The lack of formal representation of fault models leaves the intended purpose of the diagnostic software vague, making its validation difficult. To address this issue, the authors propose various model-based techniques that can generate concrete examples of the diagnoser's key properties. Such examples are represented in an intuitive and possibly visualised way, facilitating the designers/users to approve or disapprove the conformance of the diagnoser to the intended purpose. The proposed techniques work for validation of both the requirement and implementation that can be modelled as finite state machine, and are illustrated through applications on vehicle on-board diagnostic requirement.

This article focuses on H ∞ containment control and the communication network topologies that are driven by a semi-Markov chain. Moreover, the communication channels between agents exist time-varying delays and noise. Firstly, the authors extend the Markov switching topologies to semi-Markov switching topologies. Because the transition rate of the semi-Markov switching topology is time-varying and depends on the sojourn time, the analysis of containment control under semi-Markov switching topology becomes more challenging. Secondly, a control protocol with time-varying delays is adopted. The error function is derived by the property of graph theory, convex hull and communication noise. Hence, the problem of H ∞ containment control is transformed into the stability problem of the semi-Markov jump system. To avoid the zero initial condition in the traditional H ∞ control approach, a novel performance function is constructed with the initial condition considered. Finally, simulation experiments are provided to verify the effectiveness of the proposed algorithm.

Novelty detection is concerned with detecting features that do not belong to any known class or are not well represented by existing models. Ergo, in autonomous navigation novelty detection determines whether an input camera frame contains certain entities of high interest which do not correspond to a known category. One of the key requirements for the future space exploration missions is the reduction of the information to be transferred back to Earth. Thus, novelty detection techniques have been developed to select the subset of acquired images with significant measurements that justify utilisation of the limited bandwidth from the available information link. Such methods are based on the identification of salient regions, which are then evaluated against a set of trained classifiers. We explore a novelty detection approach, based on the reasoning properties of Neural Networks, which follow the same guidelines while also being trainable in an end-to-end manner. This characteristic allows for the intertwined optimisation of the individual components leading to a closer estimation of a global solution. Our experiments reveal that the proposed novelty detection system achieves better performance, as compared to hand-crafted techniques, when the learning and testing examples refer to similar environments.

This study addresses the problem of deploying a group of mobile robots over a non-convex region with obstacles. Assuming that the robots are equipped with omnidirectional range sensors of common radius, disjoint subsets of the sensed area are assigned to the robots. These proximity-based subsets are calculated using the visibility notion, where the cell of each robot is treated as an opaque obstacle for the other robots. Based on that, optimal spatially distributed coordination algorithms are derived for the area coverage problem and for the homing problem, where the swarm needs to move to specific locations. Experimental studies demonstrate the results.

This is the IET Cyber-systems and Robotics special issue of Autonomous systems: Navigation, learning, and control. Autonomous systems, as very important representatives of Artificial Intelligence technologies, combine mechanical and electronic hardware, operating system, low-level dynamic control, and high-level intelligent decision components to address challenges that demand high-level autonomy and machine intelligence. Autonomous systems such as aerial robotics, ground vehicles, unmanned surface vehicles, and even all-terrain vehicles for aerospace applications, etc., have played an essential role in many aspects. For example, during the COVID-19 pandemic, autonomous systems have been used for disinfection and food delivery to reduce infection risks. However, navigation, learning, and control are critical for realizing true autonomy, which are still left for research. Navigation is a critical technology that works as the high-level intelligent decision component and directly determines the efficiency of conducting autonomous tasks. Learning, which is regarded as the most representative technology of artificial intelligence, has become a dominating method in many fields, such as image processing and understanding. There are also trends of adopting learning-based methods on autonomous robots, which are the perfect field to adopt state-of-the-art artificial intelligence technologies. The low-level dynamic control part, which builds the foundation of autonomous tasks, receives long-term research interest from modern control theory to intelligent control theory.

Some systems, in spite of having multiple outputs, have only one control input, which makes their control a challenge. Two novel controllers are proposed that utilise an adaptive finite-time sliding mode control (AFSMC) scheme for a class of single-input multiple-output (SIMO) nonlinear systems in the presence of unknown mismatched uncertainties. To alleviate the inherent chattering phenomenon of sliding mode control, new forms of the two designed controllers are suggested by using new sliding surfaces. Not only can the proposed AFSMC scheme stabilise the system in a finite time, but also it can provide estimated data of the uncertainty upper bound in the controller. Lyapunov stability theory is used to obtain finite-time stability analysis of the closed-loop system. Finally, simulation results are carried out in Simulink/MATLAB for a four-dimensional autonomous hyper-chaotic system with mismatched uncertainties as an example of SIMO uncertain nonlinear systems to reveal the effectiveness of the proposed controllers.